2023-10-16T14:04:15.762 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T14:04:15.763 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046 --owner scheduled_rishabh@teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} --verbose -- /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/orig.config.yaml 2023-10-16T14:04:15.820 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T14:04:15.913 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046 branch: rishabh-2023oct13 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} email: null first_in_suite: false job_id: '7429046' kernel: client: branch: testing flavor: default kdb: true sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-2023oct13 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-2023oct13-b2 sha1: 33be745ec4531df08cd1bb70b4c9c1f89be58078 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6006 sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 sleep_before_teardown: 0 subset: 402/512 suite: fs suite_branch: rishabh-2023oct13-b2 suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 33be745ec4531df08cd1bb70b4c9c1f89be58078 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi186.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_12:17:50 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T14:04:15.913 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa; will attempt to use it 2023-10-16T14:04:15.914 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks 2023-10-16T14:04:15.915 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T14:04:15.917 INFO:teuthology.task.internal:Checking packages... 2023-10-16T14:04:15.941 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0' 2023-10-16T14:04:15.941 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T14:04:15.942 INFO:teuthology.packaging:ref: None 2023-10-16T14:04:15.942 INFO:teuthology.packaging:tag: None 2023-10-16T14:04:15.942 INFO:teuthology.packaging:branch: rishabh-2023oct13 2023-10-16T14:04:15.943 INFO:teuthology.packaging:sha1: 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:04:15.943 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-2023oct13 2023-10-16T14:04:16.170 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6730.g4ba8a190 2023-10-16T14:04:16.170 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T14:04:16.177 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T14:04:16.177 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T14:04:16.192 INFO:teuthology.task.internal:Saving configuration 2023-10-16T14:04:16.212 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T14:04:16.219 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T14:04:16.245 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.685532', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:04:16.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.687067', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:04:16.288 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.688296', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:04:16.288 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T14:04:16.295 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-10-16T14:04:16.295 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-10-16T14:04:16.295 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-10-16T14:04:16.296 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T14:04:16.405 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7418569af0>, signals=[15]) 2023-10-16T14:04:16.407 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T14:04:16.413 INFO:teuthology.task.internal:Opening connections... 2023-10-16T14:04:16.414 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:04:16.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:04:16.490 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T14:04:16.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:04:16.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2023-10-16T14:04:16.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:04:16.626 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T14:04:16.633 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-10-16T14:04:16.653 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-10-16T14:04:16.654 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-10-16T14:04:16.717 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-10-16T14:04:16.718 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-10-16T14:04:16.718 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-10-16T14:04:16.718 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-10-16T14:04:16.719 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-10-16T14:04:16.719 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-10-16T14:04:16.719 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-16T14:04:16.719 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:04:16.720 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-16T14:04:16.721 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-16T14:04:16.721 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-16T14:04:16.721 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-16T14:04:16.722 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-10-16T14:04:16.748 DEBUG:teuthology.orchestra.run.smithi186:> uname -m 2023-10-16T14:04:16.768 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2023-10-16T14:04:16.769 DEBUG:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2023-10-16T14:04:16.830 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="Red Hat Enterprise Linux" 2023-10-16T14:04:16.830 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="8.6 (Ootpa)" 2023-10-16T14:04:16.830 INFO:teuthology.orchestra.run.smithi186.stdout:ID="rhel" 2023-10-16T14:04:16.830 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="fedora" 2023-10-16T14:04:16.831 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="8.6" 2023-10-16T14:04:16.831 INFO:teuthology.orchestra.run.smithi186.stdout:PLATFORM_ID="platform:el8" 2023-10-16T14:04:16.831 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-16T14:04:16.831 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2023-10-16T14:04:16.831 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-16T14:04:16.832 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.redhat.com/" 2023-10-16T14:04:16.832 INFO:teuthology.orchestra.run.smithi186.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-16T14:04:16.832 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T14:04:16.832 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:04:16.832 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-16T14:04:16.833 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-16T14:04:16.833 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-16T14:04:16.833 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-16T14:04:16.833 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2023-10-16T14:04:16.856 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2023-10-16T14:04:16.877 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2023-10-16T14:04:16.878 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-10-16T14:04:16.937 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Red Hat Enterprise Linux" 2023-10-16T14:04:16.938 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8.6 (Ootpa)" 2023-10-16T14:04:16.938 INFO:teuthology.orchestra.run.smithi187.stdout:ID="rhel" 2023-10-16T14:04:16.938 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="fedora" 2023-10-16T14:04:16.938 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8.6" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.redhat.com/" 2023-10-16T14:04:16.939 INFO:teuthology.orchestra.run.smithi187.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-16T14:04:16.940 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-16T14:04:16.940 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:04:16.940 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-16T14:04:16.940 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-10-16T14:04:16.940 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-16T14:04:16.941 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-10-16T14:04:16.941 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-10-16T14:04:16.964 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T14:04:16.974 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T14:04:16.980 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T14:04:16.980 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2023-10-16T14:04:16.983 DEBUG:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2023-10-16T14:04:16.985 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2023-10-16T14:04:17.003 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T14:04:17.009 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T14:04:17.009 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2023-10-16T14:04:17.045 DEBUG:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2023-10-16T14:04:17.048 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2023-10-16T14:04:17.074 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T14:04:17.153 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T14:04:17.167 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '47672119149b103ec7cec2cc65c063ccbb001ac7'} 2023-10-16T14:04:17.167 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-10-16T14:04:17.168 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-10-16T14:04:17.168 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T14:04:17.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:04:17.188 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-16T14:04:17.461 DEBUG:teuthology.task.kernel:sha1 for client.0 is 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T14:04:17.461 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "47672119149b103ec7cec2cc65c063ccbb001ac7"... 2023-10-16T14:04:17.462 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2023-10-16T14:04:17.482 INFO:teuthology.orchestra.run.smithi178.stdout:4.18.0-372.9.1.el8.x86_64 2023-10-16T14:04:17.482 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T14:04:17.483 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-16T14:04:17.484 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum upgrade -y linux-firmware 2023-10-16T14:04:18.608 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-16T14:04:18.608 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-10-16T14:04:20.000 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-10-16T14:04:20.717 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-16T14:04:24.260 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 440 kB/s | 24 kB 00:00 2023-10-16T14:04:25.027 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-16T14:04:25.028 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-10-16T14:04:25.028 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-16T14:04:25.170 INFO:teuthology.task.kernel:Downloading kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-16T14:04:25.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=47672119149b103ec7cec2cc65c063ccbb001ac7 2023-10-16T14:04:25.436 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ 2023-10-16T14:04:25.436 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-16T14:04:25.439 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi178.front.sepia.ceph.com to complete... 2023-10-16T14:04:38.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16 14:04:38 URL:https://3.chacra.ceph.com/r/kernel/testing/47672119149b103ec7cec2cc65c063ccbb001ac7/centos/8/flavors/default/x86_64/kernel-6.6.0_rc5_g47672119149b-1.x86_64.rpm [383161052/383161052] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-16T14:04:38.961 INFO:teuthology.orchestra.run.smithi178.stderr:FINISHED --2023-10-16 14:04:38-- 2023-10-16T14:04:38.961 INFO:teuthology.orchestra.run.smithi178.stderr:Total wall clock time: 13s 2023-10-16T14:04:38.961 INFO:teuthology.orchestra.run.smithi178.stderr:Downloaded: 1 files, 365M in 13s (27.8 MB/s) 2023-10-16T14:04:38.963 INFO:teuthology.task.kernel:Installing kernel 47672119149b103ec7cec2cc65c063ccbb001ac7 on client.0... 2023-10-16T14:04:38.963 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-16T14:04:40.542 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying... ######################################## 2023-10-16T14:04:40.596 INFO:teuthology.orchestra.run.smithi178.stdout:Preparing... ######################################## 2023-10-16T14:04:40.687 INFO:teuthology.orchestra.run.smithi178.stdout:Updating / installing... 2023-10-16T14:05:19.085 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-6.6.0_rc5_g47672119149b-1 ######################################## 2023-10-16T14:05:23.542 INFO:teuthology.orchestra.run.smithi178.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-10-16T14:05:43.088 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-16T14:05:43.089 DEBUG:teuthology.orchestra.run.smithi178:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-16T14:05:43.138 INFO:teuthology.orchestra.run.smithi178.stdout:/boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.139 INFO:teuthology.orchestra.run.smithi178.stdout:/boot/config-6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.139 INFO:teuthology.orchestra.run.smithi178.stdout:/boot/vmlinuz-6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.139 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.139 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel 2023-10-16T14:05:43.140 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch 2023-10-16T14:05:43.140 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86 2023-10-16T14:05:43.140 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto 2023-10-16T14:05:43.140 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-16T14:05:43.140 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-16T14:05:43.141 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-16T14:05:43.141 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-16T14:05:43.141 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-16T14:05:43.141 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-16T14:05:43.142 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-16T14:05:43.142 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-16T14:05:43.142 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-16T14:05:43.142 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-16T14:05:43.142 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-16T14:05:43.143 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-16T14:05:43.144 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-16T14:05:43.144 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-16T14:05:43.144 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-16T14:05:43.144 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel 2023-10-16T14:05:43.144 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu 2023-10-16T14:05:43.145 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce 2023-10-16T14:05:43.145 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-16T14:05:43.145 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm 2023-10-16T14:05:43.145 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-16T14:05:43.145 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-16T14:05:43.146 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/arch/x86/kvm/kvm.ko 2023-10-16T14:05:43.146 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto 2023-10-16T14:05:43.146 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ansi_cprng.ko 2023-10-16T14:05:43.146 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/anubis.ko 2023-10-16T14:05:43.146 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/arc4.ko 2023-10-16T14:05:43.147 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx 2023-10-16T14:05:43.147 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-16T14:05:43.147 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_pq.ko 2023-10-16T14:05:43.147 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-16T14:05:43.147 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_tx.ko 2023-10-16T14:05:43.148 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/async_xor.ko 2023-10-16T14:05:43.148 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/async_tx/raid6test.ko 2023-10-16T14:05:43.148 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authenc.ko 2023-10-16T14:05:43.148 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/authencesn.ko 2023-10-16T14:05:43.149 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blake2b_generic.ko 2023-10-16T14:05:43.149 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_common.ko 2023-10-16T14:05:43.149 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/blowfish_generic.ko 2023-10-16T14:05:43.149 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/camellia_generic.ko 2023-10-16T14:05:43.149 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast5_generic.ko 2023-10-16T14:05:43.150 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast6_generic.ko 2023-10-16T14:05:43.150 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cast_common.ko 2023-10-16T14:05:43.150 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/ccm.ko 2023-10-16T14:05:43.150 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cmac.ko 2023-10-16T14:05:43.150 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crc32_generic.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_engine.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/crypto_user.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/cts.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/des_generic.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/dh_generic.ko 2023-10-16T14:05:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/echainiv.ko 2023-10-16T14:05:43.152 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/essiv.ko 2023-10-16T14:05:43.152 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/fcrypt.ko 2023-10-16T14:05:43.152 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/khazad.ko 2023-10-16T14:05:43.152 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4.ko 2023-10-16T14:05:43.152 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/lz4hc.ko 2023-10-16T14:05:43.153 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/md4.ko 2023-10-16T14:05:43.153 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/michael_mic.ko 2023-10-16T14:05:43.153 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcbc.ko 2023-10-16T14:05:43.153 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/pcrypt.ko 2023-10-16T14:05:43.153 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/rmd160.ko 2023-10-16T14:05:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/seed.ko 2023-10-16T14:05:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/serpent_generic.ko 2023-10-16T14:05:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tcrypt.ko 2023-10-16T14:05:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/tea.ko 2023-10-16T14:05:43.154 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_common.ko 2023-10-16T14:05:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/twofish_generic.ko 2023-10-16T14:05:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/vmac.ko 2023-10-16T14:05:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/wp512.ko 2023-10-16T14:05:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xcbc.ko 2023-10-16T14:05:43.155 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xor.ko 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/crypto/xxhash_generic.ko 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/acpi_pad.ko 2023-10-16T14:05:43.156 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/custom_method.ko 2023-10-16T14:05:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/ec_sys.ko 2023-10-16T14:05:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/platform_profile.ko 2023-10-16T14:05:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbs.ko 2023-10-16T14:05:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/sbshc.ko 2023-10-16T14:05:43.157 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/acpi/video.ko 2023-10-16T14:05:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata 2023-10-16T14:05:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/acard-ahci.ko 2023-10-16T14:05:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ahci_platform.ko 2023-10-16T14:05:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/ata_generic.ko 2023-10-16T14:05:43.158 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/libahci_platform.ko 2023-10-16T14:05:43.159 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_acpi.ko 2023-10-16T14:05:43.159 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ali.ko 2023-10-16T14:05:43.159 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_amd.ko 2023-10-16T14:05:43.159 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_artop.ko 2023-10-16T14:05:43.159 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atiixp.ko 2023-10-16T14:05:43.160 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_atp867x.ko 2023-10-16T14:05:43.160 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd640.ko 2023-10-16T14:05:43.160 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-16T14:05:43.160 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_cypress.ko 2023-10-16T14:05:43.160 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_efar.ko 2023-10-16T14:05:43.161 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt366.ko 2023-10-16T14:05:43.161 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-16T14:05:43.161 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-16T14:05:43.161 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-16T14:05:43.161 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it8213.ko 2023-10-16T14:05:43.162 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_it821x.ko 2023-10-16T14:05:43.162 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_jmicron.ko 2023-10-16T14:05:43.162 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_marvell.ko 2023-10-16T14:05:43.162 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_mpiix.ko 2023-10-16T14:05:43.162 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_netcell.ko 2023-10-16T14:05:43.163 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ninja32.ko 2023-10-16T14:05:43.163 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87410.ko 2023-10-16T14:05:43.163 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_ns87415.ko 2023-10-16T14:05:43.163 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-16T14:05:43.163 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_opti.ko 2023-10-16T14:05:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_optidma.ko 2023-10-16T14:05:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-16T14:05:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-16T14:05:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-16T14:05:43.164 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_piccolo.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_rdc.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sch.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_serverworks.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sil680.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sis.ko 2023-10-16T14:05:43.165 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-16T14:05:43.166 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_triflex.ko 2023-10-16T14:05:43.166 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pata_via.ko 2023-10-16T14:05:43.166 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/pdc_adma.ko 2023-10-16T14:05:43.166 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_inic162x.ko 2023-10-16T14:05:43.166 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_mv.ko 2023-10-16T14:05:43.167 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_nv.ko 2023-10-16T14:05:43.167 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_promise.ko 2023-10-16T14:05:43.167 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_qstor.ko 2023-10-16T14:05:43.167 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil.ko 2023-10-16T14:05:43.167 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sil24.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sis.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_svw.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_sx4.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_uli.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_via.ko 2023-10-16T14:05:43.168 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ata/sata_vsc.ko 2023-10-16T14:05:43.169 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay 2023-10-16T14:05:43.169 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-16T14:05:43.169 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base 2023-10-16T14:05:43.169 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap 2023-10-16T14:05:43.169 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-16T14:05:43.170 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-16T14:05:43.170 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma 2023-10-16T14:05:43.170 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/bcma/bcma.ko 2023-10-16T14:05:43.170 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block 2023-10-16T14:05:43.170 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe 2023-10-16T14:05:43.171 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/aoe/aoe.ko 2023-10-16T14:05:43.171 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/brd.ko 2023-10-16T14:05:43.171 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd 2023-10-16T14:05:43.171 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/drbd/drbd.ko 2023-10-16T14:05:43.171 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/floppy.ko 2023-10-16T14:05:43.172 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/loop.ko 2023-10-16T14:05:43.172 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx 2023-10-16T14:05:43.172 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-16T14:05:43.172 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/nbd.ko 2023-10-16T14:05:43.172 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/pktcdvd.ko 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/rbd.ko 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/virtio_blk.ko 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback 2023-10-16T14:05:43.173 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-16T14:05:43.174 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/xen-blkfront.ko 2023-10-16T14:05:43.174 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram 2023-10-16T14:05:43.174 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/block/zram/zram.ko 2023-10-16T14:05:43.174 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char 2023-10-16T14:05:43.174 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hangcheck-timer.ko 2023-10-16T14:05:43.175 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random 2023-10-16T14:05:43.175 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-16T14:05:43.175 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-16T14:05:43.175 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-16T14:05:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-16T14:05:43.177 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-16T14:05:43.177 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-16T14:05:43.177 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-16T14:05:43.177 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/lp.ko 2023-10-16T14:05:43.177 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave 2023-10-16T14:05:43.178 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/mwave/mwave.ko 2023-10-16T14:05:43.178 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/ppdev.ko 2023-10-16T14:05:43.178 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tlclk.ko 2023-10-16T14:05:43.178 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm 2023-10-16T14:05:43.178 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm.ko 2023-10-16T14:05:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-16T14:05:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-16T14:05:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-16T14:05:43.179 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/uv_mmtimer.ko 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/char/virtio_console.ko 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-16T14:05:43.180 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-16T14:05:43.181 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-16T14:05:43.181 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-16T14:05:43.181 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-16T14:05:43.181 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-16T14:05:43.181 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto 2023-10-16T14:05:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp 2023-10-16T14:05:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-16T14:05:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-16T14:05:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel 2023-10-16T14:05:43.182 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-aes.ko 2023-10-16T14:05:43.183 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/padlock-sha.ko 2023-10-16T14:05:43.184 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio 2023-10-16T14:05:43.184 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-16T14:05:43.184 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca 2023-10-16T14:05:43.184 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dca/dca.ko 2023-10-16T14:05:43.184 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq 2023-10-16T14:05:43.185 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-16T14:05:43.185 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma 2023-10-16T14:05:43.185 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw 2023-10-16T14:05:43.185 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-16T14:05:43.186 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat 2023-10-16T14:05:43.186 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-16T14:05:43.186 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac 2023-10-16T14:05:43.186 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/amd64_edac.ko 2023-10-16T14:05:43.186 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/e752x_edac.ko 2023-10-16T14:05:43.187 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-16T14:05:43.187 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3000_edac.ko 2023-10-16T14:05:43.187 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i3200_edac.ko 2023-10-16T14:05:43.187 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5100_edac.ko 2023-10-16T14:05:43.187 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i5400_edac.ko 2023-10-16T14:05:43.188 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7300_edac.ko 2023-10-16T14:05:43.188 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i7core_edac.ko 2023-10-16T14:05:43.188 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/i82975x_edac.ko 2023-10-16T14:05:43.188 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/ie31200_edac.ko 2023-10-16T14:05:43.188 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/sb_edac.ko 2023-10-16T14:05:43.189 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/edac/x38_edac.ko 2023-10-16T14:05:43.189 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware 2023-10-16T14:05:43.189 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/edd.ko 2023-10-16T14:05:43.189 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-16T14:05:43.189 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio 2023-10-16T14:05:43.190 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-ich.ko 2023-10-16T14:05:43.190 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-16T14:05:43.190 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid 2023-10-16T14:05:43.190 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-appleir.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-aureal.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-axff.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-dr.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elecom.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-elo.ko 2023-10-16T14:05:43.191 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-emsff.ko 2023-10-16T14:05:43.192 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gaff.ko 2023-10-16T14:05:43.192 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gt683r.ko 2023-10-16T14:05:43.192 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-gyration.ko 2023-10-16T14:05:43.192 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-16T14:05:43.192 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-16T14:05:43.193 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-holtekff.ko 2023-10-16T14:05:43.193 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-hyperv.ko 2023-10-16T14:05:43.193 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-icade.ko 2023-10-16T14:05:43.193 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-keytouch.ko 2023-10-16T14:05:43.193 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-kye.ko 2023-10-16T14:05:43.194 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lcpower.ko 2023-10-16T14:05:43.194 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-led.ko 2023-10-16T14:05:43.194 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-lenovo.ko 2023-10-16T14:05:43.194 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-multitouch.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-ortek.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-penmount.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-petalynx.ko 2023-10-16T14:05:43.195 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-picolcd.ko 2023-10-16T14:05:43.196 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-pl.ko 2023-10-16T14:05:43.196 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-plantronics.ko 2023-10-16T14:05:43.196 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-primax.ko 2023-10-16T14:05:43.196 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-rmi.ko 2023-10-16T14:05:43.196 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-16T14:05:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-16T14:05:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-16T14:05:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-16T14:05:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-16T14:05:43.197 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-16T14:05:43.198 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-16T14:05:43.198 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-16T14:05:43.198 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-16T14:05:43.198 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-16T14:05:43.198 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-roccat.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-saitek.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-samsung.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sjoy.ko 2023-10-16T14:05:43.199 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sony.ko 2023-10-16T14:05:43.200 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-speedlink.ko 2023-10-16T14:05:43.200 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-steelseries.ko 2023-10-16T14:05:43.200 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-sunplus.ko 2023-10-16T14:05:43.200 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-16T14:05:43.200 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tivo.ko 2023-10-16T14:05:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-tmff.ko 2023-10-16T14:05:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-topseed.ko 2023-10-16T14:05:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-twinhan.ko 2023-10-16T14:05:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-uclogic.ko 2023-10-16T14:05:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-waltop.ko 2023-10-16T14:05:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-wiimote.ko 2023-10-16T14:05:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-xinmo.ko 2023-10-16T14:05:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zpff.ko 2023-10-16T14:05:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/hid-zydacron.ko 2023-10-16T14:05:43.202 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/uhid.ko 2023-10-16T14:05:43.203 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hid/wacom.ko 2023-10-16T14:05:43.203 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv 2023-10-16T14:05:43.203 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_balloon.ko 2023-10-16T14:05:43.203 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_utils.ko 2023-10-16T14:05:43.203 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hv/hv_vmbus.ko 2023-10-16T14:05:43.204 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon 2023-10-16T14:05:43.204 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru.ko 2023-10-16T14:05:43.204 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/abituguru3.ko 2023-10-16T14:05:43.204 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-16T14:05:43.204 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7414.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ad7418.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adc128d818.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1025.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1026.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1029.ko 2023-10-16T14:05:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm1031.ko 2023-10-16T14:05:43.206 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adm9240.ko 2023-10-16T14:05:43.206 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ads7828.ko 2023-10-16T14:05:43.206 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7410.ko 2023-10-16T14:05:43.206 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7411.ko 2023-10-16T14:05:43.206 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7462.ko 2023-10-16T14:05:43.207 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7470.ko 2023-10-16T14:05:43.207 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7475.ko 2023-10-16T14:05:43.207 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/adt7x10.ko 2023-10-16T14:05:43.207 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/amc6821.ko 2023-10-16T14:05:43.207 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/applesmc.ko 2023-10-16T14:05:43.208 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asb100.ko 2023-10-16T14:05:43.208 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asc7621.ko 2023-10-16T14:05:43.208 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-16T14:05:43.208 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/atxp1.ko 2023-10-16T14:05:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/coretemp.ko 2023-10-16T14:05:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-16T14:05:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/dme1737.ko 2023-10-16T14:05:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds1621.ko 2023-10-16T14:05:43.209 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ds620.ko 2023-10-16T14:05:43.210 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc1403.ko 2023-10-16T14:05:43.210 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/emc6w201.ko 2023-10-16T14:05:43.210 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71805f.ko 2023-10-16T14:05:43.210 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f71882fg.ko 2023-10-16T14:05:43.210 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/f75375s.ko 2023-10-16T14:05:43.211 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-16T14:05:43.211 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/fschmd.ko 2023-10-16T14:05:43.211 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g760a.ko 2023-10-16T14:05:43.211 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/g762.ko 2023-10-16T14:05:43.211 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl518sm.ko 2023-10-16T14:05:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/gl520sm.ko 2023-10-16T14:05:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-16T14:05:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-16T14:05:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-16T14:05:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmaem.ko 2023-10-16T14:05:43.213 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ibmpex.ko 2023-10-16T14:05:43.213 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina209.ko 2023-10-16T14:05:43.213 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ina2xx.ko 2023-10-16T14:05:43.213 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/it87.ko 2023-10-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k10temp.ko 2023-10-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/k8temp.ko 2023-10-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm63.ko 2023-10-16T14:05:43.214 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm73.ko 2023-10-16T14:05:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm75.ko 2023-10-16T14:05:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm77.ko 2023-10-16T14:05:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm78.ko 2023-10-16T14:05:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm80.ko 2023-10-16T14:05:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm83.ko 2023-10-16T14:05:43.216 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm85.ko 2023-10-16T14:05:43.216 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm87.ko 2023-10-16T14:05:43.216 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm90.ko 2023-10-16T14:05:43.216 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm92.ko 2023-10-16T14:05:43.217 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm93.ko 2023-10-16T14:05:43.217 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95234.ko 2023-10-16T14:05:43.217 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95241.ko 2023-10-16T14:05:43.217 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/lm95245.ko 2023-10-16T14:05:43.217 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc2945.ko 2023-10-16T14:05:43.218 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4151.ko 2023-10-16T14:05:43.218 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4215.ko 2023-10-16T14:05:43.218 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4222.ko 2023-10-16T14:05:43.218 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4245.ko 2023-10-16T14:05:43.218 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4260.ko 2023-10-16T14:05:43.219 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/ltc4261.ko 2023-10-16T14:05:43.219 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max16065.ko 2023-10-16T14:05:43.219 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1619.ko 2023-10-16T14:05:43.219 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max1668.ko 2023-10-16T14:05:43.219 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max197.ko 2023-10-16T14:05:43.220 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6639.ko 2023-10-16T14:05:43.220 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6650.ko 2023-10-16T14:05:43.220 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/max6697.ko 2023-10-16T14:05:43.220 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/mcp3021.ko 2023-10-16T14:05:43.221 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6683.ko 2023-10-16T14:05:43.221 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-16T14:05:43.221 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct6775.ko 2023-10-16T14:05:43.221 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/nct7802.ko 2023-10-16T14:05:43.221 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87360.ko 2023-10-16T14:05:43.222 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pc87427.ko 2023-10-16T14:05:43.222 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pcf8591.ko 2023-10-16T14:05:43.222 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus 2023-10-16T14:05:43.222 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-16T14:05:43.222 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-16T14:05:43.223 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-16T14:05:43.223 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-16T14:05:43.223 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-16T14:05:43.223 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-16T14:05:43.223 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-16T14:05:43.224 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-16T14:05:43.224 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-16T14:05:43.224 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-16T14:05:43.224 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-16T14:05:43.224 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-16T14:05:43.225 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/powr1220.ko 2023-10-16T14:05:43.225 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5627.ko 2023-10-16T14:05:43.225 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch5636.ko 2023-10-16T14:05:43.225 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-16T14:05:43.226 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht15.ko 2023-10-16T14:05:43.226 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sht21.ko 2023-10-16T14:05:43.226 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/shtc1.ko 2023-10-16T14:05:43.226 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/sis5595.ko 2023-10-16T14:05:43.226 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-16T14:05:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-16T14:05:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-16T14:05:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/thmc50.ko 2023-10-16T14:05:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp102.ko 2023-10-16T14:05:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp103.ko 2023-10-16T14:05:43.228 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp401.ko 2023-10-16T14:05:43.228 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/tmp421.ko 2023-10-16T14:05:43.228 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-16T14:05:43.228 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/via686a.ko 2023-10-16T14:05:43.228 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt1211.ko 2023-10-16T14:05:43.229 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/vt8231.ko 2023-10-16T14:05:43.229 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-16T14:05:43.229 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83627hf.ko 2023-10-16T14:05:43.229 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83781d.ko 2023-10-16T14:05:43.229 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83791d.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83792d.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83793.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83795.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-16T14:05:43.230 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c 2023-10-16T14:05:43.231 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos 2023-10-16T14:05:43.231 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-16T14:05:43.231 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-16T14:05:43.231 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses 2023-10-16T14:05:43.231 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-16T14:05:43.232 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-16T14:05:43.232 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-16T14:05:43.232 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-16T14:05:43.232 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-16T14:05:43.232 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-16T14:05:43.233 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-16T14:05:43.233 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-16T14:05:43.233 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-16T14:05:43.233 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-16T14:05:43.233 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-16T14:05:43.234 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-16T14:05:43.234 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-16T14:05:43.234 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-16T14:05:43.234 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-16T14:05:43.234 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-16T14:05:43.235 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-16T14:05:43.235 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-16T14:05:43.235 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-16T14:05:43.235 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-16T14:05:43.235 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-dev.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-mux.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-16T14:05:43.236 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-16T14:05:43.237 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/i2c/i2c-stub.ko 2023-10-16T14:05:43.237 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband 2023-10-16T14:05:43.237 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core 2023-10-16T14:05:43.237 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-16T14:05:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-16T14:05:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-16T14:05:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-16T14:05:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-16T14:05:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-16T14:05:43.239 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-16T14:05:43.239 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw 2023-10-16T14:05:43.239 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4 2023-10-16T14:05:43.239 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-16T14:05:43.239 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4 2023-10-16T14:05:43.240 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-16T14:05:43.240 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5 2023-10-16T14:05:43.240 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-16T14:05:43.240 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca 2023-10-16T14:05:43.240 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-16T14:05:43.241 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp 2023-10-16T14:05:43.241 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib 2023-10-16T14:05:43.241 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-16T14:05:43.241 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser 2023-10-16T14:05:43.241 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-16T14:05:43.242 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert 2023-10-16T14:05:43.242 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-16T14:05:43.242 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp 2023-10-16T14:05:43.242 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-16T14:05:43.242 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt 2023-10-16T14:05:43.243 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-16T14:05:43.243 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input 2023-10-16T14:05:43.243 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport 2023-10-16T14:05:43.243 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-16T14:05:43.243 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-16T14:05:43.244 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/gameport/gameport.ko 2023-10-16T14:05:43.244 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/joydev.ko 2023-10-16T14:05:43.244 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard 2023-10-16T14:05:43.244 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-16T14:05:43.244 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse 2023-10-16T14:05:43.245 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/appletouch.ko 2023-10-16T14:05:43.245 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-16T14:05:43.245 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-16T14:05:43.245 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-16T14:05:43.245 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/sermouse.ko 2023-10-16T14:05:43.246 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-16T14:05:43.246 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-16T14:05:43.246 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-16T14:05:43.246 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4 2023-10-16T14:05:43.246 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/serio/serio_raw.ko 2023-10-16T14:05:43.247 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/input/sparse-keymap.ko 2023-10-16T14:05:43.248 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu 2023-10-16T14:05:43.248 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd 2023-10-16T14:05:43.248 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-16T14:05:43.248 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds 2023-10-16T14:05:43.248 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-blinkm.ko 2023-10-16T14:05:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lm3530.ko 2023-10-16T14:05:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-lp3944.ko 2023-10-16T14:05:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/leds-ss4200.ko 2023-10-16T14:05:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger 2023-10-16T14:05:43.249 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-16T14:05:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-16T14:05:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-16T14:05:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-16T14:05:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-16T14:05:43.250 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/bcache/bcache.ko 2023-10-16T14:05:43.251 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-bio-prison.ko 2023-10-16T14:05:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache-smq.ko 2023-10-16T14:05:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-cache.ko 2023-10-16T14:05:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-crypt.ko 2023-10-16T14:05:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-delay.ko 2023-10-16T14:05:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-flakey.ko 2023-10-16T14:05:43.253 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-log-userspace.ko 2023-10-16T14:05:43.253 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-multipath.ko 2023-10-16T14:05:43.253 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-queue-length.ko 2023-10-16T14:05:43.253 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-raid.ko 2023-10-16T14:05:43.253 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-round-robin.ko 2023-10-16T14:05:43.254 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-service-time.ko 2023-10-16T14:05:43.254 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-switch.ko 2023-10-16T14:05:43.254 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-thin-pool.ko 2023-10-16T14:05:43.254 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/dm-verity.ko 2023-10-16T14:05:43.254 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/faulty.ko 2023-10-16T14:05:43.255 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/linear.ko 2023-10-16T14:05:43.255 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/multipath.ko 2023-10-16T14:05:43.255 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data 2023-10-16T14:05:43.255 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-16T14:05:43.255 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid0.ko 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid1.ko 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid10.ko 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/md/raid456.ko 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion 2023-10-16T14:05:43.256 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptbase.ko 2023-10-16T14:05:43.257 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptctl.ko 2023-10-16T14:05:43.257 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptfc.ko 2023-10-16T14:05:43.257 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptlan.ko 2023-10-16T14:05:43.257 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptsas.ko 2023-10-16T14:05:43.257 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-16T14:05:43.258 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/message/fusion/mptspi.ko 2023-10-16T14:05:43.258 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd 2023-10-16T14:05:43.258 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_ich.ko 2023-10-16T14:05:43.258 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/lpc_sch.ko 2023-10-16T14:05:43.258 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/mfd-core.ko 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/sm501.ko 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/viperboard.ko 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/vx855.ko 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/mfd/wl1273-core.ko 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc 2023-10-16T14:05:43.259 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl 2023-10-16T14:05:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-16T14:05:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds9802als.ko 2023-10-16T14:05:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/apds990x.ko 2023-10-16T14:05:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/bh1770glc.ko 2023-10-16T14:05:43.260 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710 2023-10-16T14:05:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/cb710/cb710.ko 2023-10-16T14:05:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom 2023-10-16T14:05:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/at24.ko 2023-10-16T14:05:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-16T14:05:43.261 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-16T14:05:43.262 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-16T14:05:43.262 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/enclosure.ko 2023-10-16T14:05:43.262 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/hpilo.ko 2023-10-16T14:05:43.262 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29003.ko 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/isl29020.ko 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei 2023-10-16T14:05:43.263 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-me.ko 2023-10-16T14:05:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-16T14:05:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/mei/mei.ko 2023-10-16T14:05:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru 2023-10-16T14:05:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-16T14:05:43.265 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tifm_core.ko 2023-10-16T14:05:43.266 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/tsl2550.ko 2023-10-16T14:05:43.266 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_balloon.ko 2023-10-16T14:05:43.266 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci 2023-10-16T14:05:43.266 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-16T14:05:43.267 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net 2023-10-16T14:05:43.267 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding 2023-10-16T14:05:43.267 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/bonding/bonding.ko 2023-10-16T14:05:43.267 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa 2023-10-16T14:05:43.267 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53 2023-10-16T14:05:43.268 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-16T14:05:43.268 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-16T14:05:43.268 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-16T14:05:43.268 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-16T14:05:43.268 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-16T14:05:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/dummy.ko 2023-10-16T14:05:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/eql.ko 2023-10-16T14:05:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet 2023-10-16T14:05:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com 2023-10-16T14:05:43.269 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-16T14:05:43.270 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-16T14:05:43.270 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-16T14:05:43.270 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-16T14:05:43.270 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390 2023-10-16T14:05:43.271 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-16T14:05:43.271 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-16T14:05:43.271 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-16T14:05:43.271 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-16T14:05:43.271 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec 2023-10-16T14:05:43.272 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-16T14:05:43.272 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere 2023-10-16T14:05:43.272 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-16T14:05:43.272 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon 2023-10-16T14:05:43.272 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-16T14:05:43.273 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera 2023-10-16T14:05:43.273 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-16T14:05:43.273 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd 2023-10-16T14:05:43.273 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-16T14:05:43.273 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-16T14:05:43.274 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-16T14:05:43.274 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros 2023-10-16T14:05:43.274 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx 2023-10-16T14:05:43.274 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-16T14:05:43.274 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-16T14:05:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-16T14:05:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-16T14:05:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-16T14:05:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-16T14:05:43.275 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-16T14:05:43.276 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-16T14:05:43.276 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom 2023-10-16T14:05:43.276 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-16T14:05:43.276 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-16T14:05:43.276 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-16T14:05:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade 2023-10-16T14:05:43.278 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna 2023-10-16T14:05:43.278 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-16T14:05:43.278 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio 2023-10-16T14:05:43.278 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-16T14:05:43.278 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-16T14:05:43.279 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-16T14:05:43.279 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-16T14:05:43.279 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-16T14:05:43.279 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-16T14:05:43.279 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic 2023-10-16T14:05:43.280 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-16T14:05:43.281 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec 2023-10-16T14:05:43.281 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip 2023-10-16T14:05:43.281 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-16T14:05:43.281 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-16T14:05:43.281 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-16T14:05:43.282 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-16T14:05:43.282 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-16T14:05:43.282 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-16T14:05:43.282 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink 2023-10-16T14:05:43.282 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-16T14:05:43.283 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-16T14:05:43.283 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/dnet.ko 2023-10-16T14:05:43.283 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex 2023-10-16T14:05:43.283 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet 2023-10-16T14:05:43.283 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-16T14:05:43.285 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-16T14:05:43.286 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-16T14:05:43.286 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel 2023-10-16T14:05:43.286 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-16T14:05:43.286 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000 2023-10-16T14:05:43.286 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-16T14:05:43.287 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-16T14:05:43.287 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-16T14:05:43.287 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-16T14:05:43.287 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-16T14:05:43.287 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e 2023-10-16T14:05:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-16T14:05:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf 2023-10-16T14:05:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-16T14:05:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-16T14:05:43.289 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-16T14:05:43.290 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-16T14:05:43.290 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/jme.ko 2023-10-16T14:05:43.290 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell 2023-10-16T14:05:43.290 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-16T14:05:43.290 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-16T14:05:43.291 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-16T14:05:43.291 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox 2023-10-16T14:05:43.291 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-16T14:05:43.291 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-16T14:05:43.291 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-16T14:05:43.292 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-16T14:05:43.292 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-16T14:05:43.292 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-16T14:05:43.292 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel 2023-10-16T14:05:43.293 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-16T14:05:43.293 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom 2023-10-16T14:05:43.293 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-16T14:05:43.293 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-16T14:05:43.293 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi 2023-10-16T14:05:43.294 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-16T14:05:43.294 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-16T14:05:43.294 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion 2023-10-16T14:05:43.294 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-16T14:05:43.294 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia 2023-10-16T14:05:43.295 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-16T14:05:43.295 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines 2023-10-16T14:05:43.295 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-16T14:05:43.295 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-16T14:05:43.295 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic 2023-10-16T14:05:43.296 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-16T14:05:43.296 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-16T14:05:43.296 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-16T14:05:43.296 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-16T14:05:43.296 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-16T14:05:43.297 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc 2023-10-16T14:05:43.297 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-16T14:05:43.297 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek 2023-10-16T14:05:43.297 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-16T14:05:43.297 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-16T14:05:43.298 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-16T14:05:43.298 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-16T14:05:43.298 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker 2023-10-16T14:05:43.298 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-16T14:05:43.298 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc 2023-10-16T14:05:43.299 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-16T14:05:43.299 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan 2023-10-16T14:05:43.299 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-16T14:05:43.299 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis 2023-10-16T14:05:43.299 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-16T14:05:43.300 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-16T14:05:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro 2023-10-16T14:05:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-16T14:05:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-16T14:05:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-16T14:05:43.301 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun 2023-10-16T14:05:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-16T14:05:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-16T14:05:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-16T14:05:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-16T14:05:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti 2023-10-16T14:05:43.303 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-16T14:05:43.303 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti 2023-10-16T14:05:43.303 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-16T14:05:43.303 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via 2023-10-16T14:05:43.303 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-16T14:05:43.304 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-16T14:05:43.304 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet 2023-10-16T14:05:43.304 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-16T14:05:43.304 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-16T14:05:43.304 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom 2023-10-16T14:05:43.305 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-16T14:05:43.305 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/geneve.ko 2023-10-16T14:05:43.305 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv 2023-10-16T14:05:43.305 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-16T14:05:43.305 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ifb.ko 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvlan.ko 2023-10-16T14:05:43.306 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/macvtap.ko 2023-10-16T14:05:43.307 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio 2023-10-16T14:05:43.307 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio.ko 2023-10-16T14:05:43.307 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-16T14:05:43.307 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-16T14:05:43.307 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-16T14:05:43.308 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/mii.ko 2023-10-16T14:05:43.308 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/net_failover.ko 2023-10-16T14:05:43.308 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/netconsole.ko 2023-10-16T14:05:43.308 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/nlmon.ko 2023-10-16T14:05:43.308 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs 2023-10-16T14:05:43.309 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-16T14:05:43.309 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-16T14:05:43.309 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy 2023-10-16T14:05:43.309 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/amd.ko 2023-10-16T14:05:43.309 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-16T14:05:43.310 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-16T14:05:43.310 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-16T14:05:43.310 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-16T14:05:43.310 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/broadcom.ko 2023-10-16T14:05:43.310 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/cicada.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/davicom.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/dp83640.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/et1011c.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/icplus.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/lxt.ko 2023-10-16T14:05:43.311 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/marvell.ko 2023-10-16T14:05:43.312 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/micrel.ko 2023-10-16T14:05:43.312 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/national.ko 2023-10-16T14:05:43.312 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/phylink.ko 2023-10-16T14:05:43.312 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/qsemi.ko 2023-10-16T14:05:43.312 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/realtek.ko 2023-10-16T14:05:43.313 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/smsc.ko 2023-10-16T14:05:43.313 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-16T14:05:43.313 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/phy/vitesse.ko 2023-10-16T14:05:43.313 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp 2023-10-16T14:05:43.313 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-16T14:05:43.314 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-16T14:05:43.314 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-16T14:05:43.314 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-16T14:05:43.314 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppoe.ko 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pppox.ko 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/ppp/pptp.ko 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip 2023-10-16T14:05:43.315 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slhc.ko 2023-10-16T14:05:43.316 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/slip/slip.ko 2023-10-16T14:05:43.316 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/sungem_phy.ko 2023-10-16T14:05:43.316 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tap.ko 2023-10-16T14:05:43.316 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team 2023-10-16T14:05:43.316 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_random.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-16T14:05:43.317 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/tun.ko 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/veth.ko 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/virtio_net.ko 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan 2023-10-16T14:05:43.318 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-16T14:05:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback 2023-10-16T14:05:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-16T14:05:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/net/xen-netfront.ko 2023-10-16T14:05:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb 2023-10-16T14:05:43.319 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ntb/ntb.ko 2023-10-16T14:05:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme 2023-10-16T14:05:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host 2023-10-16T14:05:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-16T14:05:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/nvme/host/nvme.ko 2023-10-16T14:05:43.320 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport 2023-10-16T14:05:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport.ko 2023-10-16T14:05:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_cs.ko 2023-10-16T14:05:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_pc.ko 2023-10-16T14:05:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/parport/parport_serial.ko 2023-10-16T14:05:43.321 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci 2023-10-16T14:05:43.322 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug 2023-10-16T14:05:43.322 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-16T14:05:43.322 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie 2023-10-16T14:05:43.322 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-16T14:05:43.322 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pci/xen-pcifront.ko 2023-10-16T14:05:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia 2023-10-16T14:05:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/i82092.ko 2023-10-16T14:05:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/pd6729.ko 2023-10-16T14:05:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-16T14:05:43.323 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl 2023-10-16T14:05:43.324 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel 2023-10-16T14:05:43.324 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-16T14:05:43.324 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform 2023-10-16T14:05:43.324 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86 2023-10-16T14:05:43.324 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-16T14:05:43.325 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-16T14:05:43.326 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-16T14:05:43.326 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-16T14:05:43.326 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-16T14:05:43.326 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-16T14:05:43.326 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/hdaps.ko 2023-10-16T14:05:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel 2023-10-16T14:05:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-16T14:05:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-16T14:05:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-16T14:05:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-16T14:05:43.328 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-16T14:05:43.329 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-16T14:05:43.329 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/platform/x86/wmi.ko 2023-10-16T14:05:43.329 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap 2023-10-16T14:05:43.329 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-16T14:05:43.329 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-16T14:05:43.330 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps 2023-10-16T14:05:43.330 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients 2023-10-16T14:05:43.330 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-16T14:05:43.330 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-16T14:05:43.330 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-16T14:05:43.331 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/pps/pps_core.ko 2023-10-16T14:05:43.331 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp 2023-10-16T14:05:43.331 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp.ko 2023-10-16T14:05:43.331 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-16T14:05:43.331 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-16T14:05:43.332 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-16T14:05:43.333 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-16T14:05:43.333 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-16T14:05:43.333 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-16T14:05:43.333 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-16T14:05:43.333 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-16T14:05:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-16T14:05:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-16T14:05:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-16T14:05:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-16T14:05:43.334 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-16T14:05:43.335 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-16T14:05:43.336 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-16T14:05:43.336 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-16T14:05:43.336 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-16T14:05:43.336 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-16T14:05:43.336 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-16T14:05:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-16T14:05:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-16T14:05:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-16T14:05:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-16T14:05:43.337 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-sas.ko 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/BusLogic.ko 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/a100u2w.ko 2023-10-16T14:05:43.338 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid 2023-10-16T14:05:43.339 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-16T14:05:43.339 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/advansys.ko 2023-10-16T14:05:43.339 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx 2023-10-16T14:05:43.339 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-16T14:05:43.339 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/am53c974.ko 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/atp870u.ko 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi 2023-10-16T14:05:43.340 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-16T14:05:43.341 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa 2023-10-16T14:05:43.341 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-16T14:05:43.341 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc 2023-10-16T14:05:43.341 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-16T14:05:43.341 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i 2023-10-16T14:05:43.342 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-16T14:05:43.342 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ch.ko 2023-10-16T14:05:43.342 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor 2023-10-16T14:05:43.342 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-16T14:05:43.342 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-16T14:05:43.343 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dc395x.ko 2023-10-16T14:05:43.344 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler 2023-10-16T14:05:43.344 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-16T14:05:43.344 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-16T14:05:43.344 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-16T14:05:43.344 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-16T14:05:43.345 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/dmx3191d.ko 2023-10-16T14:05:43.345 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r 2023-10-16T14:05:43.345 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-16T14:05:43.345 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/esp_scsi.ko 2023-10-16T14:05:43.345 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe 2023-10-16T14:05:43.346 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-16T14:05:43.346 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-16T14:05:43.346 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic 2023-10-16T14:05:43.346 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-16T14:05:43.346 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hpsa.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hptiop.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/initio.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ipr.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ips.ko 2023-10-16T14:05:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci 2023-10-16T14:05:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/isci/isci.ko 2023-10-16T14:05:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-16T14:05:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-16T14:05:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc 2023-10-16T14:05:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-16T14:05:43.349 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi.ko 2023-10-16T14:05:43.349 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-16T14:05:43.349 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas 2023-10-16T14:05:43.349 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-16T14:05:43.349 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc 2023-10-16T14:05:43.350 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-16T14:05:43.350 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid 2023-10-16T14:05:43.350 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid.ko 2023-10-16T14:05:43.350 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-16T14:05:43.350 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-16T14:05:43.351 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-16T14:05:43.351 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas 2023-10-16T14:05:43.351 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-16T14:05:43.351 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas 2023-10-16T14:05:43.351 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-16T14:05:43.352 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/mvumi.ko 2023-10-16T14:05:43.352 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001 2023-10-16T14:05:43.352 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-16T14:05:43.352 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/pmcraid.ko 2023-10-16T14:05:43.352 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla1280.ko 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-16T14:05:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/raid_class.ko 2023-10-16T14:05:43.354 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_debug.ko 2023-10-16T14:05:43.354 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-16T14:05:43.354 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-16T14:05:43.354 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-16T14:05:43.354 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-16T14:05:43.355 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-16T14:05:43.355 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/ses.ko 2023-10-16T14:05:43.355 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/st.ko 2023-10-16T14:05:43.355 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/stex.ko 2023-10-16T14:05:43.355 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/scsi/wd719x.ko 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb 2023-10-16T14:05:43.356 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ssb/ssb.ko 2023-10-16T14:05:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target 2023-10-16T14:05:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi 2023-10-16T14:05:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-16T14:05:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback 2023-10-16T14:05:43.357 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-16T14:05:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_file.ko 2023-10-16T14:05:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_iblock.ko 2023-10-16T14:05:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_mod.ko 2023-10-16T14:05:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_pscsi.ko 2023-10-16T14:05:43.358 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/target_core_user.ko 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-16T14:05:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-16T14:05:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-16T14:05:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-16T14:05:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-16T14:05:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-16T14:05:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-16T14:05:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-16T14:05:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-16T14:05:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-16T14:05:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-16T14:05:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-16T14:05:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-16T14:05:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-16T14:05:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-16T14:05:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-16T14:05:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_gsm.ko 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/n_hdlc.ko 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/nozomi.ko 2023-10-16T14:05:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial 2023-10-16T14:05:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250 2023-10-16T14:05:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-16T14:05:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-16T14:05:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm 2023-10-16T14:05:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-16T14:05:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs 2023-10-16T14:05:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core 2023-10-16T14:05:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-16T14:05:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host 2023-10-16T14:05:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio.ko 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_aec.ko 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_cif.ko 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-16T14:05:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/uio/uio_sercos3.ko 2023-10-16T14:05:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb 2023-10-16T14:05:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class 2023-10-16T14:05:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-16T14:05:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-16T14:05:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usblp.ko 2023-10-16T14:05:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/class/usbtmc.ko 2023-10-16T14:05:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host 2023-10-16T14:05:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-16T14:05:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image 2023-10-16T14:05:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/mdc800.ko 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/image/microtek.ko 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/adutux.ko 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi26.ko 2023-10-16T14:05:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/emi62.ko 2023-10-16T14:05:43.370 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ezusb.ko 2023-10-16T14:05:43.370 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/idmouse.ko 2023-10-16T14:05:43.370 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-16T14:05:43.370 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-16T14:05:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/ldusb.ko 2023-10-16T14:05:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-16T14:05:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga 2023-10-16T14:05:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-16T14:05:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-16T14:05:43.372 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usb3503.ko 2023-10-16T14:05:43.372 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usblcd.ko 2023-10-16T14:05:43.372 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-16T14:05:43.372 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/uss720.ko 2023-10-16T14:05:43.372 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/misc/yurex.ko 2023-10-16T14:05:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy 2023-10-16T14:05:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-16T14:05:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial 2023-10-16T14:05:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/aircable.ko 2023-10-16T14:05:43.373 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ark3116.ko 2023-10-16T14:05:43.374 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-16T14:05:43.374 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ch341.ko 2023-10-16T14:05:43.374 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cp210x.ko 2023-10-16T14:05:43.374 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-16T14:05:43.374 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-16T14:05:43.375 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-16T14:05:43.375 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/empeg.ko 2023-10-16T14:05:43.375 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-16T14:05:43.375 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-16T14:05:43.375 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-16T14:05:43.376 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/io_ti.ko 2023-10-16T14:05:43.376 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipaq.ko 2023-10-16T14:05:43.376 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ipw.ko 2023-10-16T14:05:43.376 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-16T14:05:43.376 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-16T14:05:43.377 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan.ko 2023-10-16T14:05:43.377 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-16T14:05:43.377 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-16T14:05:43.377 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-16T14:05:43.378 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-16T14:05:43.378 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7720.ko 2023-10-16T14:05:43.378 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/mos7840.ko 2023-10-16T14:05:43.378 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/navman.ko 2023-10-16T14:05:43.378 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/omninet.ko 2023-10-16T14:05:43.379 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/opticon.ko 2023-10-16T14:05:43.379 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/option.ko 2023-10-16T14:05:43.379 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/oti6858.ko 2023-10-16T14:05:43.379 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/pl2303.ko 2023-10-16T14:05:43.379 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcaux.ko 2023-10-16T14:05:43.380 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/qcserial.ko 2023-10-16T14:05:43.380 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/quatech2.ko 2023-10-16T14:05:43.380 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-16T14:05:43.380 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/sierra.ko 2023-10-16T14:05:43.380 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-16T14:05:43.381 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ssu100.ko 2023-10-16T14:05:43.381 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-16T14:05:43.381 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-16T14:05:43.381 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-16T14:05:43.381 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-16T14:05:43.382 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-16T14:05:43.382 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/visor.ko 2023-10-16T14:05:43.382 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-16T14:05:43.382 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-16T14:05:43.382 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/uas.ko 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-16T14:05:43.383 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-16T14:05:43.384 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-16T14:05:43.384 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-16T14:05:43.384 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-16T14:05:43.384 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-16T14:05:43.384 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-16T14:05:43.385 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-16T14:05:43.385 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-16T14:05:43.385 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-16T14:05:43.385 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-16T14:05:43.385 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip 2023-10-16T14:05:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-16T14:05:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-16T14:05:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-16T14:05:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio 2023-10-16T14:05:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio.ko 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost 2023-10-16T14:05:43.387 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost.ko 2023-10-16T14:05:43.388 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-16T14:05:43.388 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_net.ko 2023-10-16T14:05:43.388 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-16T14:05:43.388 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video 2023-10-16T14:05:43.388 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/lcd.ko 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio.ko 2023-10-16T14:05:43.389 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-16T14:05:43.390 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-16T14:05:43.390 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci.ko 2023-10-16T14:05:43.390 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-16T14:05:43.390 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-16T14:05:43.390 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/virtio/virtio_ring.ko 2023-10-16T14:05:43.391 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1 2023-10-16T14:05:43.391 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters 2023-10-16T14:05:43.391 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2482.ko 2023-10-16T14:05:43.391 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/masters/ds2490.ko 2023-10-16T14:05:43.391 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-16T14:05:43.392 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-16T14:05:43.393 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-16T14:05:43.393 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-16T14:05:43.393 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-16T14:05:43.393 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-16T14:05:43.393 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-16T14:05:43.394 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/w1/wire.ko 2023-10-16T14:05:43.394 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog 2023-10-16T14:05:43.394 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-16T14:05:43.394 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-16T14:05:43.394 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/hpwdt.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/i6300esb.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-16T14:05:43.395 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ibmasr.ko 2023-10-16T14:05:43.396 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-16T14:05:43.396 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-16T14:05:43.396 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-16T14:05:43.396 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/machzwd.ko 2023-10-16T14:05:43.396 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/nv_tco.ko 2023-10-16T14:05:43.397 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-16T14:05:43.397 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-16T14:05:43.397 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-16T14:05:43.397 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-16T14:05:43.397 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/softdog.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/via_wdt.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-16T14:05:43.398 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-16T14:05:43.399 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-16T14:05:43.399 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen 2023-10-16T14:05:43.399 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-16T14:05:43.399 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-evtchn.ko 2023-10-16T14:05:43.399 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-gntdev.ko 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xen-privcmd.ko 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs 2023-10-16T14:05:43.400 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-16T14:05:43.401 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs 2023-10-16T14:05:43.401 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p 2023-10-16T14:05:43.401 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/9p/9p.ko 2023-10-16T14:05:43.401 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs 2023-10-16T14:05:43.401 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/affs/affs.ko 2023-10-16T14:05:43.402 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs 2023-10-16T14:05:43.402 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/befs/befs.ko 2023-10-16T14:05:43.402 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/binfmt_misc.ko 2023-10-16T14:05:43.402 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs 2023-10-16T14:05:43.402 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/btrfs/btrfs.ko 2023-10-16T14:05:43.403 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles 2023-10-16T14:05:43.403 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cachefiles/cachefiles.ko 2023-10-16T14:05:43.403 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph 2023-10-16T14:05:43.403 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ceph/ceph.ko 2023-10-16T14:05:43.403 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda 2023-10-16T14:05:43.404 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/coda/coda.ko 2023-10-16T14:05:43.404 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs 2023-10-16T14:05:43.404 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/cramfs/cramfs.ko 2023-10-16T14:05:43.404 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm 2023-10-16T14:05:43.404 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/dlm/dlm.ko 2023-10-16T14:05:43.405 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs 2023-10-16T14:05:43.405 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-16T14:05:43.405 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs 2023-10-16T14:05:43.405 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/f2fs/f2fs.ko 2023-10-16T14:05:43.405 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/fat.ko 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/msdos.ko 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fat/vfat.ko 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fscache/fscache.ko 2023-10-16T14:05:43.406 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse 2023-10-16T14:05:43.407 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/cuse.ko 2023-10-16T14:05:43.407 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/fuse/fuse.ko 2023-10-16T14:05:43.407 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2 2023-10-16T14:05:43.407 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/gfs2/gfs2.ko 2023-10-16T14:05:43.407 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs 2023-10-16T14:05:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfs/hfs.ko 2023-10-16T14:05:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus 2023-10-16T14:05:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/hfsplus/hfsplus.ko 2023-10-16T14:05:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs 2023-10-16T14:05:43.408 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/isofs/isofs.ko 2023-10-16T14:05:43.409 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs 2023-10-16T14:05:43.409 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/jfs/jfs.ko 2023-10-16T14:05:43.409 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd 2023-10-16T14:05:43.409 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/lockd/lockd.ko 2023-10-16T14:05:43.409 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix 2023-10-16T14:05:43.410 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/minix/minix.ko 2023-10-16T14:05:43.410 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs 2023-10-16T14:05:43.410 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/netfs/netfs.ko 2023-10-16T14:05:43.410 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs 2023-10-16T14:05:43.410 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout 2023-10-16T14:05:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-16T14:05:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout 2023-10-16T14:05:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-16T14:05:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout 2023-10-16T14:05:43.411 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-16T14:05:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfs.ko 2023-10-16T14:05:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv3.ko 2023-10-16T14:05:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs/nfsv4.ko 2023-10-16T14:05:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common 2023-10-16T14:05:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/grace.ko 2023-10-16T14:05:43.413 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-16T14:05:43.413 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd 2023-10-16T14:05:43.413 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nfsd/nfsd.ko 2023-10-16T14:05:43.413 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2 2023-10-16T14:05:43.413 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nilfs2/nilfs2.ko 2023-10-16T14:05:43.414 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls 2023-10-16T14:05:43.414 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-celtic.ko 2023-10-16T14:05:43.414 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-centeuro.ko 2023-10-16T14:05:43.414 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-croatian.ko 2023-10-16T14:05:43.414 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-cyrillic.ko 2023-10-16T14:05:43.415 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-gaelic.ko 2023-10-16T14:05:43.415 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-greek.ko 2023-10-16T14:05:43.415 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-iceland.ko 2023-10-16T14:05:43.415 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-inuit.ko 2023-10-16T14:05:43.415 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-roman.ko 2023-10-16T14:05:43.416 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-romanian.ko 2023-10-16T14:05:43.416 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/mac-turkish.ko 2023-10-16T14:05:43.416 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1250.ko 2023-10-16T14:05:43.416 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1251.ko 2023-10-16T14:05:43.416 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp1255.ko 2023-10-16T14:05:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp737.ko 2023-10-16T14:05:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp775.ko 2023-10-16T14:05:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp850.ko 2023-10-16T14:05:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp852.ko 2023-10-16T14:05:43.417 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp855.ko 2023-10-16T14:05:43.418 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp857.ko 2023-10-16T14:05:43.418 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp860.ko 2023-10-16T14:05:43.418 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp861.ko 2023-10-16T14:05:43.418 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp862.ko 2023-10-16T14:05:43.418 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp863.ko 2023-10-16T14:05:43.419 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp864.ko 2023-10-16T14:05:43.419 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp865.ko 2023-10-16T14:05:43.419 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp866.ko 2023-10-16T14:05:43.419 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp869.ko 2023-10-16T14:05:43.419 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp874.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp932.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp936.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp949.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_cp950.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_euc-jp.ko 2023-10-16T14:05:43.420 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-16T14:05:43.421 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-16T14:05:43.421 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-16T14:05:43.421 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-16T14:05:43.421 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-16T14:05:43.421 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-16T14:05:43.422 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-16T14:05:43.422 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-16T14:05:43.422 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-16T14:05:43.422 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-16T14:05:43.422 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-16T14:05:43.423 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-r.ko 2023-10-16T14:05:43.423 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-16T14:05:43.423 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_koi8-u.ko 2023-10-16T14:05:43.423 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_ucs2_utils.ko 2023-10-16T14:05:43.423 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/nls/nls_utf8.ko 2023-10-16T14:05:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2 2023-10-16T14:05:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster 2023-10-16T14:05:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-16T14:05:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm 2023-10-16T14:05:43.424 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-16T14:05:43.425 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs 2023-10-16T14:05:43.425 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-16T14:05:43.425 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2.ko 2023-10-16T14:05:43.425 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-16T14:05:43.425 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/overlayfs/overlay.ko 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/pstore/ramoops.ko 2023-10-16T14:05:43.426 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs 2023-10-16T14:05:43.427 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/reiserfs/reiserfs.ko 2023-10-16T14:05:43.427 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs 2023-10-16T14:05:43.427 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/romfs/romfs.ko 2023-10-16T14:05:43.427 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb 2023-10-16T14:05:43.427 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/client/cifs.ko 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/smb/common/cifs_md4.ko 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs 2023-10-16T14:05:43.428 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/squashfs/squashfs.ko 2023-10-16T14:05:43.429 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv 2023-10-16T14:05:43.429 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/sysv/sysv.ko 2023-10-16T14:05:43.429 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf 2023-10-16T14:05:43.429 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/udf/udf.ko 2023-10-16T14:05:43.429 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs 2023-10-16T14:05:43.430 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/ufs/ufs.ko 2023-10-16T14:05:43.430 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs 2023-10-16T14:05:43.430 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/fs/xfs/xfs.ko 2023-10-16T14:05:43.430 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel 2023-10-16T14:05:43.430 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/rcu/rcutorture.ko 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/torture.ko 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib 2023-10-16T14:05:43.431 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/asn1_encoder.ko 2023-10-16T14:05:43.432 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc-itu-t.ko 2023-10-16T14:05:43.432 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crc8.ko 2023-10-16T14:05:43.432 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto 2023-10-16T14:05:43.432 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libarc4.ko 2023-10-16T14:05:43.432 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/crypto/libdes.ko 2023-10-16T14:05:43.433 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/libcrc32c.ko 2023-10-16T14:05:43.433 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lru_cache.ko 2023-10-16T14:05:43.433 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4 2023-10-16T14:05:43.433 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4_compress.ko 2023-10-16T14:05:43.433 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/math/cordic.ko 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/raid6/raid6_pq.ko 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon 2023-10-16T14:05:43.434 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-16T14:05:43.435 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_bm.ko 2023-10-16T14:05:43.435 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_fsm.ko 2023-10-16T14:05:43.435 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/ts_kmp.ko 2023-10-16T14:05:43.435 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd 2023-10-16T14:05:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/lib/zstd/zstd_compress.ko 2023-10-16T14:05:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm 2023-10-16T14:05:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/mm/hwpoison-inject.ko 2023-10-16T14:05:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net 2023-10-16T14:05:43.436 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/6lowpan.ko 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_dest.ko 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_hop.ko 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-16T14:05:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-16T14:05:43.438 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_routing.ko 2023-10-16T14:05:43.438 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/6lowpan/nhc_udp.ko 2023-10-16T14:05:43.438 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802 2023-10-16T14:05:43.438 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/garp.ko 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/mrp.ko 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/p8022.ko 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/psnap.ko 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/802/stp.ko 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q 2023-10-16T14:05:43.439 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/8021q/8021q.ko 2023-10-16T14:05:43.440 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p 2023-10-16T14:05:43.440 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet.ko 2023-10-16T14:05:43.440 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_fd.ko 2023-10-16T14:05:43.440 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_rdma.ko 2023-10-16T14:05:43.440 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/9p/9pnet_virtio.ko 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/batman-adv/batman-adv.ko 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/br_netfilter.ko 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/bridge.ko 2023-10-16T14:05:43.441 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter 2023-10-16T14:05:43.442 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-16T14:05:43.442 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-16T14:05:43.442 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-16T14:05:43.442 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-16T14:05:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-16T14:05:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-16T14:05:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-16T14:05:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-16T14:05:43.443 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-16T14:05:43.444 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-16T14:05:43.445 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-16T14:05:43.445 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-16T14:05:43.445 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-16T14:05:43.445 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-16T14:05:43.445 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph 2023-10-16T14:05:43.446 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ceph/libceph.ko 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/failover.ko 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/core/pktgen.ko 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp.ko 2023-10-16T14:05:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_diag.ko 2023-10-16T14:05:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv4.ko 2023-10-16T14:05:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dccp/dccp_ipv6.ko 2023-10-16T14:05:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver 2023-10-16T14:05:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-16T14:05:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/dsa_core.ko 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_brcm.ko 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_dsa.ko 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_none.ko 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/dsa/tag_trailer.ko 2023-10-16T14:05:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154 2023-10-16T14:05:43.450 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan 2023-10-16T14:05:43.450 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-16T14:05:43.450 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154.ko 2023-10-16T14:05:43.450 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-16T14:05:43.450 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4 2023-10-16T14:05:43.451 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ah4.ko 2023-10-16T14:05:43.451 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/esp4.ko 2023-10-16T14:05:43.451 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/fou.ko 2023-10-16T14:05:43.451 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/gre.ko 2023-10-16T14:05:43.451 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/inet_diag.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_gre.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_tunnel.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ip_vti.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipcomp.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/ipip.ko 2023-10-16T14:05:43.452 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter 2023-10-16T14:05:43.453 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-16T14:05:43.453 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-16T14:05:43.453 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-16T14:05:43.453 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-16T14:05:43.453 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-16T14:05:43.454 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-16T14:05:43.454 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-16T14:05:43.454 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-16T14:05:43.454 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-16T14:05:43.454 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-16T14:05:43.455 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-16T14:05:43.455 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-16T14:05:43.455 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-16T14:05:43.455 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-16T14:05:43.455 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-16T14:05:43.456 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-16T14:05:43.456 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-16T14:05:43.456 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-16T14:05:43.456 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-16T14:05:43.456 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_bic.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_diag.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-16T14:05:43.457 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_htcp.ko 2023-10-16T14:05:43.458 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_hybla.ko 2023-10-16T14:05:43.458 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_illinois.ko 2023-10-16T14:05:43.458 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_lp.ko 2023-10-16T14:05:43.458 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_scalable.ko 2023-10-16T14:05:43.459 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_vegas.ko 2023-10-16T14:05:43.459 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_veno.ko 2023-10-16T14:05:43.459 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_westwood.ko 2023-10-16T14:05:43.459 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tcp_yeah.ko 2023-10-16T14:05:43.459 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/tunnel4.ko 2023-10-16T14:05:43.460 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_diag.ko 2023-10-16T14:05:43.460 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/udp_tunnel.ko 2023-10-16T14:05:43.460 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-16T14:05:43.460 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6 2023-10-16T14:05:43.460 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ah6.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/esp6.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/fou6.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ip6_vti.ko 2023-10-16T14:05:43.461 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/ipcomp6.ko 2023-10-16T14:05:43.462 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter 2023-10-16T14:05:43.462 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-16T14:05:43.462 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-16T14:05:43.462 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-16T14:05:43.462 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-16T14:05:43.463 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-16T14:05:43.464 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-16T14:05:43.465 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-16T14:05:43.466 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-16T14:05:43.466 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-16T14:05:43.466 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-16T14:05:43.466 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/sit.ko 2023-10-16T14:05:43.466 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/tunnel6.ko 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/key/af_key.ko 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_core.ko 2023-10-16T14:05:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-16T14:05:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_eth.ko 2023-10-16T14:05:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip.ko 2023-10-16T14:05:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-16T14:05:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-16T14:05:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-16T14:05:43.469 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc 2023-10-16T14:05:43.469 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/llc/llc.ko 2023-10-16T14:05:43.469 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154 2023-10-16T14:05:43.469 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mac802154/mac802154.ko 2023-10-16T14:05:43.469 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls 2023-10-16T14:05:43.470 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/mpls/mpls_gso.ko 2023-10-16T14:05:43.470 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter 2023-10-16T14:05:43.470 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset 2023-10-16T14:05:43.470 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-16T14:05:43.470 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-16T14:05:43.471 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-16T14:05:43.472 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-16T14:05:43.472 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-16T14:05:43.472 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-16T14:05:43.472 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-16T14:05:43.472 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-16T14:05:43.473 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-16T14:05:43.474 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-16T14:05:43.474 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-16T14:05:43.474 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-16T14:05:43.474 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-16T14:05:43.474 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-16T14:05:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-16T14:05:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-16T14:05:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-16T14:05:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-16T14:05:43.475 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-16T14:05:43.476 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-16T14:05:43.476 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-16T14:05:43.476 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conncount.ko 2023-10-16T14:05:43.476 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack.ko 2023-10-16T14:05:43.476 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-16T14:05:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-16T14:05:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-16T14:05:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-16T14:05:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-16T14:05:43.477 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-16T14:05:43.478 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-16T14:05:43.479 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-16T14:05:43.479 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table.ko 2023-10-16T14:05:43.479 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-16T14:05:43.479 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-16T14:05:43.479 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat.ko 2023-10-16T14:05:43.480 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-16T14:05:43.480 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-16T14:05:43.480 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-16T14:05:43.480 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-16T14:05:43.480 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-16T14:05:43.481 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-16T14:05:43.481 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nf_tables.ko 2023-10-16T14:05:43.481 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink.ko 2023-10-16T14:05:43.481 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-16T14:05:43.481 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-16T14:05:43.482 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-16T14:05:43.482 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-16T14:05:43.482 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-16T14:05:43.482 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_compat.ko 2023-10-16T14:05:43.482 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_ct.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fib.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_hash.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_limit.ko 2023-10-16T14:05:43.483 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_log.ko 2023-10-16T14:05:43.484 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_masq.ko 2023-10-16T14:05:43.484 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_nat.ko 2023-10-16T14:05:43.484 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_numgen.ko 2023-10-16T14:05:43.484 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_queue.ko 2023-10-16T14:05:43.484 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_quota.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_redir.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_socket.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/nft_tunnel.ko 2023-10-16T14:05:43.485 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-16T14:05:43.486 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-16T14:05:43.486 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-16T14:05:43.486 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-16T14:05:43.486 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_CT.ko 2023-10-16T14:05:43.486 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_DSCP.ko 2023-10-16T14:05:43.487 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HL.ko 2023-10-16T14:05:43.487 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_HMARK.ko 2023-10-16T14:05:43.487 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-16T14:05:43.487 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LED.ko 2023-10-16T14:05:43.487 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_LOG.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-16T14:05:43.488 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-16T14:05:43.489 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-16T14:05:43.489 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-16T14:05:43.489 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-16T14:05:43.489 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TEE.ko 2023-10-16T14:05:43.489 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_TRACE.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_addrtype.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_bpf.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cgroup.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cluster.ko 2023-10-16T14:05:43.490 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_comment.ko 2023-10-16T14:05:43.491 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connbytes.ko 2023-10-16T14:05:43.491 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlabel.ko 2023-10-16T14:05:43.491 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connlimit.ko 2023-10-16T14:05:43.491 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_connmark.ko 2023-10-16T14:05:43.492 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_conntrack.ko 2023-10-16T14:05:43.492 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_cpu.ko 2023-10-16T14:05:43.492 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dccp.ko 2023-10-16T14:05:43.492 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_devgroup.ko 2023-10-16T14:05:43.492 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_dscp.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ecn.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_esp.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_helper.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_hl.ko 2023-10-16T14:05:43.493 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-16T14:05:43.494 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_iprange.ko 2023-10-16T14:05:43.494 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_ipvs.ko 2023-10-16T14:05:43.494 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_l2tp.ko 2023-10-16T14:05:43.494 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_length.ko 2023-10-16T14:05:43.494 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_limit.ko 2023-10-16T14:05:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mac.ko 2023-10-16T14:05:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_mark.ko 2023-10-16T14:05:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_multiport.ko 2023-10-16T14:05:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nat.ko 2023-10-16T14:05:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_nfacct.ko 2023-10-16T14:05:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_osf.ko 2023-10-16T14:05:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_owner.ko 2023-10-16T14:05:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_physdev.ko 2023-10-16T14:05:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_pkttype.ko 2023-10-16T14:05:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_policy.ko 2023-10-16T14:05:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_quota.ko 2023-10-16T14:05:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_rateest.ko 2023-10-16T14:05:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_realm.ko 2023-10-16T14:05:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_recent.ko 2023-10-16T14:05:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_sctp.ko 2023-10-16T14:05:43.498 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_set.ko 2023-10-16T14:05:43.498 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_socket.ko 2023-10-16T14:05:43.498 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_state.ko 2023-10-16T14:05:43.498 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_statistic.ko 2023-10-16T14:05:43.498 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_string.ko 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_time.ko 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netfilter/xt_u32.ko 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/netlink/netlink_diag.ko 2023-10-16T14:05:43.499 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh 2023-10-16T14:05:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/nsh/nsh.ko 2023-10-16T14:05:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch 2023-10-16T14:05:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/openvswitch.ko 2023-10-16T14:05:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-geneve.ko 2023-10-16T14:05:43.500 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-gre.ko 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/packet/af_packet_diag.ko 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds.ko 2023-10-16T14:05:43.501 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_rdma.ko 2023-10-16T14:05:43.502 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/rds/rds_tcp.ko 2023-10-16T14:05:43.502 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched 2023-10-16T14:05:43.502 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_csum.ko 2023-10-16T14:05:43.502 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_gact.ko 2023-10-16T14:05:43.502 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_ipt.ko 2023-10-16T14:05:43.503 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_mirred.ko 2023-10-16T14:05:43.503 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_nat.ko 2023-10-16T14:05:43.503 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_pedit.ko 2023-10-16T14:05:43.503 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_police.ko 2023-10-16T14:05:43.503 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_simple.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_skbedit.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/act_vlan.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_basic.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_bpf.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_flow.ko 2023-10-16T14:05:43.504 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_fw.ko 2023-10-16T14:05:43.505 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_route.ko 2023-10-16T14:05:43.505 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/cls_u32.ko 2023-10-16T14:05:43.505 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_cmp.ko 2023-10-16T14:05:43.505 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_ipset.ko 2023-10-16T14:05:43.505 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_meta.ko 2023-10-16T14:05:43.506 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_nbyte.ko 2023-10-16T14:05:43.506 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_text.ko 2023-10-16T14:05:43.506 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/em_u32.ko 2023-10-16T14:05:43.506 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_choke.ko 2023-10-16T14:05:43.506 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_codel.ko 2023-10-16T14:05:43.507 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_drr.ko 2023-10-16T14:05:43.507 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_fq.ko 2023-10-16T14:05:43.507 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_gred.ko 2023-10-16T14:05:43.507 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hfsc.ko 2023-10-16T14:05:43.507 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_hhf.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_htb.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_ingress.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_multiq.ko 2023-10-16T14:05:43.508 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_netem.ko 2023-10-16T14:05:43.509 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_pie.ko 2023-10-16T14:05:43.509 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_plug.ko 2023-10-16T14:05:43.509 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_prio.ko 2023-10-16T14:05:43.509 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_qfq.ko 2023-10-16T14:05:43.509 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_red.ko 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfb.ko 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_sfq.ko 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_tbf.ko 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sched/sch_teql.ko 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp 2023-10-16T14:05:43.510 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp.ko 2023-10-16T14:05:43.511 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sctp/sctp_diag.ko 2023-10-16T14:05:43.511 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc 2023-10-16T14:05:43.511 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss 2023-10-16T14:05:43.511 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-16T14:05:43.511 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/sunrpc.ko 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/diag.ko 2023-10-16T14:05:43.512 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/tipc/tipc.ko 2023-10-16T14:05:43.513 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix 2023-10-16T14:05:43.513 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/unix/unix_diag.ko 2023-10-16T14:05:43.513 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock 2023-10-16T14:05:43.513 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-16T14:05:43.513 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-16T14:05:43.514 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock.ko 2023-10-16T14:05:43.514 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-16T14:05:43.514 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-16T14:05:43.514 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm 2023-10-16T14:05:43.514 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys 2023-10-16T14:05:43.515 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-16T14:05:43.516 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt 2023-10-16T14:05:43.516 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib 2023-10-16T14:05:43.516 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/kernel/virt/lib/irqbypass.ko 2023-10-16T14:05:43.516 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias 2023-10-16T14:05:43.516 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.alias.bin 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.bin 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.builtin.modinfo 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.dep.bin 2023-10-16T14:05:43.517 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.devname 2023-10-16T14:05:43.518 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.order 2023-10-16T14:05:43.518 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.softdep 2023-10-16T14:05:43.518 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols 2023-10-16T14:05:43.518 INFO:teuthology.orchestra.run.smithi178.stdout:/lib/modules/6.6.0-rc5-g47672119149b/modules.symbols.bin 2023-10-16T14:05:43.519 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.519 DEBUG:teuthology.orchestra.run.smithi178:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-16T14:05:43.546 INFO:teuthology.orchestra.run.smithi178.stdout:postinstall scriptlet (using /bin/sh): 2023-10-16T14:05:43.546 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g47672119149b -a -r /boot/System.map-6.6.0-rc5-g47672119149b ]; then 2023-10-16T14:05:43.546 INFO:teuthology.orchestra.run.smithi178.stdout:cp /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm 2023-10-16T14:05:43.546 INFO:teuthology.orchestra.run.smithi178.stdout:cp /boot/System.map-6.6.0-rc5-g47672119149b /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T14:05:43.547 INFO:teuthology.orchestra.run.smithi178.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g47672119149b /boot/System.map-6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.547 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/installkernel 6.6.0-rc5-g47672119149b /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T14:05:43.547 INFO:teuthology.orchestra.run.smithi178.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g47672119149b-rpm /boot/.System.map-6.6.0-rc5-g47672119149b-rpm 2023-10-16T14:05:43.547 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2023-10-16T14:05:43.547 INFO:teuthology.orchestra.run.smithi178.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-16T14:05:43.548 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-16T14:05:43.548 INFO:teuthology.orchestra.run.smithi178.stdout:new-kernel-pkg --remove 6.6.0-rc5-g47672119149b --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g47672119149b.img 2023-10-16T14:05:43.548 INFO:teuthology.orchestra.run.smithi178.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-16T14:05:43.548 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-install remove 6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.548 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2023-10-16T14:05:43.549 INFO:teuthology.orchestra.run.smithi178.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-16T14:05:43.549 INFO:teuthology.orchestra.run.smithi178.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-16T14:05:43.549 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.549 INFO:teuthology.orchestra.run.smithi178.stdout:fi 2023-10-16T14:05:43.550 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2023-10-16T14:05:43.627 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2023-10-16T14:05:43.627 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2023-10-16T14:05:43.627 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.02 2023-10-16T14:05:43.627 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 123.el8 2023-10-16T14:05:43.627 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2023-10-16T14:05:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-10-16T14:05:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:Group : System Environment/Base 2023-10-16T14:05:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 9265450 2023-10-16T14:05:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2023-10-16T14:05:43.628 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-10-16T14:05:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-10-16T14:05:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-10-16T14:05:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-10-16T14:05:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:Relocations : (not relocatable) 2023-10-16T14:05:43.629 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : Red Hat, Inc. 2023-10-16T14:05:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : Red Hat, Inc. 2023-10-16T14:05:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-16T14:05:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2023-10-16T14:05:43.630 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2023-10-16T14:05:43.631 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:05:43.632 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2023-10-16T14:05:43.633 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-16T14:05:43.633 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 6.6.0-rc5-g47672119149b 2023-10-16T14:05:43.633 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-16T14:05:48.344 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2023-10-16T14:05:50.516 INFO:teuthology.orchestra.run.smithi178.stderr:done 2023-10-16T14:05:50.517 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2023-10-16T14:05:50.534 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.9QtjimAj8c 2023-10-16T14:05:50.535 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9QtjimAj8c 2023-10-16T14:05:50.611 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.9QtjimAj8c 2023-10-16T14:05:50.757 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.9QtjimAj8c is 6KB 2023-10-16T14:05:50.805 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.9QtjimAj8c 2023-10-16T14:05:50.821 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2023-10-16T14:05:50.886 INFO:teuthology.orchestra.run.smithi178.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-10-16T14:05:50.886 INFO:teuthology.orchestra.run.smithi178.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-10-16T14:05:50.887 INFO:teuthology.orchestra.run.smithi178.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b.conf 2023-10-16T14:05:50.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-rc5-g47672119149b 2023-10-16T14:05:51.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2023-10-16T14:06:21.053 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-16T14:06:21.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:06:21.054 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:06:39.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-10-16T14:06:48.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:06:48.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:06:51.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-10-16T14:07:03.548 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:07:03.549 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:07:06.625 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2023-10-16T14:07:21.627 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:07:21.627 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:07:53.579 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-10-16T14:07:53.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:07:53.977 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g47672119149b"... 2023-10-16T14:07:53.978 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2023-10-16T14:07:54.021 INFO:teuthology.orchestra.run.smithi178.stdout:6.6.0-rc5-g47672119149b 2023-10-16T14:07:54.021 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g47672119149b vs 6.6.0-rc5-g47672119149b 2023-10-16T14:07:54.022 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T14:07:54.022 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-16T14:07:55.023 DEBUG:teuthology.parallel:result is None 2023-10-16T14:07:55.023 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T14:07:55.033 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T14:07:55.033 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T14:07:55.038 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T14:07:55.041 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T14:07:55.080 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T14:07:55.087 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T14:07:55.093 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T14:07:55.093 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T14:07:55.129 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T14:07:55.131 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T14:07:55.188 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T14:07:55.195 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T14:07:55.196 DEBUG:teuthology.orchestra.run.smithi186:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T14:07:55.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:07:55.217 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T14:07:55.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:07:55.240 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T14:07:55.244 DEBUG:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T14:07:55.260 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T14:07:55.295 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.315 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.321 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.343 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.421 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T14:07:55.424 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T14:07:55.432 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T14:07:55.432 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T14:07:55.436 DEBUG:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T14:07:55.441 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T14:07:55.500 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T14:07:55.509 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T14:07:55.509 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T14:07:55.514 DEBUG:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T14:07:55.516 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T14:07:55.552 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:55.668 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:55.755 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:55.826 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:55.912 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:07:55.912 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T14:07:55.980 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:56.025 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:56.083 DEBUG:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:56.171 DEBUG:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:56.214 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:07:56.215 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T14:07:56.284 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:56.329 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T14:07:56.393 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:56.437 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T14:07:56.483 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:07:56.483 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T14:07:56.552 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2023-10-16T14:07:56.557 DEBUG:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2023-10-16T14:07:56.559 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2023-10-16T14:07:56.626 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:07:56.651 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:07:56.678 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:07:57.147 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T14:07:57.155 INFO:teuthology.task.internal:Starting timer... 2023-10-16T14:07:57.156 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T14:07:57.172 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T14:07:57.180 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-16T14:07:57.181 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2023-10-16T14:07:57.317 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2023-10-16T14:07:57.321 DEBUG:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2023-10-16T14:07:57.372 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: 2023-10-16T14:07:57.374 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2023-10-16T14:07:57.435 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2023-10-16T14:07:57.436 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T14:07:57.437 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2023-10-16T14:07:57.491 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2023-10-16T14:07:57.491 DEBUG:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2023-10-16T14:07:57.521 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2023-10-16T14:07:57.522 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2023-10-16T14:07:57.548 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2023-10-16T14:07:57.548 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi178.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2023-10-16T14:07:57.549 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:07:57.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:07:57.619 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:07:57.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:07:57.649 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:07:57.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:07:57.676 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T14:07:57.677 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2023-10-16T14:07:57.745 DEBUG:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2023-10-16T14:07:57.783 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2023-10-16T14:07:57.808 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T14:07:57.818 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-16T14:07:57.819 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T14:07:57.838 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T14:07:57.839 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi178.front.sepia.ceph.com,smithi186.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T14:20:07.195 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi178.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2023-10-16T14:20:07.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:20:07.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:20:07.286 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-10-16T14:20:07.404 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-10-16T14:20:07.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T14:20:07.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi186.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:20:07.473 DEBUG:teuthology.orchestra.run.smithi186:> true 2023-10-16T14:20:07.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi186.front.sepia.ceph.com' 2023-10-16T14:20:07.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-10-16T14:20:07.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T14:20:07.614 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-10-16T14:20:07.689 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-10-16T14:20:07.689 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T14:20:07.700 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T14:20:07.701 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T14:20:07.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:20:07.706 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T14:20:07.706 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:20:07.709 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T14:20:07.709 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:20:07.741 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T14:20:07.757 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T14:20:07.762 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T14:20:07.775 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-16T14:20:07.777 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T14:20:07.810 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-16T14:20:07.822 INFO:teuthology.orchestra.run.smithi186.stderr:sudo: ntpd: command not found 2023-10-16T14:20:07.834 INFO:teuthology.orchestra.run.smithi186.stdout:506 Cannot talk to daemon 2023-10-16T14:20:07.841 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2023-10-16T14:20:07.849 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T14:20:07.853 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2023-10-16T14:20:07.863 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T14:20:07.867 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T14:20:07.881 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T14:20:07.903 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2023-10-16T14:20:07.925 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-16T14:20:07.929 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:20:07.929 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-16T14:20:07.930 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.930 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.930 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.930 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.932 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2023-10-16T14:20:07.966 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-16T14:20:07.980 INFO:teuthology.orchestra.run.smithi187.stderr:bash: ntpq: command not found 2023-10-16T14:20:07.983 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:20:07.983 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2023-10-16T14:20:07.984 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.984 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.984 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.984 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:07.999 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-16T14:20:08.092 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-10-16T14:20:08.099 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:20:08.100 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-10-16T14:20:08.100 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:08.100 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:08.100 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:08.100 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:20:08.101 INFO:teuthology.run_tasks:Running task pexec... 2023-10-16T14:20:08.110 INFO:teuthology.task.pexec:Executing custom commands... 2023-10-16T14:20:08.111 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:20:08.111 DEBUG:teuthology.orchestra.run.smithi178:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-16T14:20:08.112 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T14:20:08.112 DEBUG:teuthology.orchestra.run.smithi186:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-16T14:20:08.113 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi187.front.sepia.ceph.com 2023-10-16T14:20:08.113 DEBUG:teuthology.orchestra.run.smithi187:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-10-16T14:20:08.484 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:08.484 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:08.590 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:09.314 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:04:31 ago on Mon 16 Oct 2023 02:15:37 PM UTC. 2023-10-16T14:20:09.315 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:30 ago on Mon 16 Oct 2023 02:15:38 PM UTC. 2023-10-16T14:20:09.531 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Oct 2023 02:18:24 PM UTC. 2023-10-16T14:20:11.028 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-10-16T14:20:11.029 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-10-16T14:20:11.029 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2023-10-16T14:20:11.029 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-10-16T14:20:11.030 INFO:teuthology.orchestra.run.smithi187.stdout:Resetting modules: 2023-10-16T14:20:11.030 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools 2023-10-16T14:20:11.030 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:11.030 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-10-16T14:20:11.030 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-10-16T14:20:11.031 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:11.037 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-16T14:20:11.038 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-16T14:20:11.038 INFO:teuthology.orchestra.run.smithi186.stdout: Package Architecture Version Repository Size 2023-10-16T14:20:11.039 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-16T14:20:11.039 INFO:teuthology.orchestra.run.smithi186.stdout:Resetting modules: 2023-10-16T14:20:11.039 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools 2023-10-16T14:20:11.039 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:11.039 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-16T14:20:11.040 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2023-10-16T14:20:11.040 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:11.336 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-16T14:20:11.337 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-16T14:20:11.337 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2023-10-16T14:20:11.338 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-16T14:20:11.338 INFO:teuthology.orchestra.run.smithi178.stdout:Resetting modules: 2023-10-16T14:20:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools 2023-10-16T14:20:11.338 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:11.338 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-16T14:20:11.339 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2023-10-16T14:20:11.339 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:12.005 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:04:34 ago on Mon 16 Oct 2023 02:15:37 PM UTC. 2023-10-16T14:20:12.010 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:33 ago on Mon 16 Oct 2023 02:15:38 PM UTC. 2023-10-16T14:20:12.588 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Oct 2023 02:18:24 PM UTC. 2023-10-16T14:20:15.018 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-16T14:20:15.051 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-10-16T14:20:15.245 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-16T14:20:15.486 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:15.526 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:15.850 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:16.298 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:04:38 ago on Mon 16 Oct 2023 02:15:37 PM UTC. 2023-10-16T14:20:16.326 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:37 ago on Mon 16 Oct 2023 02:15:38 PM UTC. 2023-10-16T14:20:16.824 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:52 ago on Mon 16 Oct 2023 02:18:24 PM UTC. 2023-10-16T14:20:18.150 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-16T14:20:18.177 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-10-16T14:20:18.246 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-16T14:20:18.247 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-16T14:20:18.249 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-16T14:20:18.249 INFO:teuthology.orchestra.run.smithi186.stdout:Installing group/module packages: 2023-10-16T14:20:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-16T14:20:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-16T14:20:18.250 INFO:teuthology.orchestra.run.smithi186.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-16T14:20:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-16T14:20:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-16T14:20:18.251 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-16T14:20:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-16T14:20:18.252 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-16T14:20:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-16T14:20:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-16T14:20:18.253 INFO:teuthology.orchestra.run.smithi186.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-16T14:20:18.254 INFO:teuthology.orchestra.run.smithi186.stdout:Installing module profiles: 2023-10-16T14:20:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools/common 2023-10-16T14:20:18.254 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:18.255 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-16T14:20:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: container-tools rhel8 2023-10-16T14:20:18.255 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:18.256 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-16T14:20:18.256 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================================================ 2023-10-16T14:20:18.256 INFO:teuthology.orchestra.run.smithi186.stdout:Install 10 Packages 2023-10-16T14:20:18.257 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:18.258 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 18 M 2023-10-16T14:20:18.258 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 65 M 2023-10-16T14:20:18.259 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-16T14:20:18.268 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-10-16T14:20:18.268 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-10-16T14:20:18.268 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-10-16T14:20:18.268 INFO:teuthology.orchestra.run.smithi187.stdout:Installing group/module packages: 2023-10-16T14:20:18.269 INFO:teuthology.orchestra.run.smithi187.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-16T14:20:18.269 INFO:teuthology.orchestra.run.smithi187.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-16T14:20:18.269 INFO:teuthology.orchestra.run.smithi187.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-16T14:20:18.269 INFO:teuthology.orchestra.run.smithi187.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-16T14:20:18.269 INFO:teuthology.orchestra.run.smithi187.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-16T14:20:18.270 INFO:teuthology.orchestra.run.smithi187.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-16T14:20:18.270 INFO:teuthology.orchestra.run.smithi187.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-16T14:20:18.270 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-10-16T14:20:18.270 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-16T14:20:18.270 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout:Installing module profiles: 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools/common 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout:Enabling module streams: 2023-10-16T14:20:18.271 INFO:teuthology.orchestra.run.smithi187.stdout: container-tools rhel8 2023-10-16T14:20:18.272 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:18.272 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-10-16T14:20:18.272 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================================ 2023-10-16T14:20:18.272 INFO:teuthology.orchestra.run.smithi187.stdout:Install 10 Packages 2023-10-16T14:20:18.272 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:18.273 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 18 M 2023-10-16T14:20:18.273 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 65 M 2023-10-16T14:20:18.274 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-10-16T14:20:18.320 INFO:teuthology.orchestra.run.smithi186.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-10-16T14:20:18.346 INFO:teuthology.orchestra.run.smithi186.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 258 kB/s | 25 kB 00:00 2023-10-16T14:20:18.371 INFO:teuthology.orchestra.run.smithi186.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 336 kB/s | 41 kB 00:00 2023-10-16T14:20:18.396 INFO:teuthology.orchestra.run.smithi186.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-10-16T14:20:18.430 INFO:teuthology.orchestra.run.smithi186.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-16T14:20:18.433 INFO:teuthology.orchestra.run.smithi187.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 577 kB/s | 94 kB 00:00 2023-10-16T14:20:18.455 INFO:teuthology.orchestra.run.smithi186.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-16T14:20:18.458 INFO:teuthology.orchestra.run.smithi187.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 132 kB/s | 25 kB 00:00 2023-10-16T14:20:18.480 INFO:teuthology.orchestra.run.smithi186.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-16T14:20:18.484 INFO:teuthology.orchestra.run.smithi187.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 191 kB/s | 41 kB 00:00 2023-10-16T14:20:18.509 INFO:teuthology.orchestra.run.smithi187.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 653 kB/s | 49 kB 00:00 2023-10-16T14:20:18.543 INFO:teuthology.orchestra.run.smithi187.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-16T14:20:18.568 INFO:teuthology.orchestra.run.smithi187.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-10-16T14:20:18.593 INFO:teuthology.orchestra.run.smithi187.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-10-16T14:20:18.596 INFO:teuthology.orchestra.run.smithi186.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-10-16T14:20:18.601 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-16T14:20:18.698 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-16T14:20:18.699 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-10-16T14:20:18.699 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-16T14:20:18.699 INFO:teuthology.orchestra.run.smithi178.stdout:Installing group/module packages: 2023-10-16T14:20:18.699 INFO:teuthology.orchestra.run.smithi178.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-10-16T14:20:18.699 INFO:teuthology.orchestra.run.smithi178.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-10-16T14:20:18.700 INFO:teuthology.orchestra.run.smithi178.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-10-16T14:20:18.700 INFO:teuthology.orchestra.run.smithi178.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-10-16T14:20:18.700 INFO:teuthology.orchestra.run.smithi178.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-10-16T14:20:18.700 INFO:teuthology.orchestra.run.smithi178.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-10-16T14:20:18.700 INFO:teuthology.orchestra.run.smithi178.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-10-16T14:20:18.701 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-10-16T14:20:18.701 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-10-16T14:20:18.701 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-10-16T14:20:18.701 INFO:teuthology.orchestra.run.smithi178.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-10-16T14:20:18.701 INFO:teuthology.orchestra.run.smithi178.stdout:Installing module profiles: 2023-10-16T14:20:18.702 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools/common 2023-10-16T14:20:18.702 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:18.702 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling module streams: 2023-10-16T14:20:18.702 INFO:teuthology.orchestra.run.smithi178.stdout: container-tools rhel8 2023-10-16T14:20:18.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:18.703 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-16T14:20:18.703 INFO:teuthology.orchestra.run.smithi178.stdout:============================================================================================================ 2023-10-16T14:20:18.703 INFO:teuthology.orchestra.run.smithi178.stdout:Install 10 Packages 2023-10-16T14:20:18.703 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:18.704 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 18 M 2023-10-16T14:20:18.704 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 65 M 2023-10-16T14:20:18.705 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-10-16T14:20:18.710 INFO:teuthology.orchestra.run.smithi187.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 19 MB/s | 2.2 MB 00:00 2023-10-16T14:20:18.764 INFO:teuthology.orchestra.run.smithi186.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 23 MB/s | 7.0 MB 00:00 2023-10-16T14:20:18.784 INFO:teuthology.orchestra.run.smithi178.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.1 MB/s | 94 kB 00:00 2023-10-16T14:20:18.811 INFO:teuthology.orchestra.run.smithi178.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 226 kB/s | 25 kB 00:00 2023-10-16T14:20:18.819 INFO:teuthology.orchestra.run.smithi187.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-10-16T14:20:18.837 INFO:teuthology.orchestra.run.smithi178.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 301 kB/s | 41 kB 00:00 2023-10-16T14:20:18.856 INFO:teuthology.orchestra.run.smithi186.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-10-16T14:20:18.857 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:20:18.857 INFO:teuthology.orchestra.run.smithi186.stdout:Total 30 MB/s | 18 MB 00:00 2023-10-16T14:20:18.864 INFO:teuthology.orchestra.run.smithi178.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 638 kB/s | 49 kB 00:00 2023-10-16T14:20:18.896 INFO:teuthology.orchestra.run.smithi178.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-10-16T14:20:18.902 INFO:teuthology.orchestra.run.smithi187.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-10-16T14:20:18.903 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:20:18.903 INFO:teuthology.orchestra.run.smithi187.stdout:Total 29 MB/s | 18 MB 00:00 2023-10-16T14:20:18.946 INFO:teuthology.orchestra.run.smithi178.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.7 MB/s | 149 kB 00:00 2023-10-16T14:20:18.963 INFO:teuthology.orchestra.run.smithi178.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.1 MB/s | 213 kB 00:00 2023-10-16T14:20:19.058 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-16T14:20:19.072 INFO:teuthology.orchestra.run.smithi178.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-10-16T14:20:19.077 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-16T14:20:19.077 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-16T14:20:19.102 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-10-16T14:20:19.121 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-10-16T14:20:19.122 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-10-16T14:20:19.214 INFO:teuthology.orchestra.run.smithi178.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 26 MB/s | 7.0 MB 00:00 2023-10-16T14:20:19.259 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-16T14:20:19.262 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-16T14:20:19.302 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-10-16T14:20:19.306 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-10-16T14:20:19.331 INFO:teuthology.orchestra.run.smithi178.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-10-16T14:20:19.332 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:20:19.332 INFO:teuthology.orchestra.run.smithi178.stdout:Total 29 MB/s | 18 MB 00:00 2023-10-16T14:20:19.666 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-10-16T14:20:19.688 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-10-16T14:20:19.689 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-10-16T14:20:19.774 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-16T14:20:19.832 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-10-16T14:20:19.959 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-16T14:20:20.011 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-16T14:20:20.153 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:20.167 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:20.261 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-10-16T14:20:20.265 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-10-16T14:20:20.355 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-16T14:20:20.581 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-16T14:20:20.612 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-16T14:20:20.808 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-16T14:20:20.848 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-10-16T14:20:20.990 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-16T14:20:21.063 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-10-16T14:20:21.147 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-16T14:20:21.269 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:21.488 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-10-16T14:20:21.711 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-10-16T14:20:21.796 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-16T14:20:21.932 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-16T14:20:22.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-10-16T14:20:22.687 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-16T14:20:22.790 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-16T14:20:22.918 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-16T14:20:22.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-10-16T14:20:23.003 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-16T14:20:23.110 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-16T14:20:23.175 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-16T14:20:23.296 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:23.377 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:23.570 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:23.570 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-16T14:20:23.571 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-16T14:20:23.572 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-16T14:20:23.572 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-16T14:20:23.640 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:23.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-16T14:20:23.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:23.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-16T14:20:23.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-16T14:20:23.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-16T14:20:23.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-16T14:20:23.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-16T14:20:23.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-16T14:20:23.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-16T14:20:23.908 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-10-16T14:20:24.212 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-10-16T14:20:24.452 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-10-16T14:20:24.710 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:24.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-16T14:20:24.730 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:04:47 ago on Mon 16 Oct 2023 02:15:37 PM UTC. 2023-10-16T14:20:24.828 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-16T14:20:24.828 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:04:46 ago on Mon 16 Oct 2023 02:15:38 PM UTC. 2023-10-16T14:20:25.096 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-10-16T14:20:25.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-10-16T14:20:25.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-10-16T14:20:25.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-10-16T14:20:25.097 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-10-16T14:20:25.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-10-16T14:20:25.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-10-16T14:20:25.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-10-16T14:20:25.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-10-16T14:20:25.098 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-10-16T14:20:26.649 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-10-16T14:20:26.649 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:02 ago on Mon 16 Oct 2023 02:18:24 PM UTC. 2023-10-16T14:20:30.564 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-16T14:20:30.774 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:30.774 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-16T14:20:30.774 INFO:teuthology.orchestra.run.smithi186.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:30.775 INFO:teuthology.orchestra.run.smithi186.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:30.775 INFO:teuthology.orchestra.run.smithi186.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:30.775 INFO:teuthology.orchestra.run.smithi186.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-16T14:20:30.775 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-16T14:20:30.775 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-16T14:20:30.776 INFO:teuthology.orchestra.run.smithi186.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:30.776 INFO:teuthology.orchestra.run.smithi186.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:30.776 INFO:teuthology.orchestra.run.smithi186.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:30.776 INFO:teuthology.orchestra.run.smithi186.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-16T14:20:30.776 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:20:30.777 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-16T14:20:30.930 DEBUG:teuthology.parallel:result is None 2023-10-16T14:20:31.807 INFO:teuthology.orchestra.run.smithi187.stdout:Installed products updated. 2023-10-16T14:20:32.021 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:32.022 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-10-16T14:20:32.022 INFO:teuthology.orchestra.run.smithi187.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.022 INFO:teuthology.orchestra.run.smithi187.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.023 INFO:teuthology.orchestra.run.smithi187.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.024 INFO:teuthology.orchestra.run.smithi187.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:32.024 INFO:teuthology.orchestra.run.smithi187.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-16T14:20:32.024 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:20:32.024 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-10-16T14:20:32.194 DEBUG:teuthology.parallel:result is None 2023-10-16T14:20:32.372 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-10-16T14:20:32.587 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:32.588 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-10-16T14:20:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:32.588 INFO:teuthology.orchestra.run.smithi178.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-10-16T14:20:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-10-16T14:20:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-10-16T14:20:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.589 INFO:teuthology.orchestra.run.smithi178.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-10-16T14:20:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-10-16T14:20:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: yajl-2.1.0-10.el8.x86_64 2023-10-16T14:20:32.590 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:20:32.590 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-16T14:20:32.805 DEBUG:teuthology.parallel:result is None 2023-10-16T14:20:32.806 INFO:teuthology.run_tasks:Running task install... 2023-10-16T14:20:32.818 DEBUG:teuthology.task.install:project ceph 2023-10-16T14:20:32.819 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'}} 2023-10-16T14:20:32.819 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0'} 2023-10-16T14:20:32.819 INFO:teuthology.task.install:Using flavor: default 2023-10-16T14:20:32.826 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T14:20:32.826 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T14:20:32.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'tag': None, 'wait_for_package': False} 2023-10-16T14:20:32.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:20:32.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'tag': None, 'wait_for_package': False} 2023-10-16T14:20:32.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:20:32.831 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'tag': None, 'wait_for_package': False} 2023-10-16T14:20:32.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:20:32.985 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/ 2023-10-16T14:20:32.985 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6730.g4ba8a190 2023-10-16T14:20:33.080 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/ 2023-10-16T14:20:33.080 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6730.g4ba8a190 2023-10-16T14:20:33.085 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/ 2023-10-16T14:20:33.085 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6730.g4ba8a190 2023-10-16T14:20:33.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T14:20:33.099 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:20:33.099 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T14:20:33.132 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T14:20:33.132 DEBUG:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T14:20:33.190 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T14:20:33.191 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:20:33.191 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T14:20:33.202 DEBUG:teuthology.orchestra.run.smithi186:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T14:20:33.259 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T14:20:33.260 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T14:20:33.277 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-16T14:20:33.278 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:20:33.278 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-16T14:20:33.281 DEBUG:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T14:20:33.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-10-16T14:20:33.306 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-16T14:20:33.348 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2023-10-16T14:20:33.351 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum clean all 2023-10-16T14:20:33.369 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T14:20:33.373 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-16T14:20:33.451 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T14:20:33.464 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-16T14:20:33.517 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2023-10-16T14:20:33.518 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2023-10-16T14:20:33.531 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2023-10-16T14:20:33.534 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2023-10-16T14:20:33.808 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:33.979 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:33.979 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:34.825 INFO:teuthology.orchestra.run.smithi186.stdout:44 files removed 2023-10-16T14:20:34.878 DEBUG:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T14:20:34.996 INFO:teuthology.orchestra.run.smithi187.stdout:44 files removed 2023-10-16T14:20:35.049 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T14:20:35.117 INFO:teuthology.orchestra.run.smithi178.stdout:44 files removed 2023-10-16T14:20:35.175 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-10-16T14:20:35.223 INFO:teuthology.orchestra.run.smithi186.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:35.406 INFO:teuthology.orchestra.run.smithi187.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:35.618 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-10-16T14:20:36.790 INFO:teuthology.orchestra.run.smithi186.stdout:ceph packages for x86_64 182 kB/s | 81 kB 00:00 2023-10-16T14:20:36.886 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 197 kB/s | 81 kB 00:00 2023-10-16T14:20:37.108 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 203 kB/s | 81 kB 00:00 2023-10-16T14:20:37.285 INFO:teuthology.orchestra.run.smithi186.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-10-16T14:20:37.356 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 22 kB/s | 9.4 kB 00:00 2023-10-16T14:20:37.383 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 39 kB/s | 9.4 kB 00:00 2023-10-16T14:20:37.654 INFO:teuthology.orchestra.run.smithi186.stdout:ceph source packages 4.3 kB/s | 1.5 kB 00:00 2023-10-16T14:20:37.708 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-10-16T14:20:37.768 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2023-10-16T14:20:37.897 INFO:teuthology.orchestra.run.smithi186.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-10-16T14:20:37.985 INFO:teuthology.orchestra.run.smithi187.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-10-16T14:20:38.127 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2023-10-16T14:20:39.558 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-10-16T14:20:39.980 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2023-10-16T14:20:40.661 INFO:teuthology.orchestra.run.smithi186.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 16 MB 00:02 2023-10-16T14:20:43.123 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 455 kB/s | 24 kB 00:00 2023-10-16T14:20:44.058 INFO:teuthology.orchestra.run.smithi187.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-10-16T14:20:44.069 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 388 kB/s | 24 kB 00:00 2023-10-16T14:20:44.159 INFO:teuthology.orchestra.run.smithi186.stdout:lab-extras 444 kB/s | 24 kB 00:00 2023-10-16T14:20:45.044 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-10-16T14:20:45.255 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-10-16T14:20:52.734 INFO:teuthology.orchestra.run.smithi187.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 MB/s | 53 MB 00:01 2023-10-16T14:20:53.119 INFO:teuthology.orchestra.run.smithi186.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 62 MB/s | 47 MB 00:00 2023-10-16T14:20:53.938 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 MB/s | 47 MB 00:01 2023-10-16T14:21:03.882 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:03.883 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:03.897 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:03.898 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:03.899 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-16T14:21:03.900 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-16T14:21:03.903 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-16T14:21:03.906 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-16T14:21:03.907 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-16T14:21:04.038 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-10-16T14:21:04.050 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-10-16T14:21:04.050 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-10-16T14:21:04.051 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-10-16T14:21:04.051 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-10-16T14:21:04.051 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-16T14:21:04.051 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-16T14:21:04.052 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.4 k 2023-10-16T14:21:04.052 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 5.4 M 2023-10-16T14:21:04.052 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 918 k 2023-10-16T14:21:04.052 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 167 k 2023-10-16T14:21:04.052 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 1.6 M 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 132 k 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 1.8 M 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 7.4 M 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 47 k 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 14 M 2023-10-16T14:21:04.053 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 49 M 2023-10-16T14:21:04.054 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 261 k 2023-10-16T14:21:04.054 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 192 k 2023-10-16T14:21:04.054 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 261 k 2023-10-16T14:21:04.054 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 18 k 2023-10-16T14:21:04.055 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-16T14:21:04.055 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-16T14:21:04.055 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-16T14:21:04.055 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 31 k 2023-10-16T14:21:04.055 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 839 k 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 139 k 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-16T14:21:04.056 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-16T14:21:04.057 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-16T14:21:04.057 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 197 k 2023-10-16T14:21:04.057 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 371 k 2023-10-16T14:21:04.057 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 365 k 2023-10-16T14:21:04.057 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 107 k 2023-10-16T14:21:04.058 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 94 k 2023-10-16T14:21:04.058 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.6 M 2023-10-16T14:21:04.058 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 179 k 2023-10-16T14:21:04.058 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-16T14:21:04.059 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2023-10-16T14:21:04.059 INFO:teuthology.orchestra.run.smithi187.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-16T14:21:04.059 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:04.059 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:04.059 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-10-16T14:21:04.060 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-16T14:21:04.060 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 23 M 2023-10-16T14:21:04.060 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 23 k 2023-10-16T14:21:04.060 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 2.3 M 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 242 k 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 4.6 M 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 18 M 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 15 k 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 24 k 2023-10-16T14:21:04.061 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-16T14:21:04.062 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-16T14:21:04.062 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-16T14:21:04.062 INFO:teuthology.orchestra.run.smithi187.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-16T14:21:04.062 INFO:teuthology.orchestra.run.smithi187.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-16T14:21:04.062 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 184 k 2023-10-16T14:21:04.063 INFO:teuthology.orchestra.run.smithi187.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-16T14:21:04.063 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-16T14:21:04.063 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-16T14:21:04.063 INFO:teuthology.orchestra.run.smithi187.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-16T14:21:04.063 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-16T14:21:04.064 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-16T14:21:04.064 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 518 k 2023-10-16T14:21:04.064 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-16T14:21:04.064 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.9 M 2023-10-16T14:21:04.064 INFO:teuthology.orchestra.run.smithi187.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-16T14:21:04.065 INFO:teuthology.orchestra.run.smithi187.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-16T14:21:04.065 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-16T14:21:04.065 INFO:teuthology.orchestra.run.smithi187.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-16T14:21:04.065 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-16T14:21:04.065 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-16T14:21:04.066 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-16T14:21:04.066 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-16T14:21:04.066 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-16T14:21:04.066 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-16T14:21:04.066 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-16T14:21:04.067 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-16T14:21:04.067 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-16T14:21:04.067 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-16T14:21:04.067 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-16T14:21:04.067 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-16T14:21:04.068 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 45 k 2023-10-16T14:21:04.068 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 126 k 2023-10-16T14:21:04.068 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-16T14:21:04.068 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-16T14:21:04.068 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-16T14:21:04.069 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-16T14:21:04.069 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-16T14:21:04.069 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-16T14:21:04.069 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-16T14:21:04.069 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-16T14:21:04.070 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-16T14:21:04.070 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-16T14:21:04.070 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-16T14:21:04.070 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:04.070 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-16T14:21:04.071 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-16T14:21:04.071 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-16T14:21:04.071 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-16T14:21:04.071 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-16T14:21:04.071 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-16T14:21:04.072 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-16T14:21:04.073 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-16T14:21:04.073 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-16T14:21:04.073 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-16T14:21:04.073 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-16T14:21:04.073 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-16T14:21:04.074 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-16T14:21:04.074 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:04.074 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-16T14:21:04.074 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-16T14:21:04.074 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-16T14:21:04.075 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-16T14:21:04.075 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-16T14:21:04.075 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-16T14:21:04.075 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-16T14:21:04.075 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-16T14:21:04.076 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-16T14:21:04.076 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-16T14:21:04.076 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-16T14:21:04.076 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-16T14:21:04.076 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-16T14:21:04.077 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-16T14:21:04.077 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-16T14:21:04.077 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-16T14:21:04.077 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-16T14:21:04.077 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-16T14:21:04.078 INFO:teuthology.orchestra.run.smithi187.stdout:Enabling module streams: 2023-10-16T14:21:04.078 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql 10 2023-10-16T14:21:04.078 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:21:04.078 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-10-16T14:21:04.078 INFO:teuthology.orchestra.run.smithi187.stdout:=========================================================================================================================== 2023-10-16T14:21:04.079 INFO:teuthology.orchestra.run.smithi187.stdout:Install 128 Packages 2023-10-16T14:21:04.079 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 3 Packages 2023-10-16T14:21:04.079 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:21:04.080 INFO:teuthology.orchestra.run.smithi186.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:04.080 INFO:teuthology.orchestra.run.smithi186.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:04.080 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 209 M 2023-10-16T14:21:04.081 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-10-16T14:21:04.091 INFO:teuthology.orchestra.run.smithi186.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:04.092 INFO:teuthology.orchestra.run.smithi186.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:04.093 INFO:teuthology.orchestra.run.smithi186.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-16T14:21:04.094 INFO:teuthology.orchestra.run.smithi186.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-16T14:21:04.096 INFO:teuthology.orchestra.run.smithi186.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-16T14:21:04.099 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-16T14:21:04.100 INFO:teuthology.orchestra.run.smithi186.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-16T14:21:04.233 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies resolved. 2023-10-16T14:21:04.245 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-16T14:21:04.245 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2023-10-16T14:21:04.246 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-16T14:21:04.246 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2023-10-16T14:21:04.246 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-16T14:21:04.246 INFO:teuthology.orchestra.run.smithi186.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-16T14:21:04.246 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.4 k 2023-10-16T14:21:04.247 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 5.4 M 2023-10-16T14:21:04.247 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 918 k 2023-10-16T14:21:04.247 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 167 k 2023-10-16T14:21:04.247 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 1.6 M 2023-10-16T14:21:04.248 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 132 k 2023-10-16T14:21:04.248 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 1.8 M 2023-10-16T14:21:04.248 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 7.4 M 2023-10-16T14:21:04.248 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 47 k 2023-10-16T14:21:04.249 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 14 M 2023-10-16T14:21:04.249 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 49 M 2023-10-16T14:21:04.249 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 261 k 2023-10-16T14:21:04.249 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 192 k 2023-10-16T14:21:04.249 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 261 k 2023-10-16T14:21:04.250 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 18 k 2023-10-16T14:21:04.250 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-16T14:21:04.250 INFO:teuthology.orchestra.run.smithi186.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-16T14:21:04.250 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-16T14:21:04.250 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 31 k 2023-10-16T14:21:04.251 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 839 k 2023-10-16T14:21:04.251 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 139 k 2023-10-16T14:21:04.251 INFO:teuthology.orchestra.run.smithi186.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-16T14:21:04.251 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-16T14:21:04.251 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-16T14:21:04.252 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-16T14:21:04.252 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-16T14:21:04.252 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-16T14:21:04.252 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 197 k 2023-10-16T14:21:04.252 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 371 k 2023-10-16T14:21:04.253 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 365 k 2023-10-16T14:21:04.253 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 107 k 2023-10-16T14:21:04.253 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 94 k 2023-10-16T14:21:04.253 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.6 M 2023-10-16T14:21:04.253 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 179 k 2023-10-16T14:21:04.254 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-16T14:21:04.254 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrading: 2023-10-16T14:21:04.254 INFO:teuthology.orchestra.run.smithi186.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-16T14:21:04.254 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:04.254 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:04.255 INFO:teuthology.orchestra.run.smithi186.stdout:Installing dependencies: 2023-10-16T14:21:04.255 INFO:teuthology.orchestra.run.smithi186.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-16T14:21:04.255 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 23 M 2023-10-16T14:21:04.255 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 23 k 2023-10-16T14:21:04.255 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 2.3 M 2023-10-16T14:21:04.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 242 k 2023-10-16T14:21:04.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 4.6 M 2023-10-16T14:21:04.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 18 M 2023-10-16T14:21:04.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 15 k 2023-10-16T14:21:04.256 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 24 k 2023-10-16T14:21:04.257 INFO:teuthology.orchestra.run.smithi186.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-16T14:21:04.257 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-16T14:21:04.257 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-16T14:21:04.257 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-16T14:21:04.257 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 184 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-16T14:21:04.258 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-16T14:21:04.259 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-16T14:21:04.259 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 518 k 2023-10-16T14:21:04.259 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-16T14:21:04.259 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.9 M 2023-10-16T14:21:04.260 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-16T14:21:04.260 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-16T14:21:04.260 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-16T14:21:04.260 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-16T14:21:04.260 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-16T14:21:04.261 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-16T14:21:04.262 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-16T14:21:04.262 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-16T14:21:04.262 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-16T14:21:04.262 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-16T14:21:04.262 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-16T14:21:04.263 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 45 k 2023-10-16T14:21:04.263 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 126 k 2023-10-16T14:21:04.263 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-16T14:21:04.263 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-16T14:21:04.263 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-16T14:21:04.264 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-16T14:21:04.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-16T14:21:04.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-16T14:21:04.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:04.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-16T14:21:04.265 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-16T14:21:04.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-16T14:21:04.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-16T14:21:04.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-16T14:21:04.266 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-16T14:21:04.267 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-16T14:21:04.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-16T14:21:04.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-16T14:21:04.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-16T14:21:04.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-16T14:21:04.268 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-16T14:21:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-16T14:21:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-16T14:21:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-16T14:21:04.269 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-16T14:21:04.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-16T14:21:04.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-16T14:21:04.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-16T14:21:04.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-16T14:21:04.270 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-16T14:21:04.271 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-16T14:21:04.272 INFO:teuthology.orchestra.run.smithi186.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-16T14:21:04.272 INFO:teuthology.orchestra.run.smithi186.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-16T14:21:04.272 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-16T14:21:04.272 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-16T14:21:04.272 INFO:teuthology.orchestra.run.smithi186.stdout:Enabling module streams: 2023-10-16T14:21:04.273 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql 10 2023-10-16T14:21:04.273 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:21:04.273 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2023-10-16T14:21:04.273 INFO:teuthology.orchestra.run.smithi186.stdout:=========================================================================================================================== 2023-10-16T14:21:04.273 INFO:teuthology.orchestra.run.smithi186.stdout:Install 128 Packages 2023-10-16T14:21:04.274 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade 3 Packages 2023-10-16T14:21:04.274 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:21:04.274 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 209 M 2023-10-16T14:21:04.275 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading Packages: 2023-10-16T14:21:04.518 INFO:teuthology.orchestra.run.smithi187.stdout:(1/131): ceph-18.0.0-6730.g4ba8a190.el8.x86_64. 16 kB/s | 6.4 kB 00:00 2023-10-16T14:21:04.596 INFO:teuthology.orchestra.run.smithi186.stdout:(1/131): ceph-18.0.0-6730.g4ba8a190.el8.x86_64. 24 kB/s | 6.4 kB 00:00 2023-10-16T14:21:05.465 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:05.466 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-16T14:21:05.481 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:05.482 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-10-16T14:21:05.483 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-10-16T14:21:05.484 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-16T14:21:05.487 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-16T14:21:05.490 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-16T14:21:05.492 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-16T14:21:05.629 INFO:teuthology.orchestra.run.smithi186.stdout:(2/131): ceph-fuse-18.0.0-6730.g4ba8a190.el8.x8 889 kB/s | 918 kB 00:01 2023-10-16T14:21:05.630 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-10-16T14:21:05.643 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-16T14:21:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-10-16T14:21:05.643 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-16T14:21:05.643 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-10-16T14:21:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-10-16T14:21:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-16T14:21:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.4 k 2023-10-16T14:21:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 5.4 M 2023-10-16T14:21:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 918 k 2023-10-16T14:21:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 167 k 2023-10-16T14:21:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 1.6 M 2023-10-16T14:21:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 132 k 2023-10-16T14:21:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 1.8 M 2023-10-16T14:21:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 7.4 M 2023-10-16T14:21:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 47 k 2023-10-16T14:21:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 14 M 2023-10-16T14:21:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 49 M 2023-10-16T14:21:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 261 k 2023-10-16T14:21:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 192 k 2023-10-16T14:21:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 261 k 2023-10-16T14:21:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 18 k 2023-10-16T14:21:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-10-16T14:21:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-16T14:21:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-16T14:21:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 31 k 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 839 k 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 139 k 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-16T14:21:05.648 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-10-16T14:21:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-10-16T14:21:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-10-16T14:21:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 197 k 2023-10-16T14:21:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 371 k 2023-10-16T14:21:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 365 k 2023-10-16T14:21:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 107 k 2023-10-16T14:21:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 94 k 2023-10-16T14:21:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.6 M 2023-10-16T14:21:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 179 k 2023-10-16T14:21:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-10-16T14:21:05.651 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2023-10-16T14:21:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-10-16T14:21:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 3.7 M 2023-10-16T14:21:05.651 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-10-16T14:21:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-16T14:21:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 23 M 2023-10-16T14:21:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 23 k 2023-10-16T14:21:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 2.3 M 2023-10-16T14:21:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 242 k 2023-10-16T14:21:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 4.6 M 2023-10-16T14:21:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 18 M 2023-10-16T14:21:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6730.g4ba8a190.el8 ceph-noarch 15 k 2023-10-16T14:21:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 24 k 2023-10-16T14:21:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 184 k 2023-10-16T14:21:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-10-16T14:21:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-10-16T14:21:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-16T14:21:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-10-16T14:21:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-10-16T14:21:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-16T14:21:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 518 k 2023-10-16T14:21:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-10-16T14:21:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 6.9 M 2023-10-16T14:21:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-16T14:21:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-10-16T14:21:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-16T14:21:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-16T14:21:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-16T14:21:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-16T14:21:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-16T14:21:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-10-16T14:21:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-16T14:21:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-16T14:21:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-16T14:21:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-16T14:21:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-16T14:21:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-16T14:21:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-16T14:21:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-16T14:21:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 45 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:18.0.0-6730.g4ba8a190.el8 ceph 126 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-16T14:21:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-16T14:21:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-16T14:21:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-16T14:21:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-16T14:21:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-16T14:21:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-16T14:21:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-16T14:21:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-16T14:21:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-16T14:21:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-16T14:21:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-16T14:21:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-16T14:21:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-16T14:21:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-16T14:21:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-10-16T14:21:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-16T14:21:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-16T14:21:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-10-16T14:21:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-16T14:21:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-16T14:21:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-16T14:21:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-16T14:21:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-16T14:21:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-10-16T14:21:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-16T14:21:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-16T14:21:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-16T14:21:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-16T14:21:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-16T14:21:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-10-16T14:21:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-16T14:21:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-10-16T14:21:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-16T14:21:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-16T14:21:05.669 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling module streams: 2023-10-16T14:21:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql 10 2023-10-16T14:21:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:21:05.670 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-10-16T14:21:05.670 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-10-16T14:21:05.670 INFO:teuthology.orchestra.run.smithi178.stdout:Install 128 Packages 2023-10-16T14:21:05.670 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 3 Packages 2023-10-16T14:21:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:21:05.671 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 209 M 2023-10-16T14:21:05.671 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-10-16T14:21:05.693 INFO:teuthology.orchestra.run.smithi187.stdout:(2/131): ceph-fuse-18.0.0-6730.g4ba8a190.el8.x8 782 kB/s | 918 kB 00:01 2023-10-16T14:21:05.846 INFO:teuthology.orchestra.run.smithi186.stdout:(3/131): ceph-immutable-object-cache-18.0.0-673 771 kB/s | 167 kB 00:00 2023-10-16T14:21:05.960 INFO:teuthology.orchestra.run.smithi187.stdout:(3/131): ceph-immutable-object-cache-18.0.0-673 626 kB/s | 167 kB 00:00 2023-10-16T14:21:06.197 INFO:teuthology.orchestra.run.smithi178.stdout:(1/131): ceph-18.0.0-6730.g4ba8a190.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-10-16T14:21:06.210 INFO:teuthology.orchestra.run.smithi187.stdout:(4/131): ceph-base-18.0.0-6730.g4ba8a190.el8.x8 2.6 MB/s | 5.4 MB 00:02 2023-10-16T14:21:06.602 INFO:teuthology.orchestra.run.smithi187.stdout:(5/131): ceph-mgr-18.0.0-6730.g4ba8a190.el8.x86 4.1 MB/s | 1.6 MB 00:00 2023-10-16T14:21:07.389 INFO:teuthology.orchestra.run.smithi178.stdout:(2/131): ceph-fuse-18.0.0-6730.g4ba8a190.el8.x8 771 kB/s | 918 kB 00:01 2023-10-16T14:21:07.589 INFO:teuthology.orchestra.run.smithi178.stdout:(3/131): ceph-immutable-object-cache-18.0.0-673 837 kB/s | 167 kB 00:00 2023-10-16T14:21:07.744 INFO:teuthology.orchestra.run.smithi187.stdout:(6/131): ceph-mon-18.0.0-6730.g4ba8a190.el8.x86 4.0 MB/s | 4.6 MB 00:01 2023-10-16T14:21:07.847 INFO:teuthology.orchestra.run.smithi186.stdout:(4/131): ceph-mds-18.0.0-6730.g4ba8a190.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-10-16T14:21:09.152 INFO:teuthology.orchestra.run.smithi187.stdout:(7/131): ceph-mds-18.0.0-6730.g4ba8a190.el8.x86 741 kB/s | 2.3 MB 00:03 2023-10-16T14:21:09.498 INFO:teuthology.orchestra.run.smithi178.stdout:(4/131): ceph-mds-18.0.0-6730.g4ba8a190.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-10-16T14:21:09.630 INFO:teuthology.orchestra.run.smithi186.stdout:(5/131): ceph-mgr-18.0.0-6730.g4ba8a190.el8.x86 933 kB/s | 1.6 MB 00:01 2023-10-16T14:21:09.836 INFO:teuthology.orchestra.run.smithi187.stdout:(8/131): ceph-common-18.0.0-6730.g4ba8a190.el8. 3.9 MB/s | 23 MB 00:05 2023-10-16T14:21:09.928 INFO:teuthology.orchestra.run.smithi187.stdout:(9/131): ceph-selinux-18.0.0-6730.g4ba8a190.el8 266 kB/s | 24 kB 00:00 2023-10-16T14:21:10.330 INFO:teuthology.orchestra.run.smithi186.stdout:(6/131): ceph-base-18.0.0-6730.g4ba8a190.el8.x8 921 kB/s | 5.4 MB 00:06 2023-10-16T14:21:10.473 INFO:teuthology.orchestra.run.smithi178.stdout:(5/131): ceph-base-18.0.0-6730.g4ba8a190.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-10-16T14:21:10.623 INFO:teuthology.orchestra.run.smithi178.stdout:(6/131): ceph-mgr-18.0.0-6730.g4ba8a190.el8.x86 1.4 MB/s | 1.6 MB 00:01 2023-10-16T14:21:11.536 INFO:teuthology.orchestra.run.smithi187.stdout:(10/131): ceph-osd-18.0.0-6730.g4ba8a190.el8.x8 4.7 MB/s | 18 MB 00:03 2023-10-16T14:21:11.645 INFO:teuthology.orchestra.run.smithi187.stdout:(11/131): cephfs-mirror-18.0.0-6730.g4ba8a190.e 2.4 MB/s | 261 kB 00:00 2023-10-16T14:21:11.745 INFO:teuthology.orchestra.run.smithi187.stdout:(12/131): libcephfs-devel-18.0.0-6730.g4ba8a190 310 kB/s | 31 kB 00:00 2023-10-16T14:21:12.045 INFO:teuthology.orchestra.run.smithi187.stdout:(13/131): libcephfs2-18.0.0-6730.g4ba8a190.el8. 2.7 MB/s | 839 kB 00:00 2023-10-16T14:21:12.162 INFO:teuthology.orchestra.run.smithi187.stdout:(14/131): libcephsqlite-18.0.0-6730.g4ba8a190.e 1.5 MB/s | 184 kB 00:00 2023-10-16T14:21:12.271 INFO:teuthology.orchestra.run.smithi187.stdout:(15/131): librados-devel-18.0.0-6730.g4ba8a190. 1.3 MB/s | 139 kB 00:00 2023-10-16T14:21:12.479 INFO:teuthology.orchestra.run.smithi187.stdout:(16/131): libradosstriper1-18.0.0-6730.g4ba8a19 2.4 MB/s | 518 kB 00:00 2023-10-16T14:21:13.022 INFO:teuthology.orchestra.run.smithi186.stdout:(7/131): ceph-mon-18.0.0-6730.g4ba8a190.el8.x86 1.4 MB/s | 4.6 MB 00:03 2023-10-16T14:21:13.906 INFO:teuthology.orchestra.run.smithi178.stdout:(7/131): ceph-mon-18.0.0-6730.g4ba8a190.el8.x86 1.3 MB/s | 4.6 MB 00:03 2023-10-16T14:21:14.063 INFO:teuthology.orchestra.run.smithi187.stdout:(17/131): librgw2-18.0.0-6730.g4ba8a190.el8.x86 4.4 MB/s | 6.9 MB 00:01 2023-10-16T14:21:14.163 INFO:teuthology.orchestra.run.smithi187.stdout:(18/131): python3-ceph-argparse-18.0.0-6730.g4b 448 kB/s | 45 kB 00:00 2023-10-16T14:21:14.272 INFO:teuthology.orchestra.run.smithi187.stdout:(19/131): python3-ceph-common-18.0.0-6730.g4ba8 1.1 MB/s | 126 kB 00:00 2023-10-16T14:21:14.380 INFO:teuthology.orchestra.run.smithi187.stdout:(20/131): python3-cephfs-18.0.0-6730.g4ba8a190. 1.8 MB/s | 197 kB 00:00 2023-10-16T14:21:14.497 INFO:teuthology.orchestra.run.smithi187.stdout:(21/131): python3-rados-18.0.0-6730.g4ba8a190.e 3.1 MB/s | 371 kB 00:00 2023-10-16T14:21:14.623 INFO:teuthology.orchestra.run.smithi187.stdout:(22/131): python3-rbd-18.0.0-6730.g4ba8a190.el8 2.9 MB/s | 365 kB 00:00 2023-10-16T14:21:14.731 INFO:teuthology.orchestra.run.smithi187.stdout:(23/131): python3-rgw-18.0.0-6730.g4ba8a190.el8 987 kB/s | 107 kB 00:00 2023-10-16T14:21:14.848 INFO:teuthology.orchestra.run.smithi187.stdout:(24/131): rbd-fuse-18.0.0-6730.g4ba8a190.el8.x8 802 kB/s | 94 kB 00:00 2023-10-16T14:21:15.748 INFO:teuthology.orchestra.run.smithi187.stdout:(25/131): rbd-mirror-18.0.0-6730.g4ba8a190.el8. 4.0 MB/s | 3.6 MB 00:00 2023-10-16T14:21:15.857 INFO:teuthology.orchestra.run.smithi187.stdout:(26/131): rbd-nbd-18.0.0-6730.g4ba8a190.el8.x86 1.6 MB/s | 179 kB 00:00 2023-10-16T14:21:15.987 INFO:teuthology.orchestra.run.smithi187.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6730.g 177 kB/s | 23 kB 00:00 2023-10-16T14:21:16.091 INFO:teuthology.orchestra.run.smithi187.stdout:(28/131): ceph-mgr-cephadm-18.0.0-6730.g4ba8a19 1.3 MB/s | 132 kB 00:00 2023-10-16T14:21:16.591 INFO:teuthology.orchestra.run.smithi187.stdout:(29/131): ceph-mgr-dashboard-18.0.0-6730.g4ba8a 3.5 MB/s | 1.8 MB 00:00 2023-10-16T14:21:17.355 INFO:teuthology.orchestra.run.smithi186.stdout:(8/131): ceph-common-18.0.0-6730.g4ba8a190.el8. 1.7 MB/s | 23 MB 00:13 2023-10-16T14:21:17.447 INFO:teuthology.orchestra.run.smithi186.stdout:(9/131): ceph-selinux-18.0.0-6730.g4ba8a190.el8 266 kB/s | 24 kB 00:00 2023-10-16T14:21:18.214 INFO:teuthology.orchestra.run.smithi187.stdout:(30/131): ceph-radosgw-18.0.0-6730.g4ba8a190.el 1.5 MB/s | 14 MB 00:09 2023-10-16T14:21:18.357 INFO:teuthology.orchestra.run.smithi187.stdout:(31/131): ceph-mgr-diskprediction-local-18.0.0- 4.2 MB/s | 7.4 MB 00:01 2023-10-16T14:21:18.392 INFO:teuthology.orchestra.run.smithi187.stdout:(32/131): ceph-mgr-modules-core-18.0.0-6730.g4b 1.3 MB/s | 242 kB 00:00 2023-10-16T14:21:18.442 INFO:teuthology.orchestra.run.smithi187.stdout:(33/131): ceph-mgr-rook-18.0.0-6730.g4ba8a190.e 560 kB/s | 47 kB 00:00 2023-10-16T14:21:18.476 INFO:teuthology.orchestra.run.smithi187.stdout:(34/131): ceph-prometheus-alerts-18.0.0-6730.g4 179 kB/s | 15 kB 00:00 2023-10-16T14:21:18.534 INFO:teuthology.orchestra.run.smithi187.stdout:(35/131): ceph-volume-18.0.0-6730.g4ba8a190.el8 2.8 MB/s | 261 kB 00:00 2023-10-16T14:21:18.576 INFO:teuthology.orchestra.run.smithi187.stdout:(36/131): cephadm-18.0.0-6730.g4ba8a190.el8.noa 1.9 MB/s | 192 kB 00:00 2023-10-16T14:21:18.618 INFO:teuthology.orchestra.run.smithi187.stdout:(37/131): cephfs-top-18.0.0-6730.g4ba8a190.el8. 217 kB/s | 18 kB 00:00 2023-10-16T14:21:18.727 INFO:teuthology.orchestra.run.smithi187.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 543 kB/s | 59 kB 00:00 2023-10-16T14:21:18.752 INFO:teuthology.orchestra.run.smithi187.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.7 MB/s | 306 kB 00:00 2023-10-16T14:21:18.794 INFO:teuthology.orchestra.run.smithi187.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.1 MB/s | 75 kB 00:00 2023-10-16T14:21:18.844 INFO:teuthology.orchestra.run.smithi187.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 875 kB/s | 44 kB 00:00 2023-10-16T14:21:18.878 INFO:teuthology.orchestra.run.smithi187.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.7 MB/s | 476 kB 00:00 2023-10-16T14:21:18.920 INFO:teuthology.orchestra.run.smithi187.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 794 kB/s | 33 kB 00:00 2023-10-16T14:21:18.945 INFO:teuthology.orchestra.run.smithi187.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-10-16T14:21:18.962 INFO:teuthology.orchestra.run.smithi187.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 390 kB/s | 16 kB 00:00 2023-10-16T14:21:19.012 INFO:teuthology.orchestra.run.smithi187.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2023-10-16T14:21:19.046 INFO:teuthology.orchestra.run.smithi187.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-10-16T14:21:19.071 INFO:teuthology.orchestra.run.smithi187.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-16T14:21:19.088 INFO:teuthology.orchestra.run.smithi187.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2023-10-16T14:21:19.105 INFO:teuthology.orchestra.run.smithi187.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 524 kB/s | 18 kB 00:00 2023-10-16T14:21:19.140 INFO:teuthology.orchestra.run.smithi187.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-10-16T14:21:19.173 INFO:teuthology.orchestra.run.smithi187.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.7 MB/s | 59 kB 00:00 2023-10-16T14:21:19.181 INFO:teuthology.orchestra.run.smithi178.stdout:(8/131): ceph-common-18.0.0-6730.g4ba8a190.el8. 1.7 MB/s | 23 MB 00:13 2023-10-16T14:21:19.206 INFO:teuthology.orchestra.run.smithi187.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-10-16T14:21:19.248 INFO:teuthology.orchestra.run.smithi178.stdout:(9/131): ceph-selinux-18.0.0-6730.g4ba8a190.el8 367 kB/s | 24 kB 00:00 2023-10-16T14:21:19.290 INFO:teuthology.orchestra.run.smithi187.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-10-16T14:21:19.340 INFO:teuthology.orchestra.run.smithi187.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 4.6 MB/s | 1.2 MB 00:00 2023-10-16T14:21:19.365 INFO:teuthology.orchestra.run.smithi187.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 207 kB/s | 16 kB 00:00 2023-10-16T14:21:19.391 INFO:teuthology.orchestra.run.smithi187.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 671 kB/s | 34 kB 00:00 2023-10-16T14:21:19.433 INFO:teuthology.orchestra.run.smithi187.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-10-16T14:21:19.458 INFO:teuthology.orchestra.run.smithi187.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2023-10-16T14:21:19.483 INFO:teuthology.orchestra.run.smithi187.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-10-16T14:21:19.508 INFO:teuthology.orchestra.run.smithi187.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-10-16T14:21:19.534 INFO:teuthology.orchestra.run.smithi187.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-10-16T14:21:19.559 INFO:teuthology.orchestra.run.smithi187.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-10-16T14:21:19.572 INFO:teuthology.orchestra.run.smithi186.stdout:(10/131): ceph-osd-18.0.0-6730.g4ba8a190.el8.x8 1.9 MB/s | 18 MB 00:09 2023-10-16T14:21:19.601 INFO:teuthology.orchestra.run.smithi187.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.7 MB/s | 255 kB 00:00 2023-10-16T14:21:19.643 INFO:teuthology.orchestra.run.smithi187.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2023-10-16T14:21:19.668 INFO:teuthology.orchestra.run.smithi187.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 916 kB/s | 61 kB 00:00 2023-10-16T14:21:19.694 INFO:teuthology.orchestra.run.smithi187.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-10-16T14:21:19.719 INFO:teuthology.orchestra.run.smithi187.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-10-16T14:21:19.730 INFO:teuthology.orchestra.run.smithi186.stdout:(11/131): cephfs-mirror-18.0.0-6730.g4ba8a190.e 1.6 MB/s | 261 kB 00:00 2023-10-16T14:21:19.744 INFO:teuthology.orchestra.run.smithi187.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-16T14:21:19.806 INFO:teuthology.orchestra.run.smithi186.stdout:(12/131): libcephfs-devel-18.0.0-6730.g4ba8a190 415 kB/s | 31 kB 00:00 2023-10-16T14:21:19.953 INFO:teuthology.orchestra.run.smithi187.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 6.7 MB/s | 1.7 MB 00:00 2023-10-16T14:21:20.057 INFO:teuthology.orchestra.run.smithi178.stdout:(10/131): ceph-osd-18.0.0-6730.g4ba8a190.el8.x8 1.9 MB/s | 18 MB 00:09 2023-10-16T14:21:20.095 INFO:teuthology.orchestra.run.smithi187.stdout:(71/131): python3-scipy-1.0.0-21.module+el8.5.0 39 MB/s | 14 MB 00:00 2023-10-16T14:21:20.156 INFO:teuthology.orchestra.run.smithi186.stdout:(13/131): ceph-radosgw-18.0.0-6730.g4ba8a190.el 1.9 MB/s | 14 MB 00:07 2023-10-16T14:21:20.190 INFO:teuthology.orchestra.run.smithi186.stdout:(14/131): libcephfs2-18.0.0-6730.g4ba8a190.el8. 2.1 MB/s | 839 kB 00:00 2023-10-16T14:21:20.240 INFO:teuthology.orchestra.run.smithi186.stdout:(15/131): libcephsqlite-18.0.0-6730.g4ba8a190.e 2.2 MB/s | 184 kB 00:00 2023-10-16T14:21:20.241 INFO:teuthology.orchestra.run.smithi178.stdout:(11/131): cephfs-mirror-18.0.0-6730.g4ba8a190.e 1.4 MB/s | 261 kB 00:00 2023-10-16T14:21:20.257 INFO:teuthology.orchestra.run.smithi186.stdout:(16/131): librados-devel-18.0.0-6730.g4ba8a190. 2.0 MB/s | 139 kB 00:00 2023-10-16T14:21:20.307 INFO:teuthology.orchestra.run.smithi178.stdout:(12/131): libcephfs-devel-18.0.0-6730.g4ba8a190 467 kB/s | 31 kB 00:00 2023-10-16T14:21:20.424 INFO:teuthology.orchestra.run.smithi186.stdout:(17/131): libradosstriper1-18.0.0-6730.g4ba8a19 2.8 MB/s | 518 kB 00:00 2023-10-16T14:21:20.482 INFO:teuthology.orchestra.run.smithi186.stdout:(18/131): python3-ceph-argparse-18.0.0-6730.g4b 769 kB/s | 45 kB 00:00 2023-10-16T14:21:20.529 INFO:teuthology.orchestra.run.smithi187.stdout:(72/131): ceph-test-18.0.0-6730.g4ba8a190.el8.x 4.6 MB/s | 49 MB 00:10 2023-10-16T14:21:20.545 INFO:teuthology.orchestra.run.smithi187.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 122 kB/s | 54 kB 00:00 2023-10-16T14:21:20.549 INFO:teuthology.orchestra.run.smithi186.stdout:(19/131): python3-ceph-common-18.0.0-6730.g4ba8 1.8 MB/s | 126 kB 00:00 2023-10-16T14:21:20.570 INFO:teuthology.orchestra.run.smithi187.stdout:(74/131): python3-pyasn1-modules-0.3.7-6.el8.no 178 kB/s | 110 kB 00:00 2023-10-16T14:21:20.596 INFO:teuthology.orchestra.run.smithi187.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-10-16T14:21:20.607 INFO:teuthology.orchestra.run.smithi178.stdout:(13/131): libcephfs2-18.0.0-6730.g4ba8a190.el8. 2.7 MB/s | 839 kB 00:00 2023-10-16T14:21:20.613 INFO:teuthology.orchestra.run.smithi187.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 925 kB/s | 39 kB 00:00 2023-10-16T14:21:20.638 INFO:teuthology.orchestra.run.smithi187.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.6 MB/s | 323 kB 00:00 2023-10-16T14:21:20.649 INFO:teuthology.orchestra.run.smithi186.stdout:(20/131): python3-cephfs-18.0.0-6730.g4ba8a190. 1.9 MB/s | 197 kB 00:00 2023-10-16T14:21:20.680 INFO:teuthology.orchestra.run.smithi187.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-10-16T14:21:20.699 INFO:teuthology.orchestra.run.smithi178.stdout:(14/131): libcephsqlite-18.0.0-6730.g4ba8a190.e 2.0 MB/s | 184 kB 00:00 2023-10-16T14:21:20.739 INFO:teuthology.orchestra.run.smithi187.stdout:(79/131): bison-3.0.4-10.el8.x86_64.rpm 4.7 MB/s | 688 kB 00:00 2023-10-16T14:21:20.764 INFO:teuthology.orchestra.run.smithi187.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-10-16T14:21:20.783 INFO:teuthology.orchestra.run.smithi178.stdout:(15/131): librados-devel-18.0.0-6730.g4ba8a190. 1.6 MB/s | 139 kB 00:00 2023-10-16T14:21:20.783 INFO:teuthology.orchestra.run.smithi186.stdout:(21/131): python3-rados-18.0.0-6730.g4ba8a190.e 2.7 MB/s | 371 kB 00:00 2023-10-16T14:21:20.789 INFO:teuthology.orchestra.run.smithi187.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2023-10-16T14:21:20.823 INFO:teuthology.orchestra.run.smithi187.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 8.3 MB/s | 713 kB 00:00 2023-10-16T14:21:20.848 INFO:teuthology.orchestra.run.smithi187.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 455 kB/s | 27 kB 00:00 2023-10-16T14:21:20.890 INFO:teuthology.orchestra.run.smithi187.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2023-10-16T14:21:20.915 INFO:teuthology.orchestra.run.smithi187.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 658 kB/s | 44 kB 00:00 2023-10-16T14:21:20.933 INFO:teuthology.orchestra.run.smithi186.stdout:(22/131): python3-rbd-18.0.0-6730.g4ba8a190.el8 2.4 MB/s | 365 kB 00:00 2023-10-16T14:21:20.966 INFO:teuthology.orchestra.run.smithi187.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-10-16T14:21:20.983 INFO:teuthology.orchestra.run.smithi178.stdout:(16/131): libradosstriper1-18.0.0-6730.g4ba8a19 2.5 MB/s | 518 kB 00:00 2023-10-16T14:21:21.000 INFO:teuthology.orchestra.run.smithi186.stdout:(23/131): python3-rgw-18.0.0-6730.g4ba8a190.el8 1.6 MB/s | 107 kB 00:00 2023-10-16T14:21:21.008 INFO:teuthology.orchestra.run.smithi187.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-10-16T14:21:21.050 INFO:teuthology.orchestra.run.smithi187.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-10-16T14:21:21.076 INFO:teuthology.orchestra.run.smithi186.stdout:(24/131): rbd-fuse-18.0.0-6730.g4ba8a190.el8.x8 1.2 MB/s | 94 kB 00:00 2023-10-16T14:21:21.108 INFO:teuthology.orchestra.run.smithi187.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-10-16T14:21:21.175 INFO:teuthology.orchestra.run.smithi187.stdout:(90/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-10-16T14:21:21.200 INFO:teuthology.orchestra.run.smithi187.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 840 kB/s | 126 kB 00:00 2023-10-16T14:21:21.226 INFO:teuthology.orchestra.run.smithi187.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 170 kB/s | 20 kB 00:00 2023-10-16T14:21:21.251 INFO:teuthology.orchestra.run.smithi187.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-10-16T14:21:21.267 INFO:teuthology.orchestra.run.smithi178.stdout:(17/131): ceph-radosgw-18.0.0-6730.g4ba8a190.el 1.9 MB/s | 14 MB 00:07 2023-10-16T14:21:21.293 INFO:teuthology.orchestra.run.smithi187.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-10-16T14:21:21.318 INFO:teuthology.orchestra.run.smithi187.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-10-16T14:21:21.333 INFO:teuthology.orchestra.run.smithi178.stdout:(18/131): python3-ceph-argparse-18.0.0-6730.g4b 676 kB/s | 45 kB 00:00 2023-10-16T14:21:21.360 INFO:teuthology.orchestra.run.smithi187.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-10-16T14:21:21.401 INFO:teuthology.orchestra.run.smithi178.stdout:(19/131): python3-ceph-common-18.0.0-6730.g4ba8 1.8 MB/s | 126 kB 00:00 2023-10-16T14:21:21.402 INFO:teuthology.orchestra.run.smithi187.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-10-16T14:21:21.494 INFO:teuthology.orchestra.run.smithi187.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-10-16T14:21:21.518 INFO:teuthology.orchestra.run.smithi178.stdout:(20/131): python3-cephfs-18.0.0-6730.g4ba8a190. 1.7 MB/s | 197 kB 00:00 2023-10-16T14:21:21.561 INFO:teuthology.orchestra.run.smithi187.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-10-16T14:21:21.603 INFO:teuthology.orchestra.run.smithi187.stdout:(100/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-10-16T14:21:21.636 INFO:teuthology.orchestra.run.smithi187.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-10-16T14:21:21.678 INFO:teuthology.orchestra.run.smithi187.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-10-16T14:21:21.684 INFO:teuthology.orchestra.run.smithi178.stdout:(21/131): python3-rados-18.0.0-6730.g4ba8a190.e 2.2 MB/s | 371 kB 00:00 2023-10-16T14:21:21.703 INFO:teuthology.orchestra.run.smithi187.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-10-16T14:21:21.729 INFO:teuthology.orchestra.run.smithi187.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-10-16T14:21:21.754 INFO:teuthology.orchestra.run.smithi187.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-16T14:21:21.838 INFO:teuthology.orchestra.run.smithi187.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 442 kB/s | 48 kB 00:00 2023-10-16T14:21:21.860 INFO:teuthology.orchestra.run.smithi178.stdout:(22/131): python3-rbd-18.0.0-6730.g4ba8a190.el8 2.0 MB/s | 365 kB 00:00 2023-10-16T14:21:21.863 INFO:teuthology.orchestra.run.smithi187.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 799 kB/s | 87 kB 00:00 2023-10-16T14:21:21.888 INFO:teuthology.orchestra.run.smithi187.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-10-16T14:21:21.914 INFO:teuthology.orchestra.run.smithi187.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-10-16T14:21:21.927 INFO:teuthology.orchestra.run.smithi178.stdout:(23/131): python3-rgw-18.0.0-6730.g4ba8a190.el8 1.6 MB/s | 107 kB 00:00 2023-10-16T14:21:21.939 INFO:teuthology.orchestra.run.smithi187.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-10-16T14:21:21.985 INFO:teuthology.orchestra.run.smithi178.stdout:(24/131): rbd-fuse-18.0.0-6730.g4ba8a190.el8.x8 1.6 MB/s | 94 kB 00:00 2023-10-16T14:21:22.014 INFO:teuthology.orchestra.run.smithi187.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.3 MB/s | 5.1 MB 00:00 2023-10-16T14:21:22.031 INFO:teuthology.orchestra.run.smithi187.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 204 kB/s | 19 kB 00:00 2023-10-16T14:21:22.057 INFO:teuthology.orchestra.run.smithi187.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-10-16T14:21:22.082 INFO:teuthology.orchestra.run.smithi187.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.3 MB/s | 560 kB 00:00 2023-10-16T14:21:22.099 INFO:teuthology.orchestra.run.smithi187.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 428 kB/s | 18 kB 00:00 2023-10-16T14:21:22.132 INFO:teuthology.orchestra.run.smithi187.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.4 MB/s | 559 kB 00:00 2023-10-16T14:21:22.151 INFO:teuthology.orchestra.run.smithi186.stdout:(25/131): rbd-mirror-18.0.0-6730.g4ba8a190.el8. 3.3 MB/s | 3.6 MB 00:01 2023-10-16T14:21:22.174 INFO:teuthology.orchestra.run.smithi187.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-10-16T14:21:22.200 INFO:teuthology.orchestra.run.smithi187.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-10-16T14:21:22.225 INFO:teuthology.orchestra.run.smithi187.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-10-16T14:21:22.250 INFO:teuthology.orchestra.run.smithi187.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-10-16T14:21:22.260 INFO:teuthology.orchestra.run.smithi186.stdout:(26/131): rbd-nbd-18.0.0-6730.g4ba8a190.el8.x86 1.6 MB/s | 179 kB 00:00 2023-10-16T14:21:22.275 INFO:teuthology.orchestra.run.smithi187.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-10-16T14:21:22.292 INFO:teuthology.orchestra.run.smithi187.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 2.3 MB/s | 98 kB 00:00 2023-10-16T14:21:22.318 INFO:teuthology.orchestra.run.smithi187.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.3 MB/s | 58 kB 00:00 2023-10-16T14:21:22.326 INFO:teuthology.orchestra.run.smithi186.stdout:(27/131): ceph-grafana-dashboards-18.0.0-6730.g 346 kB/s | 23 kB 00:00 2023-10-16T14:21:22.343 INFO:teuthology.orchestra.run.smithi187.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-10-16T14:21:22.377 INFO:teuthology.orchestra.run.smithi187.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-10-16T14:21:22.402 INFO:teuthology.orchestra.run.smithi187.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-10-16T14:21:22.410 INFO:teuthology.orchestra.run.smithi186.stdout:(28/131): librgw2-18.0.0-6730.g4ba8a190.el8.x86 3.2 MB/s | 6.9 MB 00:02 2023-10-16T14:21:22.436 INFO:teuthology.orchestra.run.smithi187.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-10-16T14:21:22.436 INFO:teuthology.orchestra.run.smithi186.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6730.g4ba8a19 1.2 MB/s | 132 kB 00:00 2023-10-16T14:21:22.494 INFO:teuthology.orchestra.run.smithi187.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.4 MB/s | 2.3 MB 00:00 2023-10-16T14:21:22.544 INFO:teuthology.orchestra.run.smithi187.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-10-16T14:21:23.127 INFO:teuthology.orchestra.run.smithi186.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6730.g4ba8a 2.5 MB/s | 1.8 MB 00:00 2023-10-16T14:21:23.311 INFO:teuthology.orchestra.run.smithi178.stdout:(25/131): rbd-mirror-18.0.0-6730.g4ba8a190.el8. 2.7 MB/s | 3.6 MB 00:01 2023-10-16T14:21:23.361 INFO:teuthology.orchestra.run.smithi186.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6730.g4b 1.0 MB/s | 242 kB 00:00 2023-10-16T14:21:23.461 INFO:teuthology.orchestra.run.smithi186.stdout:(32/131): ceph-mgr-rook-18.0.0-6730.g4ba8a190.e 475 kB/s | 47 kB 00:00 2023-10-16T14:21:23.494 INFO:teuthology.orchestra.run.smithi178.stdout:(26/131): librgw2-18.0.0-6730.g4ba8a190.el8.x86 2.8 MB/s | 6.9 MB 00:02 2023-10-16T14:21:23.503 INFO:teuthology.orchestra.run.smithi187.stdout:(130/131): librbd1-18.0.0-6730.g4ba8a190.el8.x8 3.5 MB/s | 3.7 MB 00:01 2023-10-16T14:21:23.519 INFO:teuthology.orchestra.run.smithi178.stdout:(27/131): rbd-nbd-18.0.0-6730.g4ba8a190.el8.x86 860 kB/s | 179 kB 00:00 2023-10-16T14:21:23.536 INFO:teuthology.orchestra.run.smithi186.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6730.g4 200 kB/s | 15 kB 00:00 2023-10-16T14:21:23.570 INFO:teuthology.orchestra.run.smithi178.stdout:(28/131): ceph-grafana-dashboards-18.0.0-6730.g 307 kB/s | 23 kB 00:00 2023-10-16T14:21:23.604 INFO:teuthology.orchestra.run.smithi178.stdout:(29/131): ceph-mgr-cephadm-18.0.0-6730.g4ba8a19 1.5 MB/s | 132 kB 00:00 2023-10-16T14:21:23.687 INFO:teuthology.orchestra.run.smithi186.stdout:(34/131): ceph-volume-18.0.0-6730.g4ba8a190.el8 1.7 MB/s | 261 kB 00:00 2023-10-16T14:21:23.779 INFO:teuthology.orchestra.run.smithi186.stdout:(35/131): cephadm-18.0.0-6730.g4ba8a190.el8.noa 2.0 MB/s | 192 kB 00:00 2023-10-16T14:21:23.862 INFO:teuthology.orchestra.run.smithi186.stdout:(36/131): cephfs-top-18.0.0-6730.g4ba8a190.el8. 218 kB/s | 18 kB 00:00 2023-10-16T14:21:23.967 INFO:teuthology.orchestra.run.smithi187.stdout:(131/131): librados2-18.0.0-6730.g4ba8a190.el8. 2.4 MB/s | 3.7 MB 00:01 2023-10-16T14:21:23.970 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:21:23.970 INFO:teuthology.orchestra.run.smithi187.stdout:Total 10 MB/s | 209 MB 00:19 2023-10-16T14:21:23.996 INFO:teuthology.orchestra.run.smithi186.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2023-10-16T14:21:24.038 INFO:teuthology.orchestra.run.smithi186.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-10-16T14:21:24.080 INFO:teuthology.orchestra.run.smithi186.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-10-16T14:21:24.155 INFO:teuthology.orchestra.run.smithi186.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 6.2 MB/s | 476 kB 00:00 2023-10-16T14:21:24.197 INFO:teuthology.orchestra.run.smithi186.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-10-16T14:21:24.237 INFO:teuthology.orchestra.run.smithi178.stdout:(30/131): ceph-mgr-dashboard-18.0.0-6730.g4ba8a 2.6 MB/s | 1.8 MB 00:00 2023-10-16T14:21:24.239 INFO:teuthology.orchestra.run.smithi186.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 4.4 MB/s | 185 kB 00:00 2023-10-16T14:21:24.281 INFO:teuthology.orchestra.run.smithi186.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 797 kB/s | 33 kB 00:00 2023-10-16T14:21:24.323 INFO:teuthology.orchestra.run.smithi186.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 391 kB/s | 16 kB 00:00 2023-10-16T14:21:24.365 INFO:teuthology.orchestra.run.smithi186.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 4.1 MB/s | 173 kB 00:00 2023-10-16T14:21:24.396 INFO:teuthology.orchestra.run.smithi178.stdout:(31/131): ceph-mgr-modules-core-18.0.0-6730.g4b 1.5 MB/s | 242 kB 00:00 2023-10-16T14:21:24.415 INFO:teuthology.orchestra.run.smithi186.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 7.5 MB/s | 384 kB 00:00 2023-10-16T14:21:24.457 INFO:teuthology.orchestra.run.smithi186.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-10-16T14:21:24.491 INFO:teuthology.orchestra.run.smithi186.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 334 kB/s | 11 kB 00:00 2023-10-16T14:21:24.496 INFO:teuthology.orchestra.run.smithi178.stdout:(32/131): ceph-mgr-rook-18.0.0-6730.g4ba8a190.e 475 kB/s | 47 kB 00:00 2023-10-16T14:21:24.524 INFO:teuthology.orchestra.run.smithi186.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 530 kB/s | 18 kB 00:00 2023-10-16T14:21:24.580 INFO:teuthology.orchestra.run.smithi178.stdout:(33/131): ceph-prometheus-alerts-18.0.0-6730.g4 180 kB/s | 15 kB 00:00 2023-10-16T14:21:24.591 INFO:teuthology.orchestra.run.smithi186.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 18 MB/s | 1.2 MB 00:00 2023-10-16T14:21:24.625 INFO:teuthology.orchestra.run.smithi186.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-10-16T14:21:24.667 INFO:teuthology.orchestra.run.smithi186.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-10-16T14:21:24.709 INFO:teuthology.orchestra.run.smithi186.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-10-16T14:21:24.722 INFO:teuthology.orchestra.run.smithi178.stdout:(34/131): ceph-volume-18.0.0-6730.g4ba8a190.el8 1.8 MB/s | 261 kB 00:00 2023-10-16T14:21:24.788 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-10-16T14:21:24.793 INFO:teuthology.orchestra.run.smithi186.stdout:(54/131): ceph-mgr-diskprediction-local-18.0.0- 3.1 MB/s | 7.4 MB 00:02 2023-10-16T14:21:24.805 INFO:teuthology.orchestra.run.smithi178.stdout:(35/131): cephadm-18.0.0-6730.g4ba8a190.el8.noa 2.3 MB/s | 192 kB 00:00 2023-10-16T14:21:24.818 INFO:teuthology.orchestra.run.smithi186.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-10-16T14:21:24.860 INFO:teuthology.orchestra.run.smithi186.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 811 kB/s | 34 kB 00:00 2023-10-16T14:21:24.881 INFO:teuthology.orchestra.run.smithi178.stdout:(36/131): cephfs-top-18.0.0-6730.g4ba8a190.el8. 241 kB/s | 18 kB 00:00 2023-10-16T14:21:24.885 INFO:teuthology.orchestra.run.smithi186.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-10-16T14:21:24.907 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-10-16T14:21:24.908 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-10-16T14:21:24.910 INFO:teuthology.orchestra.run.smithi186.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-10-16T14:21:24.953 INFO:teuthology.orchestra.run.smithi186.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 960 kB/s | 64 kB 00:00 2023-10-16T14:21:24.978 INFO:teuthology.orchestra.run.smithi186.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 280 kB/s | 19 kB 00:00 2023-10-16T14:21:25.003 INFO:teuthology.orchestra.run.smithi186.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-10-16T14:21:25.028 INFO:teuthology.orchestra.run.smithi186.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-10-16T14:21:25.054 INFO:teuthology.orchestra.run.smithi186.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-10-16T14:21:25.079 INFO:teuthology.orchestra.run.smithi186.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-10-16T14:21:25.121 INFO:teuthology.orchestra.run.smithi186.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 3.7 MB/s | 251 kB 00:00 2023-10-16T14:21:25.122 INFO:teuthology.orchestra.run.smithi178.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-10-16T14:21:25.171 INFO:teuthology.orchestra.run.smithi186.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 666 kB/s | 61 kB 00:00 2023-10-16T14:21:25.181 INFO:teuthology.orchestra.run.smithi178.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-10-16T14:21:25.197 INFO:teuthology.orchestra.run.smithi186.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-16T14:21:25.222 INFO:teuthology.orchestra.run.smithi186.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 458 kB/s | 23 kB 00:00 2023-10-16T14:21:25.240 INFO:teuthology.orchestra.run.smithi178.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-10-16T14:21:25.247 INFO:teuthology.orchestra.run.smithi186.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-10-16T14:21:25.298 INFO:teuthology.orchestra.run.smithi186.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-10-16T14:21:25.339 INFO:teuthology.orchestra.run.smithi186.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-10-16T14:21:25.390 INFO:teuthology.orchestra.run.smithi186.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 963 kB/s | 43 kB 00:00 2023-10-16T14:21:25.415 INFO:teuthology.orchestra.run.smithi186.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 1.7 MB/s | 200 kB 00:00 2023-10-16T14:21:25.440 INFO:teuthology.orchestra.run.smithi186.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-10-16T14:21:25.465 INFO:teuthology.orchestra.run.smithi186.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2023-10-16T14:21:25.491 INFO:teuthology.orchestra.run.smithi186.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-10-16T14:21:25.516 INFO:teuthology.orchestra.run.smithi186.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 852 kB/s | 43 kB 00:00 2023-10-16T14:21:25.533 INFO:teuthology.orchestra.run.smithi186.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.5 MB/s | 109 kB 00:00 2023-10-16T14:21:25.565 INFO:teuthology.orchestra.run.smithi178.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 3.8 MB/s | 7.4 MB 00:01 2023-10-16T14:21:25.566 INFO:teuthology.orchestra.run.smithi186.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-10-16T14:21:25.591 INFO:teuthology.orchestra.run.smithi178.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-10-16T14:21:25.608 INFO:teuthology.orchestra.run.smithi186.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2023-10-16T14:21:25.626 INFO:teuthology.orchestra.run.smithi186.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 318 kB/s | 19 kB 00:00 2023-10-16T14:21:25.651 INFO:teuthology.orchestra.run.smithi186.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-10-16T14:21:25.685 INFO:teuthology.orchestra.run.smithi186.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-10-16T14:21:25.691 INFO:teuthology.orchestra.run.smithi178.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.9 MB/s | 185 kB 00:00 2023-10-16T14:21:25.701 INFO:teuthology.orchestra.run.smithi186.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 356 kB/s | 18 kB 00:00 2023-10-16T14:21:25.716 INFO:teuthology.orchestra.run.smithi178.stdout:(43/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 290 kB/s | 44 kB 00:00 2023-10-16T14:21:25.735 INFO:teuthology.orchestra.run.smithi186.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-10-16T14:21:25.760 INFO:teuthology.orchestra.run.smithi186.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-10-16T14:21:25.773 INFO:teuthology.orchestra.run.smithi178.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 405 kB/s | 33 kB 00:00 2023-10-16T14:21:25.800 INFO:teuthology.orchestra.run.smithi178.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-10-16T14:21:25.802 INFO:teuthology.orchestra.run.smithi186.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-10-16T14:21:25.827 INFO:teuthology.orchestra.run.smithi186.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 3.3 MB/s | 83 kB 00:00 2023-10-16T14:21:25.867 INFO:teuthology.orchestra.run.smithi178.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-10-16T14:21:25.870 INFO:teuthology.orchestra.run.smithi186.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 17 MB/s | 2.3 MB 00:00 2023-10-16T14:21:25.895 INFO:teuthology.orchestra.run.smithi186.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 897 kB/s | 60 kB 00:00 2023-10-16T14:21:25.920 INFO:teuthology.orchestra.run.smithi186.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-10-16T14:21:25.926 INFO:teuthology.orchestra.run.smithi178.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-10-16T14:21:25.945 INFO:teuthology.orchestra.run.smithi186.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-10-16T14:21:25.968 INFO:teuthology.orchestra.run.smithi178.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 268 kB/s | 11 kB 00:00 2023-10-16T14:21:25.970 INFO:teuthology.orchestra.run.smithi186.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 766 kB/s | 39 kB 00:00 2023-10-16T14:21:25.993 INFO:teuthology.orchestra.run.smithi178.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.9 MB/s | 384 kB 00:00 2023-10-16T14:21:26.004 INFO:teuthology.orchestra.run.smithi186.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-10-16T14:21:26.019 INFO:teuthology.orchestra.run.smithi178.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-10-16T14:21:26.026 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-10-16T14:21:26.029 INFO:teuthology.orchestra.run.smithi186.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-10-16T14:21:26.030 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-10-16T14:21:26.055 INFO:teuthology.orchestra.run.smithi186.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-16T14:21:26.061 INFO:teuthology.orchestra.run.smithi178.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.2 MB/s | 49 kB 00:00 2023-10-16T14:21:26.088 INFO:teuthology.orchestra.run.smithi186.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-10-16T14:21:26.119 INFO:teuthology.orchestra.run.smithi178.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-10-16T14:21:26.122 INFO:teuthology.orchestra.run.smithi186.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-10-16T14:21:26.178 INFO:teuthology.orchestra.run.smithi178.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-10-16T14:21:26.190 INFO:teuthology.orchestra.run.smithi186.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2023-10-16T14:21:26.287 INFO:teuthology.orchestra.run.smithi178.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.6 MB/s | 283 kB 00:00 2023-10-16T14:21:26.329 INFO:teuthology.orchestra.run.smithi178.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 378 kB/s | 16 kB 00:00 2023-10-16T14:21:26.371 INFO:teuthology.orchestra.run.smithi178.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 809 kB/s | 34 kB 00:00 2023-10-16T14:21:26.480 INFO:teuthology.orchestra.run.smithi178.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 2.4 MB/s | 1.2 MB 00:00 2023-10-16T14:21:26.498 INFO:teuthology.orchestra.run.smithi186.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2023-10-16T14:21:26.505 INFO:teuthology.orchestra.run.smithi178.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-10-16T14:21:26.540 INFO:teuthology.orchestra.run.smithi186.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 936 kB/s | 39 kB 00:00 2023-10-16T14:21:26.546 INFO:teuthology.orchestra.run.smithi178.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 963 kB/s | 64 kB 00:00 2023-10-16T14:21:26.572 INFO:teuthology.orchestra.run.smithi178.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 281 kB/s | 19 kB 00:00 2023-10-16T14:21:26.590 INFO:teuthology.orchestra.run.smithi186.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-10-16T14:21:26.614 INFO:teuthology.orchestra.run.smithi178.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 712 kB/s | 29 kB 00:00 2023-10-16T14:21:26.639 INFO:teuthology.orchestra.run.smithi178.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 359 kB/s | 24 kB 00:00 2023-10-16T14:21:26.640 INFO:teuthology.orchestra.run.smithi186.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-10-16T14:21:26.681 INFO:teuthology.orchestra.run.smithi178.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-10-16T14:21:26.682 INFO:teuthology.orchestra.run.smithi186.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-10-16T14:21:26.716 INFO:teuthology.orchestra.run.smithi186.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-10-16T14:21:26.781 INFO:teuthology.orchestra.run.smithi178.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.8 MB/s | 255 kB 00:00 2023-10-16T14:21:26.840 INFO:teuthology.orchestra.run.smithi178.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2023-10-16T14:21:26.858 INFO:teuthology.orchestra.run.smithi186.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 17 MB/s | 14 MB 00:00 2023-10-16T14:21:26.900 INFO:teuthology.orchestra.run.smithi186.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-10-16T14:21:26.915 INFO:teuthology.orchestra.run.smithi178.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-10-16T14:21:26.942 INFO:teuthology.orchestra.run.smithi186.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 639 kB/s | 27 kB 00:00 2023-10-16T14:21:26.957 INFO:teuthology.orchestra.run.smithi178.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 552 kB/s | 23 kB 00:00 2023-10-16T14:21:26.982 INFO:teuthology.orchestra.run.smithi178.stdout:(68/131): python3-webob-1.8.5-1.el8.1.noarch.rp 835 kB/s | 251 kB 00:00 2023-10-16T14:21:26.984 INFO:teuthology.orchestra.run.smithi186.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-10-16T14:21:27.008 INFO:teuthology.orchestra.run.smithi178.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-10-16T14:21:27.026 INFO:teuthology.orchestra.run.smithi186.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-10-16T14:21:27.088 INFO:teuthology.orchestra.run.smithi178.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 591 kB/s | 47 kB 00:00 2023-10-16T14:21:27.093 INFO:teuthology.orchestra.run.smithi186.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-10-16T14:21:27.134 INFO:teuthology.orchestra.run.smithi186.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-10-16T14:21:27.141 INFO:teuthology.orchestra.run.smithi178.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.7 MB/s | 200 kB 00:00 2023-10-16T14:21:27.168 INFO:teuthology.orchestra.run.smithi186.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 6.6 MB/s | 225 kB 00:00 2023-10-16T14:21:27.184 INFO:teuthology.orchestra.run.smithi178.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-10-16T14:21:27.226 INFO:teuthology.orchestra.run.smithi178.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-10-16T14:21:27.235 INFO:teuthology.orchestra.run.smithi186.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2023-10-16T14:21:27.252 INFO:teuthology.orchestra.run.smithi186.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-10-16T14:21:27.268 INFO:teuthology.orchestra.run.smithi178.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-10-16T14:21:27.277 INFO:teuthology.orchestra.run.smithi186.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-10-16T14:21:27.294 INFO:teuthology.orchestra.run.smithi186.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 479 kB/s | 20 kB 00:00 2023-10-16T14:21:27.310 INFO:teuthology.orchestra.run.smithi178.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.1 MB/s | 87 kB 00:00 2023-10-16T14:21:27.319 INFO:teuthology.orchestra.run.smithi186.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-10-16T14:21:27.352 INFO:teuthology.orchestra.run.smithi178.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-10-16T14:21:27.353 INFO:teuthology.orchestra.run.smithi186.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-10-16T14:21:27.395 INFO:teuthology.orchestra.run.smithi186.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-10-16T14:21:27.418 INFO:teuthology.orchestra.run.smithi178.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-10-16T14:21:27.469 INFO:teuthology.orchestra.run.smithi178.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-10-16T14:21:27.504 INFO:teuthology.orchestra.run.smithi186.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 26 MB/s | 4.9 MB 00:00 2023-10-16T14:21:27.529 INFO:teuthology.orchestra.run.smithi178.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-10-16T14:21:27.545 INFO:teuthology.orchestra.run.smithi186.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-16T14:21:27.561 INFO:teuthology.orchestra.run.smithi178.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 565 kB/s | 19 kB 00:00 2023-10-16T14:21:27.603 INFO:teuthology.orchestra.run.smithi178.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.7 MB/s | 155 kB 00:00 2023-10-16T14:21:27.621 INFO:teuthology.orchestra.run.smithi186.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-10-16T14:21:27.646 INFO:teuthology.orchestra.run.smithi186.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.4 MB/s | 354 kB 00:00 2023-10-16T14:21:27.653 INFO:teuthology.orchestra.run.smithi178.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 11 MB/s | 559 kB 00:00 2023-10-16T14:21:27.696 INFO:teuthology.orchestra.run.smithi178.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 431 kB/s | 18 kB 00:00 2023-10-16T14:21:27.705 INFO:teuthology.orchestra.run.smithi186.stdout:(125/131): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-10-16T14:21:27.780 INFO:teuthology.orchestra.run.smithi178.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-10-16T14:21:27.797 INFO:teuthology.orchestra.run.smithi186.stdout:(126/131): postgresql-server-10.21-2.module+el8 29 MB/s | 5.1 MB 00:00 2023-10-16T14:21:27.821 INFO:teuthology.orchestra.run.smithi178.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-10-16T14:21:27.839 INFO:teuthology.orchestra.run.smithi186.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-10-16T14:21:27.913 INFO:teuthology.orchestra.run.smithi178.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 2.3 MB 00:00 2023-10-16T14:21:27.955 INFO:teuthology.orchestra.run.smithi178.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-10-16T14:21:27.989 INFO:teuthology.orchestra.run.smithi178.stdout:(88/131): thrift-0.13.0-2.el8.x86_64.rpm 1.7 MB/s | 1.7 MB 00:01 2023-10-16T14:21:28.014 INFO:teuthology.orchestra.run.smithi178.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-10-16T14:21:28.092 INFO:teuthology.orchestra.run.smithi178.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 586 kB/s | 60 kB 00:00 2023-10-16T14:21:28.123 INFO:teuthology.orchestra.run.smithi178.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 954 kB/s | 98 kB 00:00 2023-10-16T14:21:28.148 INFO:teuthology.orchestra.run.smithi178.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2023-10-16T14:21:28.173 INFO:teuthology.orchestra.run.smithi178.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-10-16T14:21:28.207 INFO:teuthology.orchestra.run.smithi178.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-10-16T14:21:28.233 INFO:teuthology.orchestra.run.smithi178.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-10-16T14:21:28.258 INFO:teuthology.orchestra.run.smithi178.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-10-16T14:21:28.300 INFO:teuthology.orchestra.run.smithi178.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-10-16T14:21:28.775 INFO:teuthology.orchestra.run.smithi178.stdout:(98/131): ceph-test-18.0.0-6730.g4ba8a190.el8.x 5.1 MB/s | 49 MB 00:09 2023-10-16T14:21:28.801 INFO:teuthology.orchestra.run.smithi178.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 108 kB/s | 54 kB 00:00 2023-10-16T14:21:28.851 INFO:teuthology.orchestra.run.smithi178.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2023-10-16T14:21:28.876 INFO:teuthology.orchestra.run.smithi178.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.5 MB/s | 323 kB 00:00 2023-10-16T14:21:28.902 INFO:teuthology.orchestra.run.smithi178.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 777 kB/s | 39 kB 00:00 2023-10-16T14:21:28.944 INFO:teuthology.orchestra.run.smithi178.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-10-16T14:21:28.977 INFO:teuthology.orchestra.run.smithi178.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-10-16T14:21:29.002 INFO:teuthology.orchestra.run.smithi178.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-16T14:21:29.014 INFO:teuthology.orchestra.run.smithi186.stdout:(128/131): ceph-test-18.0.0-6730.g4ba8a190.el8. 4.2 MB/s | 49 MB 00:11 2023-10-16T14:21:29.028 INFO:teuthology.orchestra.run.smithi178.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-10-16T14:21:29.081 INFO:teuthology.orchestra.run.smithi186.stdout:(129/131): librados2-18.0.0-6730.g4ba8a190.el8. 2.9 MB/s | 3.7 MB 00:01 2023-10-16T14:21:29.102 INFO:teuthology.orchestra.run.smithi178.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 7.0 MB/s | 713 kB 00:00 2023-10-16T14:21:29.145 INFO:teuthology.orchestra.run.smithi178.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 631 kB/s | 27 kB 00:00 2023-10-16T14:21:29.149 INFO:teuthology.orchestra.run.smithi186.stdout:(130/131): librbd1-18.0.0-6730.g4ba8a190.el8.x8 2.8 MB/s | 3.7 MB 00:01 2023-10-16T14:21:29.181 INFO:teuthology.orchestra.run.smithi186.stdout:(131/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.7 MB/s | 798 kB 00:00 2023-10-16T14:21:29.184 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:21:29.184 INFO:teuthology.orchestra.run.smithi186.stdout:Total 8.4 MB/s | 209 MB 00:24 2023-10-16T14:21:29.195 INFO:teuthology.orchestra.run.smithi178.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-10-16T14:21:29.237 INFO:teuthology.orchestra.run.smithi178.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-10-16T14:21:29.357 INFO:teuthology.orchestra.run.smithi178.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2023-10-16T14:21:29.405 INFO:teuthology.orchestra.run.smithi178.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-10-16T14:21:29.488 INFO:teuthology.orchestra.run.smithi178.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-10-16T14:21:29.513 INFO:teuthology.orchestra.run.smithi178.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.0 MB/s | 225 kB 00:00 2023-10-16T14:21:29.539 INFO:teuthology.orchestra.run.smithi178.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-10-16T14:21:29.564 INFO:teuthology.orchestra.run.smithi178.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-10-16T14:21:29.589 INFO:teuthology.orchestra.run.smithi178.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 400 kB/s | 20 kB 00:00 2023-10-16T14:21:29.614 INFO:teuthology.orchestra.run.smithi178.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-10-16T14:21:29.665 INFO:teuthology.orchestra.run.smithi178.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 9.2 MB/s | 709 kB 00:00 2023-10-16T14:21:29.707 INFO:teuthology.orchestra.run.smithi178.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-10-16T14:21:29.874 INFO:teuthology.orchestra.run.smithi178.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 8.3 MB/s | 14 MB 00:01 2023-10-16T14:21:29.916 INFO:teuthology.orchestra.run.smithi178.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-10-16T14:21:29.966 INFO:teuthology.orchestra.run.smithi178.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-10-16T14:21:29.998 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2023-10-16T14:21:30.102 INFO:teuthology.orchestra.run.smithi178.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 10 MB/s | 4.9 MB 00:00 2023-10-16T14:21:30.118 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction check succeeded. 2023-10-16T14:21:30.119 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2023-10-16T14:21:30.183 INFO:teuthology.orchestra.run.smithi178.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 9.9 MB/s | 4.7 MB 00:00 2023-10-16T14:21:30.233 INFO:teuthology.orchestra.run.smithi178.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.0 MB/s | 808 kB 00:00 2023-10-16T14:21:30.292 INFO:teuthology.orchestra.run.smithi178.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 14 MB/s | 1.5 MB 00:00 2023-10-16T14:21:30.368 INFO:teuthology.orchestra.run.smithi178.stdout:(128/131): postgresql-server-10.21-2.module+el8 13 MB/s | 5.1 MB 00:00 2023-10-16T14:21:30.643 INFO:teuthology.orchestra.run.smithi178.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 2.8 MB/s | 798 kB 00:00 2023-10-16T14:21:31.002 INFO:teuthology.orchestra.run.smithi178.stdout:(130/131): librbd1-18.0.0-6730.g4ba8a190.el8.x8 5.2 MB/s | 3.7 MB 00:00 2023-10-16T14:21:31.015 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-10-16T14:21:31.199 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-16T14:21:31.254 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded. 2023-10-16T14:21:31.258 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2023-10-16T14:21:31.277 INFO:teuthology.orchestra.run.smithi178.stdout:(131/131): librados2-18.0.0-6730.g4ba8a190.el8. 3.6 MB/s | 3.7 MB 00:01 2023-10-16T14:21:31.281 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:21:31.281 INFO:teuthology.orchestra.run.smithi178.stdout:Total 8.1 MB/s | 209 MB 00:25 2023-10-16T14:21:31.374 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-16T14:21:31.582 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-10-16T14:21:31.742 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-16T14:21:31.844 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-10-16T14:21:32.076 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-16T14:21:32.481 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-10-16T14:21:32.709 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:32.955 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-10-16T14:21:33.089 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-10-16T14:21:33.089 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-10-16T14:21:33.116 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:33.300 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:33.433 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:33.609 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:21:33.708 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:21:33.901 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-16T14:21:34.128 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-16T14:21:34.328 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-16T14:21:34.694 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-10-16T14:21:34.699 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-10-16T14:21:34.715 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-16T14:21:34.965 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-16T14:21:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:35.408 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:35.498 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-16T14:21:35.775 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-10-16T14:21:35.933 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-16T14:21:36.298 INFO:teuthology.orchestra.run.smithi186.stdout: Preparing : 1/1 2023-10-16T14:21:36.519 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-16T14:21:36.534 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-10-16T14:21:36.725 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-16T14:21:36.727 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-16T14:21:36.905 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T14:21:37.013 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T14:21:37.221 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T14:21:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-10-16T14:21:37.499 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-10-16T14:21:37.619 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T14:21:37.830 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-16T14:21:38.185 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-10-16T14:21:38.221 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-16T14:21:38.413 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:38.431 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-10-16T14:21:38.577 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:38.666 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-10-16T14:21:38.779 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:38.861 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:38.878 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-10-16T14:21:39.049 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:21:39.072 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-10-16T14:21:39.240 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-16T14:21:39.280 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-10-16T14:21:39.491 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-16T14:21:39.518 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-16T14:21:39.691 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-16T14:21:39.722 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 26/134 2023-10-16T14:21:39.869 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 27/134 2023-10-16T14:21:39.892 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-10-16T14:21:40.072 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-10-16T14:21:40.080 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-16T14:21:40.307 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-10-16T14:21:40.319 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-16T14:21:40.473 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T14:21:40.479 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:40.586 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:40.638 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-10-16T14:21:40.759 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 28/134 2023-10-16T14:21:40.762 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:40.898 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:40.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T14:21:41.079 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-10-16T14:21:41.124 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-16T14:21:41.310 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-16T14:21:41.325 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-10-16T14:21:41.462 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-10-16T14:21:41.485 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-16T14:21:41.577 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-10-16T14:21:41.646 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T14:21:41.706 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-16T14:21:41.765 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-16T14:21:41.783 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T14:21:41.921 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T14:21:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 5/134 2023-10-16T14:21:42.356 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:42.496 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:21:42.532 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T14:21:42.697 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:42.720 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T14:21:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:21:43.011 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:21:43.244 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-10-16T14:21:43.308 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T14:21:43.511 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-10-16T14:21:43.513 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-10-16T14:21:43.521 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T14:21:43.736 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-10-16T14:21:43.808 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-10-16T14:21:44.018 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-16T14:21:44.188 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-10-16T14:21:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-16T14:21:44.250 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T14:21:44.414 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-10-16T14:21:44.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 36/134 2023-10-16T14:21:44.493 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-16T14:21:44.605 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:44.619 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 37/134 2023-10-16T14:21:44.723 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 14/134 2023-10-16T14:21:44.725 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-16T14:21:44.781 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-10-16T14:21:44.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:44.934 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-10-16T14:21:44.955 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-16T14:21:45.080 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 15/134 2023-10-16T14:21:45.134 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-10-16T14:21:45.135 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 26/134 2023-10-16T14:21:45.281 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 27/134 2023-10-16T14:21:45.294 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-10-16T14:21:45.328 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-10-16T14:21:45.507 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 28/134 2023-10-16T14:21:45.527 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-10-16T14:21:45.548 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-10-16T14:21:45.738 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-10-16T14:21:45.753 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T14:21:45.755 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-10-16T14:21:45.905 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T14:21:46.070 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-10-16T14:21:46.113 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-10-16T14:21:46.272 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T14:21:46.376 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-10-16T14:21:46.583 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-10-16T14:21:46.794 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-10-16T14:21:46.954 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-10-16T14:21:47.044 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-10-16T14:21:47.138 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T14:21:47.365 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-10-16T14:21:47.515 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T14:21:47.564 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-10-16T14:21:47.749 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-10-16T14:21:47.813 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-10-16T14:21:47.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-10-16T14:21:47.996 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T14:21:48.144 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-10-16T14:21:48.335 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-10-16T14:21:48.587 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-16T14:21:48.636 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-10-16T14:21:48.827 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-10-16T14:21:48.893 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-10-16T14:21:48.935 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-16T14:21:49.079 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-10-16T14:21:49.152 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-10-16T14:21:49.180 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-16T14:21:49.329 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-10-16T14:21:49.388 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-16T14:21:49.396 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-10-16T14:21:49.563 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-10-16T14:21:49.606 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-16T14:21:49.624 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-10-16T14:21:49.716 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-10-16T14:21:49.786 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 26/134 2023-10-16T14:21:49.797 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-16T14:21:49.881 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-10-16T14:21:49.951 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 27/134 2023-10-16T14:21:49.951 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-10-16T14:21:49.992 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 36/134 2023-10-16T14:21:50.176 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 37/134 2023-10-16T14:21:50.189 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 28/134 2023-10-16T14:21:50.206 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-10-16T14:21:50.352 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-16T14:21:50.396 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T14:21:50.401 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-10-16T14:21:50.512 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-16T14:21:50.547 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-16T14:21:50.648 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-10-16T14:21:50.711 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T14:21:50.782 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T14:21:50.882 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-10-16T14:21:51.032 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-10-16T14:21:51.039 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-16T14:21:51.181 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-16T14:21:51.256 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-16T14:21:51.369 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-16T14:21:51.403 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T14:21:51.468 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T14:21:51.662 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-16T14:21:51.673 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-16T14:21:51.856 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-16T14:21:51.913 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-16T14:21:52.034 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-16T14:21:52.068 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-16T14:21:52.203 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-16T14:21:52.225 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-16T14:21:52.396 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-16T14:21:52.416 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-16T14:21:52.438 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-10-16T14:21:52.547 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-16T14:21:52.618 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-16T14:21:52.782 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-16T14:21:52.802 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-16T14:21:52.967 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-16T14:21:53.103 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-16T14:21:53.297 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-16T14:21:53.319 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-16T14:21:53.550 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-16T14:21:53.570 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-10-16T14:21:53.686 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-16T14:21:53.753 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-16T14:21:53.957 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-10-16T14:21:53.996 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-16T14:21:54.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-16T14:21:54.222 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-16T14:21:54.238 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-16T14:21:54.251 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-10-16T14:21:54.421 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-16T14:21:54.466 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-16T14:21:54.525 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-10-16T14:21:54.656 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-16T14:21:54.716 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-16T14:21:54.756 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-10-16T14:21:54.967 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-10-16T14:21:55.078 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-16T14:21:55.170 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 36/134 2023-10-16T14:21:55.327 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-16T14:21:55.369 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 37/134 2023-10-16T14:21:55.528 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-16T14:21:55.554 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-10-16T14:21:55.732 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-10-16T14:21:55.764 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-16T14:21:55.766 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-16T14:21:55.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T14:21:55.973 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-16T14:21:56.032 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-10-16T14:21:56.108 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-16T14:21:56.155 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-16T14:21:56.330 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-10-16T14:21:56.379 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-16T14:21:56.453 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-16T14:21:56.592 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-10-16T14:21:56.695 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-16T14:21:56.706 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-16T14:21:56.811 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T14:21:56.910 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-10-16T14:21:56.965 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-16T14:21:56.965 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-16T14:21:57.158 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-16T14:21:57.163 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-10-16T14:21:57.203 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-16T14:21:57.358 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-10-16T14:21:57.409 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-16T14:21:57.528 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-10-16T14:21:57.596 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-16T14:21:57.698 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-10-16T14:21:57.852 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-16T14:21:57.906 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-10-16T14:21:57.969 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-16T14:21:58.058 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-10-16T14:21:58.110 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-16T14:21:58.112 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:21:58.181 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:21:58.256 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-16T14:21:58.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-10-16T14:21:58.346 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 93/134 2023-10-16T14:21:58.404 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-16T14:21:58.580 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-10-16T14:21:58.618 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-16T14:21:58.854 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-16T14:21:59.009 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-10-16T14:21:59.072 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-16T14:21:59.279 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-16T14:21:59.334 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-10-16T14:21:59.440 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-16T14:21:59.576 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-10-16T14:21:59.688 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-16T14:21:59.813 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-10-16T14:21:59.938 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-16T14:22:00.071 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-10-16T14:22:00.166 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-16T14:22:00.409 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-10-16T14:22:00.432 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-16T14:22:00.654 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-10-16T14:22:00.659 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-16T14:22:00.826 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-16T14:22:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-10-16T14:22:01.052 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-16T14:22:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-10-16T14:22:01.302 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-10-16T14:22:01.466 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-10-16T14:22:01.818 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-10-16T14:22:01.993 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:02.136 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-10-16T14:22:02.179 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-16T14:22:02.204 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:02.425 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-10-16T14:22:02.463 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-16T14:22:02.705 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-10-16T14:22:02.734 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-16T14:22:02.792 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:02.918 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-10-16T14:22:03.053 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-16T14:22:03.062 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:03.108 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-10-16T14:22:03.151 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:03.303 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:03.349 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-16T14:22:03.416 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-10-16T14:22:03.571 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-16T14:22:03.660 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-10-16T14:22:03.823 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-10-16T14:22:04.014 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-10-16T14:22:04.215 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-10-16T14:22:04.416 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-16T14:22:04.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-10-16T14:22:04.592 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:22:04.664 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:22:04.690 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-10-16T14:22:04.859 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 93/134 2023-10-16T14:22:04.872 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-10-16T14:22:05.066 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-10-16T14:22:05.296 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-10-16T14:22:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-10-16T14:22:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-10-16T14:22:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-10-16T14:22:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-10-16T14:22:06.510 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-10-16T14:22:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-10-16T14:22:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-10-16T14:22:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-10-16T14:22:08.597 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-10-16T14:22:09.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-10-16T14:22:09.309 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:09.450 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-10-16T14:22:09.709 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-10-16T14:22:09.904 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:10.193 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:10.274 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:10.442 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:10.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-10-16T14:22:10.784 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:22:10.868 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 92/134 2023-10-16T14:22:11.067 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 93/134 2023-10-16T14:22:16.095 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:16.359 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:17.050 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 94/134 2023-10-16T14:22:17.849 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:18.051 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 95/134 2023-10-16T14:22:18.182 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:22.092 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:22.092 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2023-10-16T14:22:22.092 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2023-10-16T14:22:22.092 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-10-16T14:22:22.093 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /run 2023-10-16T14:22:22.093 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2023-10-16T14:22:22.093 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2023-10-16T14:22:22.093 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2023-10-16T14:22:22.093 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2023-10-16T14:22:22.094 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-10-16T14:22:22.094 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:22:22.467 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:24.052 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:24.208 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:24.428 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:24.600 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:25.062 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:25.208 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:25.224 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:27.006 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:27.210 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:27.330 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 102/134 2023-10-16T14:22:27.532 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 103/134 2023-10-16T14:22:27.732 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:27.952 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:28.241 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:29.198 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:29.407 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:29.429 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:29.430 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /sys 2023-10-16T14:22:29.430 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /proc 2023-10-16T14:22:29.430 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-16T14:22:29.430 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /run 2023-10-16T14:22:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /mnt 2023-10-16T14:22:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /var/tmp 2023-10-16T14:22:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /home 2023-10-16T14:22:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /tmp 2023-10-16T14:22:29.431 INFO:teuthology.orchestra.run.smithi186.stdout:skipping the directory /dev 2023-10-16T14:22:29.432 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:22:29.497 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:29.781 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:29.856 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 107/134 2023-10-16T14:22:30.068 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:30.103 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:30.265 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:30.276 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:31.371 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:31.565 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:31.611 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 110/134 2023-10-16T14:22:31.785 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:31.824 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:31.857 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:31.997 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:31.998 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:32.040 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:32.198 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:32.455 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:32.564 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:32.622 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:32.638 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:32.707 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:34.441 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:34.658 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:34.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 102/134 2023-10-16T14:22:35.005 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 103/134 2023-10-16T14:22:35.204 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:35.426 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:35.756 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:36.719 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:36.921 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:37.013 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:37.391 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 107/134 2023-10-16T14:22:37.624 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 115/134 2023-10-16T14:22:37.642 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:37.678 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:37.772 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-10-16T14:22:37.837 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:37.849 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:37.961 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-10-16T14:22:38.709 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 96/134 2023-10-16T14:22:38.710 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2023-10-16T14:22:38.710 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2023-10-16T14:22:38.710 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-10-16T14:22:38.710 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /run 2023-10-16T14:22:38.711 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2023-10-16T14:22:38.711 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2023-10-16T14:22:38.711 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2023-10-16T14:22:38.711 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2023-10-16T14:22:38.711 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-10-16T14:22:38.712 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:22:38.810 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-10-16T14:22:39.137 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:39.159 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-10-16T14:22:39.197 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 110/134 2023-10-16T14:22:39.350 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 119/134 2023-10-16T14:22:39.376 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:39.410 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:39.551 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-16T14:22:39.571 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:39.613 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:39.717 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 121/134 2023-10-16T14:22:39.804 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:39.873 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T14:22:40.174 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:40.354 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:40.860 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 97/134 2023-10-16T14:22:41.053 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:41.333 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 98/134 2023-10-16T14:22:41.537 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:42.032 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:42.060 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 99/134 2023-10-16T14:22:42.280 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:42.297 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:42.303 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:42.346 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 100/134 2023-10-16T14:22:42.463 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 124/134 2023-10-16T14:22:42.754 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 125/134 2023-10-16T14:22:43.057 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 126/134 2023-10-16T14:22:43.331 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-16T14:22:43.463 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-16T14:22:43.471 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-16T14:22:43.683 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-10-16T14:22:43.862 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-10-16T14:22:43.942 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-10-16T14:22:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-10-16T14:22:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-10-16T14:22:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:44.374 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:44.374 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:22:44.601 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:22:44.870 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:45.003 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:45.220 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:45.392 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:45.392 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 1/134 2023-10-16T14:22:45.392 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2/134 2023-10-16T14:22:45.393 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 3/134 2023-10-16T14:22:45.393 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 4/134 2023-10-16T14:22:45.393 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 5/134 2023-10-16T14:22:45.393 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:22:45.393 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:22:45.394 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:22:45.394 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-16T14:22:45.394 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 10/134 2023-10-16T14:22:45.394 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 11/134 2023-10-16T14:22:45.394 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 12/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 13/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 14/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 15/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 16/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 17/134 2023-10-16T14:22:45.395 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 18/134 2023-10-16T14:22:45.396 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 19/134 2023-10-16T14:22:45.396 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 20/134 2023-10-16T14:22:45.396 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 21/134 2023-10-16T14:22:45.396 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 22/134 2023-10-16T14:22:45.396 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 23/134 2023-10-16T14:22:45.397 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 24/134 2023-10-16T14:22:45.397 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 25/134 2023-10-16T14:22:45.397 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 26/134 2023-10-16T14:22:45.397 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 27/134 2023-10-16T14:22:45.397 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 28/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 29/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 30/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 31/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 32/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 33/134 2023-10-16T14:22:45.398 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 34/134 2023-10-16T14:22:45.399 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 35/134 2023-10-16T14:22:45.399 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 36/134 2023-10-16T14:22:45.399 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 37/134 2023-10-16T14:22:45.399 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 38/134 2023-10-16T14:22:45.399 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-16T14:22:45.400 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-16T14:22:45.400 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-16T14:22:45.400 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-16T14:22:45.400 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-16T14:22:45.400 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-16T14:22:45.401 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-16T14:22:45.402 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-16T14:22:45.402 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-16T14:22:45.402 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-16T14:22:45.402 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-16T14:22:45.402 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-16T14:22:45.403 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-16T14:22:45.403 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-16T14:22:45.403 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-16T14:22:45.403 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-16T14:22:45.404 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-16T14:22:45.405 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-16T14:22:45.405 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-16T14:22:45.405 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-16T14:22:45.405 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-16T14:22:45.405 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-16T14:22:45.406 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-16T14:22:45.406 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-10-16T14:22:45.406 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-10-16T14:22:45.406 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-10-16T14:22:45.406 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-10-16T14:22:45.407 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-10-16T14:22:45.408 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-10-16T14:22:45.408 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-10-16T14:22:45.408 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-10-16T14:22:45.408 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-10-16T14:22:45.408 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-10-16T14:22:45.409 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-10-16T14:22:45.409 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-10-16T14:22:45.409 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-10-16T14:22:45.409 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-10-16T14:22:45.410 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-10-16T14:22:45.410 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-10-16T14:22:45.410 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-10-16T14:22:45.410 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-10-16T14:22:45.410 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-10-16T14:22:45.411 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-10-16T14:22:45.412 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-10-16T14:22:45.412 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-10-16T14:22:45.412 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-10-16T14:22:45.412 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-10-16T14:22:45.412 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-10-16T14:22:45.413 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-10-16T14:22:45.414 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-10-16T14:22:45.414 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-10-16T14:22:45.414 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-10-16T14:22:45.414 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-10-16T14:22:45.414 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-10-16T14:22:45.415 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-10-16T14:22:45.415 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-10-16T14:22:45.415 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-10-16T14:22:45.415 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-10-16T14:22:45.415 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-10-16T14:22:45.416 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-10-16T14:22:45.417 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 129/134 2023-10-16T14:22:45.417 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-16T14:22:45.417 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 131/134 2023-10-16T14:22:45.417 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:45.417 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-16T14:22:45.441 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 101/134 2023-10-16T14:22:45.591 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 102/134 2023-10-16T14:22:45.840 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 103/134 2023-10-16T14:22:45.877 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 115/134 2023-10-16T14:22:46.094 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:46.190 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-16T14:22:46.358 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 104/134 2023-10-16T14:22:46.381 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 117/134 2023-10-16T14:22:46.568 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:46.629 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-16T14:22:46.778 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T14:22:46.969 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T14:22:47.207 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-16T14:22:47.390 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 121/134 2023-10-16T14:22:47.554 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T14:22:47.655 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 105/134 2023-10-16T14:22:47.808 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:47.808 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:55 ago on Mon 16 Oct 2023 02:20:52 PM UTC. 2023-10-16T14:22:47.910 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:48.038 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 106/134 2023-10-16T14:22:48.415 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 107/134 2023-10-16T14:22:48.623 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:48.688 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 108/134 2023-10-16T14:22:48.886 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:48.914 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 109/134 2023-10-16T14:22:49.746 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:50.011 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:50.030 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:22:50.195 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 124/134 2023-10-16T14:22:50.247 INFO:teuthology.orchestra.run.smithi187.stdout:Installed products updated. 2023-10-16T14:22:50.377 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 110/134 2023-10-16T14:22:50.459 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2023-10-16T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-16T14:22:50.460 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-10-16T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-16T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.1-7.el8.noarch 2023-10-16T14:22:50.461 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.0.4-10.el8.x86_64 2023-10-16T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.462 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.463 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.464 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.466 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-16T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.1-9.el8.x86_64 2023-10-16T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-16T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-16T14:22:50.467 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-3.el8.x86_64 2023-10-16T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-16T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-16T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-16T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.468 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-16T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:22:50.469 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-16T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: libpq-13.5-1.el8.x86_64 2023-10-16T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-16T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.470 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.471 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-16T14:22:50.471 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.471 INFO:teuthology.orchestra.run.smithi187.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-16T14:22:50.471 INFO:teuthology.orchestra.run.smithi187.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-16T14:22:50.471 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-16T14:22:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-16T14:22:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-16T14:22:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-16T14:22:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-16T14:22:50.472 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-16T14:22:50.473 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-16T14:22:50.473 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-16T14:22:50.473 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-16T14:22:50.473 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-16T14:22:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-16T14:22:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-16T14:22:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-16T14:22:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-16T14:22:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-16T14:22:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-16T14:22:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-16T14:22:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-16T14:22:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-16T14:22:50.477 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-16T14:22:50.477 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-16T14:22:50.477 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-16T14:22:50.477 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-16T14:22:50.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-16T14:22:50.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-16T14:22:50.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-16T14:22:50.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-16T14:22:50.478 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-16T14:22:50.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-16T14:22:50.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-16T14:22:50.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-16T14:22:50.479 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-16T14:22:50.480 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-16T14:22:50.480 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-16T14:22:50.480 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-16T14:22:50.480 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-16T14:22:50.480 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-16T14:22:50.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-16T14:22:50.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-16T14:22:50.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-2.6-1.el8.noarch 2023-10-16T14:22:50.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-16T14:22:50.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-16T14:22:50.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-16T14:22:50.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-16T14:22:50.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-16T14:22:50.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-16T14:22:50.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-16T14:22:50.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-16T14:22:50.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-16T14:22:50.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-16T14:22:50.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-16T14:22:50.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-16T14:22:50.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-16T14:22:50.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-16T14:22:50.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-16T14:22:50.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-16T14:22:50.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-16T14:22:50.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-16T14:22:50.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-16T14:22:50.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-16T14:22:50.486 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-16T14:22:50.486 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.487 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.487 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:50.487 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-16T14:22:50.487 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-16T14:22:50.487 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-16T14:22:50.488 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-16T14:22:50.488 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-16T14:22:50.488 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-16T14:22:50.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:22:50.489 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-10-16T14:22:50.502 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 125/134 2023-10-16T14:22:50.613 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:50.649 DEBUG:teuthology.parallel:result is None 2023-10-16T14:22:50.675 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 111/134 2023-10-16T14:22:50.818 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 126/134 2023-10-16T14:22:50.860 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:50.931 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 112/134 2023-10-16T14:22:51.144 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:51.173 INFO:teuthology.orchestra.run.smithi186.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-16T14:22:51.360 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T14:22:51.446 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T14:22:51.578 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 113/134 2023-10-16T14:22:51.644 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T14:22:51.656 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T14:22:51.785 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:51.828 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T14:22:51.835 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T14:22:51.971 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-16T14:22:51.971 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:52.205 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:52.205 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:22:52.424 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:22:52.626 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:53.135 INFO:teuthology.orchestra.run.smithi186.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:53.136 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 1/134 2023-10-16T14:22:53.136 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2/134 2023-10-16T14:22:53.136 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 3/134 2023-10-16T14:22:53.136 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 4/134 2023-10-16T14:22:53.137 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 5/134 2023-10-16T14:22:53.137 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:22:53.137 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:22:53.137 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 10/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 11/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 12/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 13/134 2023-10-16T14:22:53.138 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 14/134 2023-10-16T14:22:53.139 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 15/134 2023-10-16T14:22:53.139 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 16/134 2023-10-16T14:22:53.139 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 17/134 2023-10-16T14:22:53.139 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 18/134 2023-10-16T14:22:53.140 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 19/134 2023-10-16T14:22:53.140 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 20/134 2023-10-16T14:22:53.140 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 21/134 2023-10-16T14:22:53.140 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 22/134 2023-10-16T14:22:53.140 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 23/134 2023-10-16T14:22:53.141 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 24/134 2023-10-16T14:22:53.141 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 25/134 2023-10-16T14:22:53.141 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 26/134 2023-10-16T14:22:53.141 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 27/134 2023-10-16T14:22:53.141 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 28/134 2023-10-16T14:22:53.142 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 29/134 2023-10-16T14:22:53.142 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 30/134 2023-10-16T14:22:53.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 31/134 2023-10-16T14:22:53.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 32/134 2023-10-16T14:22:53.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 33/134 2023-10-16T14:22:53.151 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 34/134 2023-10-16T14:22:53.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 35/134 2023-10-16T14:22:53.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 36/134 2023-10-16T14:22:53.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 37/134 2023-10-16T14:22:53.152 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 38/134 2023-10-16T14:22:53.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-16T14:22:53.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-16T14:22:53.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-16T14:22:53.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-16T14:22:53.153 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-16T14:22:53.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-16T14:22:53.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-16T14:22:53.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-16T14:22:53.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-16T14:22:53.154 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-16T14:22:53.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-16T14:22:53.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-16T14:22:53.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-16T14:22:53.155 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-16T14:22:53.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-16T14:22:53.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-16T14:22:53.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-16T14:22:53.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-16T14:22:53.156 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-16T14:22:53.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-16T14:22:53.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-16T14:22:53.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-16T14:22:53.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-16T14:22:53.158 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-16T14:22:53.158 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-16T14:22:53.158 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-16T14:22:53.158 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-16T14:22:53.158 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-16T14:22:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-16T14:22:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-16T14:22:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-16T14:22:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-16T14:22:53.159 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-16T14:22:53.160 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-16T14:22:53.160 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-16T14:22:53.160 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-16T14:22:53.160 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-16T14:22:53.161 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-16T14:22:53.162 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-16T14:22:53.162 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-16T14:22:53.162 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-16T14:22:53.162 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-16T14:22:53.163 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-16T14:22:53.163 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-16T14:22:53.163 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-16T14:22:53.163 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-16T14:22:53.163 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-16T14:22:53.164 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-16T14:22:53.165 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-16T14:22:53.165 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-16T14:22:53.165 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-16T14:22:53.165 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-16T14:22:53.165 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-16T14:22:53.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-16T14:22:53.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-16T14:22:53.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-16T14:22:53.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-16T14:22:53.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-16T14:22:53.167 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-16T14:22:53.168 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-16T14:22:53.168 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-16T14:22:53.168 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-16T14:22:53.168 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-16T14:22:53.168 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-16T14:22:53.169 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-16T14:22:53.169 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-16T14:22:53.169 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-16T14:22:53.169 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-16T14:22:53.169 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-16T14:22:53.170 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-16T14:22:53.170 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-16T14:22:53.170 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-16T14:22:53.170 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-16T14:22:53.170 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-16T14:22:53.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-16T14:22:53.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 129/134 2023-10-16T14:22:53.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-16T14:22:53.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 131/134 2023-10-16T14:22:53.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:22:53.172 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-16T14:22:55.583 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:22:55.584 INFO:teuthology.orchestra.run.smithi186.stdout:Last metadata expiration check: 0:02:03 ago on Mon 16 Oct 2023 02:20:52 PM UTC. 2023-10-16T14:22:56.811 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 114/134 2023-10-16T14:22:58.058 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 115/134 2023-10-16T14:22:58.193 INFO:teuthology.orchestra.run.smithi186.stdout:Installed products updated. 2023-10-16T14:22:58.404 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:22:58.405 INFO:teuthology.orchestra.run.smithi186.stdout:Upgraded: 2023-10-16T14:22:58.406 INFO:teuthology.orchestra.run.smithi186.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-16T14:22:58.406 INFO:teuthology.orchestra.run.smithi186.stdout: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.406 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.406 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2023-10-16T14:22:58.406 INFO:teuthology.orchestra.run.smithi186.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-16T14:22:58.407 INFO:teuthology.orchestra.run.smithi186.stdout: automake-1.16.1-7.el8.noarch 2023-10-16T14:22:58.407 INFO:teuthology.orchestra.run.smithi186.stdout: bison-3.0.4-10.el8.x86_64 2023-10-16T14:22:58.407 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.407 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.407 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.408 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.408 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.408 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.408 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.408 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.409 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.409 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.409 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.409 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.409 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.410 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.410 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.410 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.410 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.410 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:22:58.411 INFO:teuthology.orchestra.run.smithi186.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-16T14:22:58.412 INFO:teuthology.orchestra.run.smithi186.stdout: flex-2.6.1-9.el8.x86_64 2023-10-16T14:22:58.412 INFO:teuthology.orchestra.run.smithi186.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-16T14:22:58.412 INFO:teuthology.orchestra.run.smithi186.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-16T14:22:58.412 INFO:teuthology.orchestra.run.smithi186.stdout: jq-1.6-3.el8.x86_64 2023-10-16T14:22:58.412 INFO:teuthology.orchestra.run.smithi186.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-16T14:22:58.413 INFO:teuthology.orchestra.run.smithi186.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-16T14:22:58.413 INFO:teuthology.orchestra.run.smithi186.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-16T14:22:58.413 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.413 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.413 INFO:teuthology.orchestra.run.smithi186.stdout: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.414 INFO:teuthology.orchestra.run.smithi186.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-16T14:22:58.414 INFO:teuthology.orchestra.run.smithi186.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:22:58.414 INFO:teuthology.orchestra.run.smithi186.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-16T14:22:58.414 INFO:teuthology.orchestra.run.smithi186.stdout: libpq-13.5-1.el8.x86_64 2023-10-16T14:22:58.414 INFO:teuthology.orchestra.run.smithi186.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.415 INFO:teuthology.orchestra.run.smithi186.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-16T14:22:58.416 INFO:teuthology.orchestra.run.smithi186.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-16T14:22:58.416 INFO:teuthology.orchestra.run.smithi186.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-16T14:22:58.416 INFO:teuthology.orchestra.run.smithi186.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-16T14:22:58.416 INFO:teuthology.orchestra.run.smithi186.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-16T14:22:58.416 INFO:teuthology.orchestra.run.smithi186.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-16T14:22:58.417 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-16T14:22:58.418 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-16T14:22:58.418 INFO:teuthology.orchestra.run.smithi186.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-16T14:22:58.418 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:58.418 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:58.419 INFO:teuthology.orchestra.run.smithi186.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:22:58.419 INFO:teuthology.orchestra.run.smithi186.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-16T14:22:58.419 INFO:teuthology.orchestra.run.smithi186.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-16T14:22:58.419 INFO:teuthology.orchestra.run.smithi186.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-16T14:22:58.419 INFO:teuthology.orchestra.run.smithi186.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-16T14:22:58.420 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-16T14:22:58.420 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.420 INFO:teuthology.orchestra.run.smithi186.stdout: python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.420 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.420 INFO:teuthology.orchestra.run.smithi186.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-16T14:22:58.421 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-16T14:22:58.421 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-16T14:22:58.421 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-16T14:22:58.421 INFO:teuthology.orchestra.run.smithi186.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-16T14:22:58.421 INFO:teuthology.orchestra.run.smithi186.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-16T14:22:58.422 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-16T14:22:58.422 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-16T14:22:58.422 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-16T14:22:58.422 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-16T14:22:58.422 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-16T14:22:58.423 INFO:teuthology.orchestra.run.smithi186.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-16T14:22:58.424 INFO:teuthology.orchestra.run.smithi186.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-16T14:22:58.424 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-16T14:22:58.424 INFO:teuthology.orchestra.run.smithi186.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-16T14:22:58.424 INFO:teuthology.orchestra.run.smithi186.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-16T14:22:58.424 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-16T14:22:58.425 INFO:teuthology.orchestra.run.smithi186.stdout: python3-portend-2.6-1.el8.noarch 2023-10-16T14:22:58.425 INFO:teuthology.orchestra.run.smithi186.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-16T14:22:58.425 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-16T14:22:58.425 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.426 INFO:teuthology.orchestra.run.smithi186.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-16T14:22:58.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-16T14:22:58.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-16T14:22:58.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-16T14:22:58.427 INFO:teuthology.orchestra.run.smithi186.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-16T14:22:58.428 INFO:teuthology.orchestra.run.smithi186.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-16T14:22:58.428 INFO:teuthology.orchestra.run.smithi186.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-16T14:22:58.428 INFO:teuthology.orchestra.run.smithi186.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-16T14:22:58.428 INFO:teuthology.orchestra.run.smithi186.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-16T14:22:58.428 INFO:teuthology.orchestra.run.smithi186.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-16T14:22:58.429 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-16T14:22:58.429 INFO:teuthology.orchestra.run.smithi186.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-16T14:22:58.429 INFO:teuthology.orchestra.run.smithi186.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-16T14:22:58.429 INFO:teuthology.orchestra.run.smithi186.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-16T14:22:58.429 INFO:teuthology.orchestra.run.smithi186.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-16T14:22:58.430 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.430 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.430 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:22:58.430 INFO:teuthology.orchestra.run.smithi186.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-16T14:22:58.430 INFO:teuthology.orchestra.run.smithi186.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-16T14:22:58.431 INFO:teuthology.orchestra.run.smithi186.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-16T14:22:58.431 INFO:teuthology.orchestra.run.smithi186.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-16T14:22:58.431 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-16T14:22:58.432 INFO:teuthology.orchestra.run.smithi186.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-16T14:22:58.432 INFO:teuthology.orchestra.run.smithi186.stdout: 2023-10-16T14:22:58.432 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2023-10-16T14:22:58.434 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-10-16T14:22:58.602 DEBUG:teuthology.parallel:result is None 2023-10-16T14:22:58.664 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 117/134 2023-10-16T14:22:58.935 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-10-16T14:22:59.150 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T14:22:59.400 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-10-16T14:22:59.672 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-10-16T14:22:59.871 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 121/134 2023-10-16T14:23:00.059 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-10-16T14:23:03.460 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:23:03.758 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:23:03.797 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-10-16T14:23:04.117 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 124/134 2023-10-16T14:23:04.384 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 125/134 2023-10-16T14:23:04.741 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 126/134 2023-10-16T14:23:05.094 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-10-16T14:23:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T14:23:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-10-16T14:23:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T14:23:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-10-16T14:23:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T14:23:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-10-16T14:23:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-10-16T14:23:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:23:06.430 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:23:06.430 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:23:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-10-16T14:23:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:23:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:23:07.245 INFO:teuthology.orchestra.run.smithi178.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-16T14:23:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:23:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 1/134 2023-10-16T14:23:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2/134 2023-10-16T14:23:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 3/134 2023-10-16T14:23:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 4/134 2023-10-16T14:23:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a 5/134 2023-10-16T14:23:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 6/134 2023-10-16T14:23:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 7/134 2023-10-16T14:23:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 8/134 2023-10-16T14:23:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 9/134 2023-10-16T14:23:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 10/134 2023-10-16T14:23:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 11/134 2023-10-16T14:23:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 12/134 2023-10-16T14:23:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 13/134 2023-10-16T14:23:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_ 14/134 2023-10-16T14:23:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 15/134 2023-10-16T14:23:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 16/134 2023-10-16T14:23:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_6 17/134 2023-10-16T14:23:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86 18/134 2023-10-16T14:23:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 19/134 2023-10-16T14:23:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el 20/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8. 21/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_6 22/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 23/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 24/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 25/134 2023-10-16T14:23:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 26/134 2023-10-16T14:23:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 27/134 2023-10-16T14:23:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 28/134 2023-10-16T14:23:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190. 29/134 2023-10-16T14:23:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noa 30/134 2023-10-16T14:23:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.n 31/134 2023-10-16T14:23:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba 32/134 2023-10-16T14:23:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el 33/134 2023-10-16T14:23:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 34/134 2023-10-16T14:23:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.e 35/134 2023-10-16T14:23:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 36/134 2023-10-16T14:23:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 37/134 2023-10-16T14:23:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 38/134 2023-10-16T14:23:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-10-16T14:23:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-10-16T14:23:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-10-16T14:23:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-10-16T14:23:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-10-16T14:23:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-10-16T14:23:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-10-16T14:23:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-10-16T14:23:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-10-16T14:23:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-10-16T14:23:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-10-16T14:23:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-10-16T14:23:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-10-16T14:23:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-10-16T14:23:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-10-16T14:23:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-10-16T14:23:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-10-16T14:23:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-10-16T14:23:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-10-16T14:23:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-10-16T14:23:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-10-16T14:23:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-10-16T14:23:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-10-16T14:23:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-10-16T14:23:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-10-16T14:23:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-10-16T14:23:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-10-16T14:23:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-10-16T14:23:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-10-16T14:23:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-10-16T14:23:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-10-16T14:23:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-10-16T14:23:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-10-16T14:23:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-10-16T14:23:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-10-16T14:23:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-10-16T14:23:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-10-16T14:23:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-10-16T14:23:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-10-16T14:23:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-10-16T14:23:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-10-16T14:23:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-10-16T14:23:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-10-16T14:23:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-10-16T14:23:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-10-16T14:23:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-10-16T14:23:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-10-16T14:23:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-10-16T14:23:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-10-16T14:23:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-10-16T14:23:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-10-16T14:23:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-10-16T14:23:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 129/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 131/134 2023-10-16T14:23:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-10-16T14:23:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-10-16T14:23:10.710 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-10-16T14:23:10.711 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:02:17 ago on Mon 16 Oct 2023 02:20:53 PM UTC. 2023-10-16T14:23:13.877 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-10-16T14:23:14.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:23:14.121 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2023-10-16T14:23:14.122 INFO:teuthology.orchestra.run.smithi178.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-10-16T14:23:14.122 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.122 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.122 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-10-16T14:23:14.122 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-10-16T14:23:14.123 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.1-7.el8.noarch 2023-10-16T14:23:14.123 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.0.4-10.el8.x86_64 2023-10-16T14:23:14.123 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.123 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.123 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.124 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.125 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.126 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.127 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.127 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.127 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.127 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:18.0.0-6730.g4ba8a190.el8.noarch 2023-10-16T14:23:14.127 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.1-9.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.7-15.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-3.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-16T14:23:14.128 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-10-16T14:23:14.129 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-16T14:23:14.129 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.129 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.129 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.129 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-10-16T14:23:14.130 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:23:14.130 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-16T14:23:14.130 INFO:teuthology.orchestra.run.smithi178.stdout: libpq-13.5-1.el8.x86_64 2023-10-16T14:23:14.130 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-10-16T14:23:14.130 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-16T14:23:14.131 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.131 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.131 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-16T14:23:14.131 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.131 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-16T14:23:14.132 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-16T14:23:14.133 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-16T14:23:14.133 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.15-3.el8.x86_64 2023-10-16T14:23:14.133 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-10-16T14:23:14.133 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-10-16T14:23:14.133 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-10-16T14:23:14.134 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-16T14:23:14.135 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-16T14:23:14.135 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-16T14:23:14.135 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-16T14:23:14.135 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-16T14:23:14.135 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.136 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.136 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.136 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-16T14:23:14.136 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-10-16T14:23:14.136 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-16T14:23:14.137 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-16T14:23:14.138 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-16T14:23:14.138 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-16T14:23:14.138 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-16T14:23:14.138 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-16T14:23:14.138 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-16T14:23:14.139 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-16T14:23:14.140 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-16T14:23:14.140 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-16T14:23:14.140 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-2.6-1.el8.noarch 2023-10-16T14:23:14.140 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-16T14:23:14.140 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.141 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-16T14:23:14.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-16T14:23:14.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-16T14:23:14.142 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-16T14:23:14.143 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-10-16T14:23:14.143 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-16T14:23:14.143 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-16T14:23:14.143 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-16T14:23:14.143 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-16T14:23:14.145 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-16T14:23:14.145 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-16T14:23:14.145 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-16T14:23:14.145 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-16T14:23:14.146 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-16T14:23:14.146 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-16T14:23:14.146 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.146 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.146 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:18.0.0-6730.g4ba8a190.el8.x86_64 2023-10-16T14:23:14.147 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-16T14:23:14.147 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-16T14:23:14.147 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-43.el8.x86_64 2023-10-16T14:23:14.147 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-10-16T14:23:14.147 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-16T14:23:14.148 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-10-16T14:23:14.148 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:23:14.148 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-10-16T14:23:14.505 DEBUG:teuthology.parallel:result is None 2023-10-16T14:23:14.505 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:14.732 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T14:23:14.789 INFO:teuthology.orchestra.run.smithi178.stdout:18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:14.790 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:14.790 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730.g4ba8a190 is installed. 2023-10-16T14:23:14.791 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:14.935 DEBUG:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T14:23:14.964 INFO:teuthology.orchestra.run.smithi186.stdout:18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:14.964 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:14.964 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730.g4ba8a190 is installed. 2023-10-16T14:23:14.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:15.184 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-16T14:23:15.213 INFO:teuthology.orchestra.run.smithi187.stdout:18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:15.214 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6730.g4ba8a190.el8 2023-10-16T14:23:15.214 INFO:teuthology.task.install:The correct ceph version 18.0.0-6730.g4ba8a190 is installed. 2023-10-16T14:23:15.215 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T14:23:15.215 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:23:15.215 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T14:23:15.309 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:23:15.310 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T14:23:15.358 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:23:15.358 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T14:23:15.406 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T14:23:15.407 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:23:15.407 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T14:23:15.477 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T14:23:15.587 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:23:15.587 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T14:23:15.620 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T14:23:15.690 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:23:15.690 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T14:23:15.722 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T14:23:15.792 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T14:23:15.792 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:23:15.793 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T14:23:15.863 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T14:23:15.972 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:23:15.973 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T14:23:16.004 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T14:23:16.074 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:23:16.074 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T14:23:16.106 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T14:23:16.176 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T14:23:16.177 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:23:16.177 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T14:23:16.245 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T14:23:16.352 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:23:16.353 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T14:23:16.384 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T14:23:16.455 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:23:16.456 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T14:23:16.487 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T14:23:16.556 INFO:teuthology.run_tasks:Running task cephadm... 2023-10-16T14:23:16.638 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-16T14:23:16.639 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:16.639 INFO:tasks.cephadm:Cluster fsid is 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:23:16.640 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-10-16T14:23:16.640 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.178', 'mon.b': '172.21.15.186', 'mon.c': '172.21.15.187'} 2023-10-16T14:23:16.640 INFO:tasks.cephadm:First mon is mon.a on smithi178 2023-10-16T14:23:16.641 INFO:tasks.cephadm:First mgr is x 2023-10-16T14:23:16.641 INFO:tasks.cephadm:Normalizing hostnames... 2023-10-16T14:23:16.642 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2023-10-16T14:23:16.714 DEBUG:teuthology.orchestra.run.smithi186:> sudo hostname $(hostname -s) 2023-10-16T14:23:16.746 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname $(hostname -s) 2023-10-16T14:23:16.777 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-10-16T14:23:16.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:16.921 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73317/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6730-g4ba8a190', 'node_name': '172.21.4.66+confusa04', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6730.g4ba8a190'}, 'url': 'https://2.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-13 15:58:27.010264', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023oct13', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/73317/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-6730-g4ba8a190', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-6730.g4ba8a190'}, 'url': 'https://4.chacra.ceph.com/r/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-10-13 15:04:44.383342', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'rishabh-2023oct13', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-10-16T14:23:17.062 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref rishabh-2023oct13, sha1 4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 2023-10-16T14:23:17.062 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/x86_64/flavors/default/cephadm 2023-10-16T14:23:17.063 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:17.533 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 16 14:23 /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:17.533 DEBUG:teuthology.orchestra.run.smithi186:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.021 INFO:teuthology.orchestra.run.smithi186.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 16 14:23 /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.022 DEBUG:teuthology.orchestra.run.smithi187:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/rishabh-2023oct13/4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.433 INFO:teuthology.orchestra.run.smithi187.stdout:-rw-rw-r--. 1 ubuntu ubuntu 789665 Oct 16 14:23 /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.434 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.487 DEBUG:teuthology.orchestra.run.smithi186:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.510 DEBUG:teuthology.orchestra.run.smithi187:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-10-16T14:23:18.547 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 on all hosts... 2023-10-16T14:23:18.548 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 pull 2023-10-16T14:23:18.553 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 pull 2023-10-16T14:23:18.555 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 pull 2023-10-16T14:23:19.883 INFO:teuthology.orchestra.run.smithi187.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0... 2023-10-16T14:23:20.228 INFO:teuthology.orchestra.run.smithi186.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0... 2023-10-16T14:23:23.035 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0... 2023-10-16T14:23:49.069 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-10-16T14:23:49.070 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T14:23:49.070 INFO:teuthology.orchestra.run.smithi187.stdout: "image_id": "6c414e27a22dd493601dbea027b085af1f7a6d0d2000c05baef3637f1554baac", 2023-10-16T14:23:49.070 INFO:teuthology.orchestra.run.smithi187.stdout: "repo_digests": [ 2023-10-16T14:23:49.071 INFO:teuthology.orchestra.run.smithi187.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6730639e35896afbb2563e998f4d6436eacd40f6e8724777584c413b142f6992" 2023-10-16T14:23:49.071 INFO:teuthology.orchestra.run.smithi187.stdout: ] 2023-10-16T14:23:49.071 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-10-16T14:23:49.122 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T14:23:49.122 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T14:23:49.123 INFO:teuthology.orchestra.run.smithi186.stdout: "image_id": "6c414e27a22dd493601dbea027b085af1f7a6d0d2000c05baef3637f1554baac", 2023-10-16T14:23:49.123 INFO:teuthology.orchestra.run.smithi186.stdout: "repo_digests": [ 2023-10-16T14:23:49.123 INFO:teuthology.orchestra.run.smithi186.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6730639e35896afbb2563e998f4d6436eacd40f6e8724777584c413b142f6992" 2023-10-16T14:23:49.123 INFO:teuthology.orchestra.run.smithi186.stdout: ] 2023-10-16T14:23:49.123 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T14:23:57.237 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:23:57.238 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T14:23:57.238 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "6c414e27a22dd493601dbea027b085af1f7a6d0d2000c05baef3637f1554baac", 2023-10-16T14:23:57.238 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2023-10-16T14:23:57.238 INFO:teuthology.orchestra.run.smithi178.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6730639e35896afbb2563e998f4d6436eacd40f6e8724777584c413b142f6992" 2023-10-16T14:23:57.238 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:23:57.239 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:23:57.264 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2023-10-16T14:23:57.337 DEBUG:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph 2023-10-16T14:23:57.374 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph 2023-10-16T14:23:57.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2023-10-16T14:23:57.479 DEBUG:teuthology.orchestra.run.smithi186:> sudo chmod 777 /etc/ceph 2023-10-16T14:23:57.509 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /etc/ceph 2023-10-16T14:23:57.541 INFO:tasks.cephadm:Writing seed config... 2023-10-16T14:23:57.542 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-10-16T14:23:57.542 INFO:tasks.cephadm: override: [client] debug client = 20 2023-10-16T14:23:57.543 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-10-16T14:23:57.543 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-10-16T14:23:57.543 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-10-16T14:23:57.543 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-10-16T14:23:57.544 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-10-16T14:23:57.544 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-10-16T14:23:57.544 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-10-16T14:23:57.544 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-10-16T14:23:57.545 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-10-16T14:23:57.545 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-10-16T14:23:57.545 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-10-16T14:23:57.545 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-10-16T14:23:57.546 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-10-16T14:23:57.546 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-10-16T14:23:57.546 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-10-16T14:23:57.547 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-10-16T14:23:57.547 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-10-16T14:23:57.547 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-10-16T14:23:57.547 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-10-16T14:23:57.548 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-10-16T14:23:57.548 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-10-16T14:23:57.548 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-10-16T14:23:57.548 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-10-16T14:23:57.549 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-10-16T14:23:57.549 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-10-16T14:23:57.549 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-10-16T14:23:57.549 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-10-16T14:23:57.550 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-10-16T14:23:57.550 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-10-16T14:23:57.550 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-10-16T14:23:57.550 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-10-16T14:23:57.551 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-10-16T14:23:57.551 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-10-16T14:23:57.551 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-10-16T14:23:57.551 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-10-16T14:23:57.552 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-10-16T14:23:57.552 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-10-16T14:23:57.552 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-10-16T14:23:57.552 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-10-16T14:23:57.553 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-10-16T14:23:57.554 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:23:57.554 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-10-16T14:23:57.597 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-10-16T14:23:57.598 DEBUG:teuthology.orchestra.run.smithi178:mon.a> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service 2023-10-16T14:23:57.642 DEBUG:teuthology.orchestra.run.smithi178:mgr.x> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x.service 2023-10-16T14:23:57.687 INFO:tasks.cephadm:Bootstrapping... 2023-10-16T14:23:57.688 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 -v bootstrap --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.178 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:23:57.709 INFO:journalctl@ceph.mon.a.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:23:57.756 INFO:journalctl@ceph.mgr.x.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:23:57.920 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-10-16T14:23:57.921 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0', '-v', 'bootstrap', '--fsid', '8c31bde4-6c2f-11ee-8db6-212e2dc638e7', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.178', '--skip-admin-label'] 2023-10-16T14:23:58.095 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.1.1 2023-10-16T14:23:58.096 INFO:teuthology.orchestra.run.smithi178.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-10-16T14:23:58.096 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying podman|docker is present... 2023-10-16T14:23:58.267 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.1.1 2023-10-16T14:23:58.268 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying lvm2 is present... 2023-10-16T14:23:58.268 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying time synchronization is in place... 2023-10-16T14:23:58.282 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-16T14:23:58.282 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-16T14:23:58.294 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-16T14:23:58.295 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:23:58.308 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout enabled 2023-10-16T14:23:58.320 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout active 2023-10-16T14:23:58.321 INFO:teuthology.orchestra.run.smithi178.stdout:Unit chronyd.service is enabled and running 2023-10-16T14:23:58.321 INFO:teuthology.orchestra.run.smithi178.stdout:Repeating the final host check... 2023-10-16T14:23:58.492 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 4.1.1 2023-10-16T14:23:58.492 INFO:teuthology.orchestra.run.smithi178.stdout:podman (/bin/podman) version 4.1.1 is present 2023-10-16T14:23:58.492 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl is present 2023-10-16T14:23:58.492 INFO:teuthology.orchestra.run.smithi178.stdout:lvcreate is present 2023-10-16T14:23:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-10-16T14:23:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-10-16T14:23:58.521 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-10-16T14:23:58.521 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:23:58.531 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout enabled 2023-10-16T14:23:58.544 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout active 2023-10-16T14:23:58.545 INFO:teuthology.orchestra.run.smithi178.stdout:Unit chronyd.service is enabled and running 2023-10-16T14:23:58.545 INFO:teuthology.orchestra.run.smithi178.stdout:Host looks OK 2023-10-16T14:23:58.545 INFO:teuthology.orchestra.run.smithi178.stdout:Cluster fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:23:58.545 INFO:teuthology.orchestra.run.smithi178.stdout:Acquiring lock 140496584866280 on /run/cephadm/8c31bde4-6c2f-11ee-8db6-212e2dc638e7.lock 2023-10-16T14:23:58.545 INFO:teuthology.orchestra.run.smithi178.stdout:Lock 140496584866280 acquired on /run/cephadm/8c31bde4-6c2f-11ee-8db6-212e2dc638e7.lock 2023-10-16T14:23:58.546 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying IP 172.21.15.178 port 3300 ... 2023-10-16T14:23:58.546 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying IP 172.21.15.178 port 6789 ... 2023-10-16T14:23:58.546 INFO:teuthology.orchestra.run.smithi178.stdout:Base mon IP(s) is [172.21.15.178:3300, 172.21.15.178:6789], mon addrv is [v2:172.21.15.178:3300,v1:172.21.15.178:6789] 2023-10-16T14:23:58.553 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-10-16T14:23:58.555 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.178 metric 100 2023-10-16T14:23:58.560 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-10-16T14:23:58.563 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-10-16T14:23:58.563 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-10-16T14:23:58.566 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-10-16T14:23:58.567 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-10-16T14:23:58.567 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-16T14:23:58.568 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-10-16T14:23:58.568 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout inet6 fe80::92fb:5c5a:625:714e/64 scope link noprefixroute 2023-10-16T14:23:58.568 INFO:teuthology.orchestra.run.smithi178.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-10-16T14:23:58.569 INFO:teuthology.orchestra.run.smithi178.stdout:Mon IP `172.21.15.178` is in CIDR network `172.21.0.0/20` 2023-10-16T14:23:58.569 INFO:teuthology.orchestra.run.smithi178.stdout:Mon IP `172.21.15.178` is in CIDR network `172.21.0.0/20` 2023-10-16T14:23:58.569 INFO:teuthology.orchestra.run.smithi178.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-10-16T14:23:58.569 INFO:teuthology.orchestra.run.smithi178.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-10-16T14:23:58.570 INFO:teuthology.orchestra.run.smithi178.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0... 2023-10-16T14:23:59.045 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stdout 6c414e27a22dd493601dbea027b085af1f7a6d0d2000c05baef3637f1554baac 2023-10-16T14:23:59.045 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0... 2023-10-16T14:23:59.045 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Getting image source signatures 2023-10-16T14:23:59.046 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying blob sha256:a436693b466fb7b19e09eee08a577fdd5da6c190e9ac5a29f91603f2f63125a2 2023-10-16T14:23:59.046 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying blob sha256:8ed15414cb45b8b895a21f106743ceee45272cee0d9e7e5b1f49bd1e877e4a11 2023-10-16T14:23:59.046 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Copying config sha256:6c414e27a22dd493601dbea027b085af1f7a6d0d2000c05baef3637f1554baac 2023-10-16T14:23:59.046 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Writing manifest to image destination 2023-10-16T14:23:59.046 INFO:teuthology.orchestra.run.smithi178.stdout:/bin/podman: stderr Storing signatures 2023-10-16T14:24:00.122 INFO:teuthology.orchestra.run.smithi178.stdout:ceph: stdout ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev) 2023-10-16T14:24:00.122 INFO:teuthology.orchestra.run.smithi178.stdout:Ceph version: ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev) 2023-10-16T14:24:00.122 INFO:teuthology.orchestra.run.smithi178.stdout:Extracting ceph user uid/gid from container image... 2023-10-16T14:24:00.898 INFO:teuthology.orchestra.run.smithi178.stdout:stat: stdout 167 167 2023-10-16T14:24:00.898 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial keys... 2023-10-16T14:24:01.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQCBRy1loVzoEhAAngbP7pA5dWb4PdfVCKL5dA== 2023-10-16T14:24:02.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQCCRy1lN09lDBAA1l7lBl8h/ZPrn2Jnwwz2rQ== 2023-10-16T14:24:03.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph-authtool: stdout AQCDRy1l+/FwBRAAnkrEA27JYYj7SWhSAQjB/A== 2023-10-16T14:24:03.929 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial monmap... 2023-10-16T14:24:04.808 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-16T14:24:04.808 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-10-16T14:24:04.808 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout:monmaptool for a [v2:172.21.15.178:3300,v1:172.21.15.178:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout:setting min_mon_release = pacific 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: set fsid to 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-10-16T14:24:04.809 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:24:04.810 INFO:teuthology.orchestra.run.smithi178.stdout:Creating mon... 2023-10-16T14:24:05.684 INFO:teuthology.orchestra.run.smithi178.stdout:create mon.a on 2023-10-16T14:24:05.926 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-10-16T14:24:06.123 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-10-16T14:24:06.359 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target. 2023-10-16T14:24:06.359 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target → /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target. 2023-10-16T14:24:06.699 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a 2023-10-16T14:24:06.699 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service: Unit ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service not loaded. 2023-10-16T14:24:06.896 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target.wants/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service → /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@.service. 2023-10-16T14:24:07.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:06 smithi178 systemd[1]: Starting Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:24:07.664 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:07 smithi178 podman[97953]: 2023-10-16T14:24:07.711 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-16T14:24:07.711 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2023-10-16T14:24:07.726 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-16T14:24:07.727 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:24:07.727 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2023-10-16T14:24:07.727 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to enable service . firewalld.service is not available 2023-10-16T14:24:07.727 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mon to start... 2023-10-16T14:24:07.728 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mon... 2023-10-16T14:24:08.045 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:07 smithi178 bash[97953]: ffacaa592d7f39343b8a31455d8803eb891974910c3d7163a5adf591435dd8c7 2023-10-16T14:24:08.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:07 smithi178 ceph-mon[97993]: mkfs 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:08.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:07 smithi178 ceph-mon[97993]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-16T14:24:08.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:07 smithi178 systemd[1]: Started Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:24:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-16T14:24:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: monmap e1: 1 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} 2023-10-16T14:24:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: fsmap 2023-10-16T14:24:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: osdmap e1: 0 total, 0 up, 0 in 2023-10-16T14:24:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: mgrmap e1: no daemons active 2023-10-16T14:24:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:08 smithi178 ceph-mon[97993]: from='client.? 172.21.15.178:0/2354305750' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-10-16T14:24:10.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout cluster: 2023-10-16T14:24:10.219 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout id: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:10.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-10-16T14:24:10.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:10.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout services: 2023-10-16T14:24:10.220 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.732014s) 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout data: 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-10-16T14:24:10.221 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-10-16T14:24:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-10-16T14:24:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout pgs: 2023-10-16T14:24:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.320+0000 7f11a001f700 1 Processor -- start 2023-10-16T14:24:10.222 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.320+0000 7f11a001f700 1 -- start start 2023-10-16T14:24:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.321+0000 7f11a001f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.321+0000 7f11a001f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119807a8c0 con 0x7f119807be90 2023-10-16T14:24:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.322+0000 7f119ddbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:10.223 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.322+0000 7f119ddbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56080/0 (socket says 172.21.15.178:56080) 2023-10-16T14:24:10.224 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.322+0000 7f119ddbb700 1 -- 172.21.15.178:0/304514718 learned_addr learned my addr 172.21.15.178:0/304514718 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:10.224 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.322+0000 7f119ddbb700 1 -- 172.21.15.178:0/304514718 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119807aa00 con 0x7f119807be90 2023-10-16T14:24:10.224 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.322+0000 7f119ddbb700 1 --2- 172.21.15.178:0/304514718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11940098a0 tx=0x7f1194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263a7b67fa0e8e93 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:10.224 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f119cdb9700 1 -- 172.21.15.178:0/304514718 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1194004160 con 0x7f119807be90 2023-10-16T14:24:10.224 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f119cdb9700 1 -- 172.21.15.178:0/304514718 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11940042c0 con 0x7f119807be90 2023-10-16T14:24:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f119cdb9700 1 -- 172.21.15.178:0/304514718 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11940045a0 con 0x7f119807be90 2023-10-16T14:24:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f11a001f700 1 -- 172.21.15.178:0/304514718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 msgr2=0x7f119807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f11a001f700 1 --2- 172.21.15.178:0/304514718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f11940098a0 tx=0x7f1194009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f11a001f700 1 -- 172.21.15.178:0/304514718 shutdown_connections 2023-10-16T14:24:10.225 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f11a001f700 1 --2- 172.21.15.178:0/304514718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119807a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:10.226 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.323+0000 7f11a001f700 1 -- 172.21.15.178:0/304514718 >> 172.21.15.178:0/304514718 conn(0x7f1198101b70 msgr2=0x7f1198103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:10.226 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.324+0000 7f11a001f700 1 -- 172.21.15.178:0/304514718 shutdown_connections 2023-10-16T14:24:10.226 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.324+0000 7f11a001f700 1 -- 172.21.15.178:0/304514718 wait complete. 2023-10-16T14:24:10.226 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.325+0000 7f11a001f700 1 Processor -- start 2023-10-16T14:24:10.226 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.325+0000 7f11a001f700 1 -- start start 2023-10-16T14:24:10.227 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f11a001f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:10.227 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f11a001f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119819c180 con 0x7f119807be90 2023-10-16T14:24:10.227 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f119ddbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:10.227 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f119ddbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56092/0 (socket says 172.21.15.178:56092) 2023-10-16T14:24:10.227 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f119ddbb700 1 -- 172.21.15.178:0/2354305750 learned_addr learned my addr 172.21.15.178:0/2354305750 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.326+0000 7f119ddbb700 1 -- 172.21.15.178:0/2354305750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194009550 con 0x7f119807be90 2023-10-16T14:24:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f119ddbb700 1 --2- 172.21.15.178:0/2354305750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f119807b360 tx=0x7f1194004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1194011630 con 0x7f119807be90 2023-10-16T14:24:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194011790 con 0x7f119807be90 2023-10-16T14:24:10.228 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1194011a30 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119819c380 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119819c7a0 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1194022070 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.327+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f119401c450 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.328+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1198195b70 con 0x7f119807be90 2023-10-16T14:24:10.229 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.331+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f119401b450 con 0x7f119807be90 2023-10-16T14:24:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.403+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f119807ad70 con 0x7f119807be90 2023-10-16T14:24:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.403+0000 7f118effd700 1 -- 172.21.15.178:0/2354305750 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f119401b5f0 con 0x7f119807be90 2023-10-16T14:24:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 msgr2=0x7f119819bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:10.230 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 --2- 172.21.15.178:0/2354305750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f119807b360 tx=0x7f1194004750 comp rx=0 tx=0).stop 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 shutdown_connections 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 --2- 172.21.15.178:0/2354305750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f119807be90 0x7f119819bc40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 >> 172.21.15.178:0/2354305750 conn(0x7f1198101b70 msgr2=0x7f1198103490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 shutdown_connections 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:08.404+0000 7f11a001f700 1 -- 172.21.15.178:0/2354305750 wait complete. 2023-10-16T14:24:10.231 INFO:teuthology.orchestra.run.smithi178.stdout:mon is available 2023-10-16T14:24:10.232 INFO:teuthology.orchestra.run.smithi178.stdout:Assimilating anything we can from ceph.conf... 2023-10-16T14:24:11.290 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:10 smithi178 ceph-mon[97993]: from='client.? 172.21.15.178:0/2848479875' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-16T14:24:11.291 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:10 smithi178 ceph-mon[97993]: from='client.? 172.21.15.178:0/2848479875' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-10-16T14:24:11.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:11.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [global] 2023-10-16T14:24:11.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout fsid = 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:11.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.178:3300,v1:172.21.15.178:6789] 2023-10-16T14:24:11.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-16T14:24:11.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-16T14:24:11.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-16T14:24:11.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-16T14:24:11.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-16T14:24:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-16T14:24:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [osd] 2023-10-16T14:24:11.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-16T14:24:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-16T14:24:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-16T14:24:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-16T14:24:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-16T14:24:11.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.777+0000 7f05fe104700 1 Processor -- start 2023-10-16T14:24:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.777+0000 7f05fe104700 1 -- start start 2023-10-16T14:24:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05fe104700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05fe104700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8106f80 con 0x7f05f8106590 2023-10-16T14:24:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:11.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45112/0 (socket says 172.21.15.178:45112) 2023-10-16T14:24:11.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05f77fe700 1 -- 172.21.15.178:0/2065829525 learned_addr learned my addr 172.21.15.178:0/2065829525 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:11.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.778+0000 7f05f77fe700 1 -- 172.21.15.178:0/2065829525 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8107790 con 0x7f05f8106590 2023-10-16T14:24:11.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05f77fe700 1 --2- 172.21.15.178:0/2065829525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05e80098a0 tx=0x7f05e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387cc8abaec2c2f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:11.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05f67fc700 1 -- 172.21.15.178:0/2065829525 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e8004160 con 0x7f05f8106590 2023-10-16T14:24:11.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05f67fc700 1 -- 172.21.15.178:0/2065829525 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e80042c0 con 0x7f05f8106590 2023-10-16T14:24:11.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05f67fc700 1 -- 172.21.15.178:0/2065829525 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e80045a0 con 0x7f05f8106590 2023-10-16T14:24:11.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05fe104700 1 -- 172.21.15.178:0/2065829525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 msgr2=0x7f05f81069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:11.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.779+0000 7f05fe104700 1 --2- 172.21.15.178:0/2065829525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f05e80098a0 tx=0x7f05e8009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:11.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.780+0000 7f05fe104700 1 -- 172.21.15.178:0/2065829525 shutdown_connections 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.780+0000 7f05fe104700 1 --2- 172.21.15.178:0/2065829525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f81069b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.780+0000 7f05fe104700 1 -- 172.21.15.178:0/2065829525 >> 172.21.15.178:0/2065829525 conn(0x7f05f8101b70 msgr2=0x7f05f8103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.780+0000 7f05fe104700 1 -- 172.21.15.178:0/2065829525 shutdown_connections 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.780+0000 7f05fe104700 1 -- 172.21.15.178:0/2065829525 wait complete. 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05fe104700 1 Processor -- start 2023-10-16T14:24:11.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05fe104700 1 -- start start 2023-10-16T14:24:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05fe104700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05fe104700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f819c1b0 con 0x7f05f8106590 2023-10-16T14:24:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05f77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45116/0 (socket says 172.21.15.178:45116) 2023-10-16T14:24:11.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.781+0000 7f05f77fe700 1 -- 172.21.15.178:0/2848479875 learned_addr learned my addr 172.21.15.178:0/2848479875 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05f77fe700 1 -- 172.21.15.178:0/2848479875 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e8009550 con 0x7f05f8106590 2023-10-16T14:24:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05f77fe700 1 --2- 172.21.15.178:0/2848479875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f05e8004000 tx=0x7f05e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e8011630 con 0x7f05f8106590 2023-10-16T14:24:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e8011790 con 0x7f05f8106590 2023-10-16T14:24:11.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f819c3b0 con 0x7f05f8106590 2023-10-16T14:24:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05e8011a30 con 0x7f05f8106590 2023-10-16T14:24:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.782+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f819c7d0 con 0x7f05f8106590 2023-10-16T14:24:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.783+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f05e8022070 con 0x7f05f8106590 2023-10-16T14:24:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.783+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f05e801c450 con 0x7f05f8106590 2023-10-16T14:24:11.324 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.783+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f80624e0 con 0x7f05f8106590 2023-10-16T14:24:11.325 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.786+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f05e801b450 con 0x7f05f8106590 2023-10-16T14:24:11.325 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.855+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f05f8107b00 con 0x7f05f8106590 2023-10-16T14:24:11.325 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.859+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f05e801b5f0 con 0x7f05f8106590 2023-10-16T14:24:11.325 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.860+0000 7f05f4ff9700 1 -- 172.21.15.178:0/2848479875 <== mon.0 v2:172.21.15.178:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f05e801b790 con 0x7f05f8106590 2023-10-16T14:24:11.325 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 msgr2=0x7f05f819bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 --2- 172.21.15.178:0/2848479875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f05e8004000 tx=0x7f05e8004750 comp rx=0 tx=0).stop 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 shutdown_connections 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 --2- 172.21.15.178:0/2848479875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05f8106590 0x7f05f819bc70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 >> 172.21.15.178:0/2848479875 conn(0x7f05f8101b70 msgr2=0x7f05f807a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 shutdown_connections 2023-10-16T14:24:11.326 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:10.861+0000 7f05fe104700 1 -- 172.21.15.178:0/2848479875 wait complete. 2023-10-16T14:24:11.327 INFO:teuthology.orchestra.run.smithi178.stdout:Generating new minimal ceph.conf... 2023-10-16T14:24:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:11 smithi178 ceph-mon[97993]: from='client.? 172.21.15.178:0/2537324838' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:24:12.393 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.826+0000 7fd965224700 1 Processor -- start 2023-10-16T14:24:12.393 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.826+0000 7fd965224700 1 -- start start 2023-10-16T14:24:12.393 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd965224700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:12.393 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd965224700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd960106f80 con 0x7fd960106590 2023-10-16T14:24:12.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:12.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45120/0 (socket says 172.21.15.178:45120) 2023-10-16T14:24:12.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd95ed9d700 1 -- 172.21.15.178:0/806167970 learned_addr learned my addr 172.21.15.178:0/806167970 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:12.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.827+0000 7fd95ed9d700 1 -- 172.21.15.178:0/806167970 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960107790 con 0x7fd960106590 2023-10-16T14:24:12.394 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd95ed9d700 1 --2- 172.21.15.178:0/806167970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098a0 tx=0x7fd950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa3b415987bcaf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:12.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd95dd9b700 1 -- 172.21.15.178:0/806167970 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd950004160 con 0x7fd960106590 2023-10-16T14:24:12.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd95dd9b700 1 -- 172.21.15.178:0/806167970 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd9500042c0 con 0x7fd960106590 2023-10-16T14:24:12.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd95dd9b700 1 -- 172.21.15.178:0/806167970 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd950011a00 con 0x7fd960106590 2023-10-16T14:24:12.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd965224700 1 -- 172.21.15.178:0/806167970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 msgr2=0x7fd9601069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:12.395 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd965224700 1 --2- 172.21.15.178:0/806167970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9500098a0 tx=0x7fd950009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:12.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd965224700 1 -- 172.21.15.178:0/806167970 shutdown_connections 2023-10-16T14:24:12.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd965224700 1 --2- 172.21.15.178:0/806167970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd9601069b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:12.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.828+0000 7fd965224700 1 -- 172.21.15.178:0/806167970 >> 172.21.15.178:0/806167970 conn(0x7fd960101b70 msgr2=0x7fd960103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:12.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.829+0000 7fd965224700 1 -- 172.21.15.178:0/806167970 shutdown_connections 2023-10-16T14:24:12.396 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.829+0000 7fd965224700 1 -- 172.21.15.178:0/806167970 wait complete. 2023-10-16T14:24:12.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.829+0000 7fd965224700 1 Processor -- start 2023-10-16T14:24:12.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd965224700 1 -- start start 2023-10-16T14:24:12.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd965224700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:12.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd965224700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96019ca00 con 0x7fd960106590 2023-10-16T14:24:12.397 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:12.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd95ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45132/0 (socket says 172.21.15.178:45132) 2023-10-16T14:24:12.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd95ed9d700 1 -- 172.21.15.178:0/2537324838 learned_addr learned my addr 172.21.15.178:0/2537324838 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:12.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.830+0000 7fd95ed9d700 1 -- 172.21.15.178:0/2537324838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950009550 con 0x7fd960106590 2023-10-16T14:24:12.398 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd95ed9d700 1 --2- 172.21.15.178:0/2537324838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd950006af0 tx=0x7fd950011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:12.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd950003a40 con 0x7fd960106590 2023-10-16T14:24:12.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd950003ba0 con 0x7fd960106590 2023-10-16T14:24:12.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96019cc00 con 0x7fd960106590 2023-10-16T14:24:12.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd95001a700 con 0x7fd960106590 2023-10-16T14:24:12.399 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96019d020 con 0x7fd960106590 2023-10-16T14:24:12.400 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd95001ac80 con 0x7fd960106590 2023-10-16T14:24:12.400 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.831+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd950019700 con 0x7fd960106590 2023-10-16T14:24:12.400 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.832+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9600624e0 con 0x7fd960106590 2023-10-16T14:24:12.400 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.835+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fd950024ad0 con 0x7fd960106590 2023-10-16T14:24:12.400 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.904+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd96010a9c0 con 0x7fd960106590 2023-10-16T14:24:12.401 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.905+0000 7fd947fff700 1 -- 172.21.15.178:0/2537324838 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fd950024c70 con 0x7fd960106590 2023-10-16T14:24:12.401 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 msgr2=0x7fd96019c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:12.401 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 --2- 172.21.15.178:0/2537324838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd950006af0 tx=0x7fd950011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:12.401 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 shutdown_connections 2023-10-16T14:24:12.401 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 --2- 172.21.15.178:0/2537324838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd960106590 0x7fd96019c4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:12.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 >> 172.21.15.178:0/2537324838 conn(0x7fd960101b70 msgr2=0x7fd960103230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:12.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 shutdown_connections 2023-10-16T14:24:12.402 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:11.906+0000 7fd965224700 1 -- 172.21.15.178:0/2537324838 wait complete. 2023-10-16T14:24:12.402 INFO:teuthology.orchestra.run.smithi178.stdout:Restarting the monitor... 2023-10-16T14:24:12.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:12 smithi178 systemd[1]: Stopping Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:24:13.142 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[97969]: 2023-10-16T14:24:12.864+0000 7f4462400700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:24:13.143 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[97969]: 2023-10-16T14:24:12.864+0000 7f4462400700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-10-16T14:24:13.789 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:13 smithi178 bash[98332]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a 2023-10-16T14:24:14.282 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:13 smithi178 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service: Succeeded. 2023-10-16T14:24:14.283 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:13 smithi178 systemd[1]: Stopped Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:24:14.283 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:13 smithi178 systemd[1]: Starting Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:24:14.546 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 podman[98455]: 2023-10-16T14:24:14.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 bash[98455]: b22ad7fb7bba711f6b474ce770a474f977ab934d48f465c99ea5b241622e1c78 2023-10-16T14:24:14.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: set uid:gid to 167:167 (ceph:ceph) 2023-10-16T14:24:14.575 INFO:teuthology.orchestra.run.smithi178.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-10-16T14:24:14.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev), process ceph-mon, pid 7 2023-10-16T14:24:14.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: pidfile_write: ignore empty --pid-file 2023-10-16T14:24:14.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: load: jerasure load: lrc 2023-10-16T14:24:14.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: RocksDB version: 7.9.2 2023-10-16T14:24:14.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Git sha 0 2023-10-16T14:24:14.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Compile date 2023-10-13 13:48:46 2023-10-16T14:24:14.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: DB SUMMARY 2023-10-16T14:24:14.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: DB Session ID: 7UG2XLRNN2HJPTIKVXQK 2023-10-16T14:24:14.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: CURRENT file: CURRENT 2023-10-16T14:24:14.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: IDENTITY file: IDENTITY 2023-10-16T14:24:14.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-10-16T14:24:14.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-10-16T14:24:14.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98322 ; 2023-10-16T14:24:14.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.error_if_exists: 0 2023-10-16T14:24:14.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.create_if_missing: 0 2023-10-16T14:24:14.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.paranoid_checks: 1 2023-10-16T14:24:14.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-16T14:24:14.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-16T14:24:14.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-16T14:24:14.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.env: 0x55d89a88cda0 2023-10-16T14:24:14.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.fs: PosixFileSystem 2023-10-16T14:24:14.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.info_log: 0x55d89bba35e0 2023-10-16T14:24:14.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_file_opening_threads: 16 2023-10-16T14:24:14.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.statistics: (nil) 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.use_fsync: 0 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_log_file_size: 0 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.keep_log_file_num: 1000 2023-10-16T14:24:14.809 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.recycle_log_file_num: 0 2023-10-16T14:24:14.810 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_fallocate: 1 2023-10-16T14:24:14.810 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_mmap_reads: 0 2023-10-16T14:24:14.810 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_mmap_writes: 0 2023-10-16T14:24:14.810 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.use_direct_reads: 0 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.create_missing_column_families: 0 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.db_log_dir: 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.wal_dir: 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-16T14:24:14.811 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-16T14:24:14.813 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-16T14:24:14.813 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-16T14:24:14.813 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-16T14:24:14.814 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-16T14:24:14.814 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.advise_random_on_open: 1 2023-10-16T14:24:14.814 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.db_write_buffer_size: 0 2023-10-16T14:24:14.814 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.write_buffer_manager: 0x55d89bb5b360 2023-10-16T14:24:14.814 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-16T14:24:14.815 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-16T14:24:14.815 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-16T14:24:14.815 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.rate_limiter: (nil) 2023-10-16T14:24:14.815 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-16T14:24:14.815 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.wal_recovery_mode: 2 2023-10-16T14:24:14.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enable_thread_tracking: 0 2023-10-16T14:24:14.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enable_pipelined_write: 0 2023-10-16T14:24:14.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.unordered_write: 0 2023-10-16T14:24:14.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-16T14:24:14.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-16T14:24:14.817 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-16T14:24:14.817 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-16T14:24:14.817 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.row_cache: None 2023-10-16T14:24:14.817 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.wal_filter: None 2023-10-16T14:24:14.817 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-16T14:24:14.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_ingest_behind: 0 2023-10-16T14:24:14.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.two_write_queues: 0 2023-10-16T14:24:14.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.manual_wal_flush: 0 2023-10-16T14:24:14.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.wal_compression: 0 2023-10-16T14:24:14.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.atomic_flush: 0 2023-10-16T14:24:14.819 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-16T14:24:14.819 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-16T14:24:14.819 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-16T14:24:14.819 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.log_readahead_size: 0 2023-10-16T14:24:14.819 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-16T14:24:14.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.best_efforts_recovery: 0 2023-10-16T14:24:14.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-16T14:24:14.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-16T14:24:14.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.allow_data_in_errors: 0 2023-10-16T14:24:14.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.db_host_id: __hostname__ 2023-10-16T14:24:14.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-16T14:24:14.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_background_jobs: 2 2023-10-16T14:24:14.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_background_compactions: -1 2023-10-16T14:24:14.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_subcompactions: 1 2023-10-16T14:24:14.821 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-16T14:24:14.822 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-16T14:24:14.822 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-16T14:24:14.822 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_total_wal_size: 0 2023-10-16T14:24:14.822 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-16T14:24:14.822 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-16T14:24:14.823 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-16T14:24:14.823 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-16T14:24:14.823 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_open_files: -1 2023-10-16T14:24:14.823 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bytes_per_sync: 0 2023-10-16T14:24:14.823 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-16T14:24:14.824 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-16T14:24:14.824 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_readahead_size: 0 2023-10-16T14:24:14.824 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_background_flushes: -1 2023-10-16T14:24:14.824 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Compression algorithms supported: 2023-10-16T14:24:14.824 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-16T14:24:14.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kZSTD supported: 0 2023-10-16T14:24:14.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kXpressCompression supported: 0 2023-10-16T14:24:14.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kLZ4HCCompression supported: 1 2023-10-16T14:24:14.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kZlibCompression supported: 1 2023-10-16T14:24:14.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kSnappyCompression supported: 1 2023-10-16T14:24:14.826 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kLZ4Compression supported: 1 2023-10-16T14:24:14.826 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: kBZip2Compression supported: 0 2023-10-16T14:24:14.826 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-16T14:24:14.826 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-16T14:24:14.826 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-10-16T14:24:14.827 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-16T14:24:14.827 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-16T14:24:14.827 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.merge_operator: 2023-10-16T14:24:14.827 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_filter: None 2023-10-16T14:24:14.827 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_filter_factory: None 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.sst_partitioner_factory: None 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d89bba36e0) 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout: cache_index_and_filter_blocks: 1 2023-10-16T14:24:14.828 INFO:journalctl@ceph.mon.a.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-16T14:24:14.829 INFO:journalctl@ceph.mon.a.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-16T14:24:14.829 INFO:journalctl@ceph.mon.a.smithi178.stdout: pin_top_level_index_and_filter: 1 2023-10-16T14:24:14.829 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_type: 0 2023-10-16T14:24:14.829 INFO:journalctl@ceph.mon.a.smithi178.stdout: data_block_index_type: 0 2023-10-16T14:24:14.830 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_shortening: 1 2023-10-16T14:24:14.830 INFO:journalctl@ceph.mon.a.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-16T14:24:14.830 INFO:journalctl@ceph.mon.a.smithi178.stdout: checksum: 4 2023-10-16T14:24:14.830 INFO:journalctl@ceph.mon.a.smithi178.stdout: no_block_cache: 0 2023-10-16T14:24:14.831 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache: 0x55d89bbf91f0 2023-10-16T14:24:14.831 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_name: BinnedLRUCache 2023-10-16T14:24:14.831 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_options: 2023-10-16T14:24:14.831 INFO:journalctl@ceph.mon.a.smithi178.stdout: capacity : 536870912 2023-10-16T14:24:14.831 INFO:journalctl@ceph.mon.a.smithi178.stdout: num_shard_bits : 4 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: strict_capacity_limit : 0 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: high_pri_pool_ratio: 0.000 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_cache_compressed: (nil) 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: persistent_cache: (nil) 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_size: 4096 2023-10-16T14:24:14.832 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_size_deviation: 10 2023-10-16T14:24:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_restart_interval: 16 2023-10-16T14:24:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout: index_block_restart_interval: 1 2023-10-16T14:24:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout: metadata_block_size: 4096 2023-10-16T14:24:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout: partition_filters: 0 2023-10-16T14:24:14.833 INFO:journalctl@ceph.mon.a.smithi178.stdout: use_delta_encoding: 1 2023-10-16T14:24:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout: filter_policy: bloomfilter 2023-10-16T14:24:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout: whole_key_filtering: 1 2023-10-16T14:24:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout: verify_compression: 0 2023-10-16T14:24:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout: read_amp_bytes_per_bit: 0 2023-10-16T14:24:14.834 INFO:journalctl@ceph.mon.a.smithi178.stdout: format_version: 5 2023-10-16T14:24:14.835 INFO:journalctl@ceph.mon.a.smithi178.stdout: enable_index_compression: 1 2023-10-16T14:24:14.835 INFO:journalctl@ceph.mon.a.smithi178.stdout: block_align: 0 2023-10-16T14:24:14.835 INFO:journalctl@ceph.mon.a.smithi178.stdout: max_auto_readahead_size: 262144 2023-10-16T14:24:14.835 INFO:journalctl@ceph.mon.a.smithi178.stdout: prepopulate_block_cache: 0 2023-10-16T14:24:14.836 INFO:journalctl@ceph.mon.a.smithi178.stdout: initial_auto_readahead_size: 8192 2023-10-16T14:24:14.836 INFO:journalctl@ceph.mon.a.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2023-10-16T14:24:14.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.write_buffer_size: 33554432 2023-10-16T14:24:14.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_write_buffer_number: 2 2023-10-16T14:24:14.836 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression: NoCompression 2023-10-16T14:24:14.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression: Disabled 2023-10-16T14:24:14.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.prefix_extractor: nullptr 2023-10-16T14:24:14.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-16T14:24:14.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.num_levels: 7 2023-10-16T14:24:14.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-16T14:24:14.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-16T14:24:14.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-16T14:24:14.838 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-16T14:24:14.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-16T14:24:14.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-16T14:24:14.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-16T14:24:14.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-16T14:24:14.839 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-16T14:24:14.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-16T14:24:14.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-16T14:24:14.840 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-16T14:24:14.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-16T14:24:14.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.level: 32767 2023-10-16T14:24:14.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.strategy: 0 2023-10-16T14:24:14.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-16T14:24:14.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-16T14:24:14.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-16T14:24:14.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-16T14:24:14.842 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.enabled: false 2023-10-16T14:24:14.843 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-16T14:24:14.843 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-16T14:24:14.843 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-16T14:24:14.843 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-16T14:24:14.844 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.target_file_size_base: 67108864 2023-10-16T14:24:14.844 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-16T14:24:14.844 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-16T14:24:14.844 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-16T14:24:14.845 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-16T14:24:14.845 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-16T14:24:14.845 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-16T14:24:14.846 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-16T14:24:14.846 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-16T14:24:14.846 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-16T14:24:14.846 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-16T14:24:14.847 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-16T14:24:14.847 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-16T14:24:14.847 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-16T14:24:14.847 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-16T14:24:14.847 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.arena_block_size: 1048576 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.disable_auto_compactions: 0 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-16T14:24:14.848 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-16T14:24:14.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-16T14:24:14.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-16T14:24:14.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-16T14:24:14.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-16T14:24:14.849 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-16T14:24:14.850 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-16T14:24:14.850 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-16T14:24:14.850 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.table_properties_collectors: 2023-10-16T14:24:14.850 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.inplace_update_support: 0 2023-10-16T14:24:14.850 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-16T14:24:14.851 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-16T14:24:14.851 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-16T14:24:14.851 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-16T14:24:14.851 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.bloom_locality: 0 2023-10-16T14:24:14.851 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.max_successive_merges: 0 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.paranoid_file_checks: 0 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.force_consistency_checks: 1 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.report_bg_io_stats: 0 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.ttl: 2592000 2023-10-16T14:24:14.852 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-16T14:24:14.853 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-16T14:24:14.853 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-16T14:24:14.853 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enable_blob_files: false 2023-10-16T14:24:14.853 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.min_blob_size: 0 2023-10-16T14:24:14.853 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_file_size: 268435456 2023-10-16T14:24:14.854 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-16T14:24:14.854 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-16T14:24:14.854 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-16T14:24:14.854 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-16T14:24:14.854 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-16T14:24:14.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.blob_file_starting_level: 0 2023-10-16T14:24:14.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-16T14:24:14.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-10-16T14:24:14.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-10-16T14:24:14.855 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2b26c27a-9003-4b92-a2cc-885c13d44ebf 2023-10-16T14:24:14.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466254562909, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-10-16T14:24:14.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-10-16T14:24:14.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466254564624, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 319, "table_properties": {"data_size": 91402, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13414, "raw_average_key_size": 49, "raw_value_size": 84098, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697466254, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2b26c27a-9003-4b92-a2cc-885c13d44ebf", "db_session_id": "7UG2XLRNN2HJPTIKVXQK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-10-16T14:24:14.856 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466254564759, "job": 1, "event": "recovery_finished"} 2023-10-16T14:24:14.857 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-10-16T14:24:14.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-16T14:24:14.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d89bc96000 2023-10-16T14:24:14.858 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: DB pointer 0x55d89bc82000 2023-10-16T14:24:14.859 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-16T14:24:14.859 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-16T14:24:14.859 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** DB Stats ** 2023-10-16T14:24:14.859 INFO:journalctl@ceph.mon.a.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-16T14:24:14.859 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-16T14:24:14.860 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.861 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** Compaction Stats [default] ** 2023-10-16T14:24:14.861 INFO:journalctl@ceph.mon.a.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-16T14:24:14.861 INFO:journalctl@ceph.mon.a.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-16T14:24:14.861 INFO:journalctl@ceph.mon.a.smithi178.stdout: L0 2/0 93.03 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-16T14:24:14.861 INFO:journalctl@ceph.mon.a.smithi178.stdout: Sum 2/0 93.03 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-16T14:24:14.862 INFO:journalctl@ceph.mon.a.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-16T14:24:14.862 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.862 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** Compaction Stats [default] ** 2023-10-16T14:24:14.862 INFO:journalctl@ceph.mon.a.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-16T14:24:14.863 INFO:journalctl@ceph.mon.a.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-16T14:24:14.863 INFO:journalctl@ceph.mon.a.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 81.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-10-16T14:24:14.863 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.863 INFO:journalctl@ceph.mon.a.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-16T14:24:14.863 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-16T14:24:14.864 INFO:journalctl@ceph.mon.a.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-16T14:24:14.865 INFO:journalctl@ceph.mon.a.smithi178.stdout: Cumulative compaction: 0.00 GB write, 14.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-16T14:24:14.865 INFO:journalctl@ceph.mon.a.smithi178.stdout: Interval compaction: 0.00 GB write, 14.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-16T14:24:14.865 INFO:journalctl@ceph.mon.a.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-16T14:24:14.865 INFO:journalctl@ceph.mon.a.smithi178.stdout: Block cache BinnedLRUCache@0x55d89bbf91f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-10-16T14:24:14.865 INFO:journalctl@ceph.mon.a.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-10-16T14:24:14.866 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.866 INFO:journalctl@ceph.mon.a.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-16T14:24:14.866 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: starting mon.a rank 0 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:14.866 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 systemd[1]: Started Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:24:14.866 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???) e1 preinit fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:14.867 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).mds e1 new map 2023-10-16T14:24:14.867 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).mds e1 print_map 2023-10-16T14:24:14.867 INFO:journalctl@ceph.mon.a.smithi178.stdout: e1 2023-10-16T14:24:14.867 INFO:journalctl@ceph.mon.a.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-16T14:24:14.867 INFO:journalctl@ceph.mon.a.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-16T14:24:14.868 INFO:journalctl@ceph.mon.a.smithi178.stdout: legacy client fscid: -1 2023-10-16T14:24:14.868 INFO:journalctl@ceph.mon.a.smithi178.stdout: 2023-10-16T14:24:14.868 INFO:journalctl@ceph.mon.a.smithi178.stdout: No filesystems configured 2023-10-16T14:24:14.868 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-16T14:24:14.868 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:24:14.869 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:24:14.869 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:24:14.869 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-10-16T14:24:14.869 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expand map: {default=false} 2023-10-16T14:24:14.869 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta from 'false' to 'false' 2023-10-16T14:24:14.870 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expanded map: {default=false} 2023-10-16T14:24:14.870 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expand map: {default=info} 2023-10-16T14:24:14.870 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta from 'info' to 'info' 2023-10-16T14:24:14.870 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expanded map: {default=info} 2023-10-16T14:24:14.870 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expand map: {default=daemon} 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expanded map: {default=daemon} 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expand map: {default=debug} 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta from 'debug' to 'debug' 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: expand_channel_meta expanded map: {default=debug} 2023-10-16T14:24:14.871 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mon.a is new leader, mons a in quorum (ranks 0) 2023-10-16T14:24:14.872 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: monmap e1: 1 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} 2023-10-16T14:24:14.872 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: fsmap 2023-10-16T14:24:14.872 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: osdmap e1: 0 total, 0 up, 0 in 2023-10-16T14:24:14.872 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:14 smithi178 ceph-mon[98495]: mgrmap e1: no daemons active 2023-10-16T14:24:15.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.137+0000 7feb09449700 1 Processor -- start 2023-10-16T14:24:15.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.137+0000 7feb09449700 1 -- start start 2023-10-16T14:24:15.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.138+0000 7feb09449700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:15.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.138+0000 7feb09449700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0407a8c0 con 0x7feb0407be90 2023-10-16T14:24:15.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.138+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:15.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.138+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45148/0 (socket says 172.21.15.178:45148) 2023-10-16T14:24:15.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.138+0000 7feb02ffd700 1 -- 172.21.15.178:0/1275486943 learned_addr learned my addr 172.21.15.178:0/1275486943 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:15.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.139+0000 7feb02ffd700 1 -- 172.21.15.178:0/1275486943 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0407aa00 con 0x7feb0407be90 2023-10-16T14:24:15.766 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.139+0000 7feb02ffd700 1 --2- 172.21.15.178:0/1275486943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098a0 tx=0x7feaf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8825e5bec37db5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:15.766 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.139+0000 7feb01ffb700 1 -- 172.21.15.178:0/1275486943 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4004160 con 0x7feb0407be90 2023-10-16T14:24:15.766 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.139+0000 7feb01ffb700 1 -- 172.21.15.178:0/1275486943 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feaf40042c0 con 0x7feb0407be90 2023-10-16T14:24:15.766 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.139+0000 7feb01ffb700 1 -- 172.21.15.178:0/1275486943 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4011a00 con 0x7feb0407be90 2023-10-16T14:24:15.766 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 -- 172.21.15.178:0/1275486943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 msgr2=0x7feb0407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 --2- 172.21.15.178:0/1275486943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7feaf40098a0 tx=0x7feaf4009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 -- 172.21.15.178:0/1275486943 shutdown_connections 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 --2- 172.21.15.178:0/1275486943 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0407a2f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 -- 172.21.15.178:0/1275486943 >> 172.21.15.178:0/1275486943 conn(0x7feb04101b70 msgr2=0x7feb04103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 -- 172.21.15.178:0/1275486943 shutdown_connections 2023-10-16T14:24:15.767 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.140+0000 7feb09449700 1 -- 172.21.15.178:0/1275486943 wait complete. 2023-10-16T14:24:15.768 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.141+0000 7feb09449700 1 Processor -- start 2023-10-16T14:24:15.768 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.141+0000 7feb09449700 1 -- start start 2023-10-16T14:24:15.768 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.141+0000 7feb09449700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:15.768 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.141+0000 7feb09449700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0419c980 con 0x7feb0407be90 2023-10-16T14:24:15.768 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.142+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:15.769 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.142+0000 7feb02ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45164/0 (socket says 172.21.15.178:45164) 2023-10-16T14:24:15.769 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.142+0000 7feb02ffd700 1 -- 172.21.15.178:0/2120513505 learned_addr learned my addr 172.21.15.178:0/2120513505 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:15.769 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.142+0000 7feb02ffd700 1 -- 172.21.15.178:0/2120513505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4009550 con 0x7feb0407be90 2023-10-16T14:24:15.769 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.142+0000 7feb02ffd700 1 --2- 172.21.15.178:0/2120513505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006af0 tx=0x7feaf4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:15.769 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf4003a40 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feaf4003ba0 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0419cb80 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feaf401a6e0 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0419cfa0 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feaf401ac50 con 0x7feb0407be90 2023-10-16T14:24:15.770 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.143+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feaf4019700 con 0x7feb0407be90 2023-10-16T14:24:15.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.144+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0410c980 con 0x7feb0407be90 2023-10-16T14:24:15.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.147+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7feaf402de30 con 0x7feb0407be90 2023-10-16T14:24:15.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.216+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7feb0419fc80 con 0x7feb0407be90 2023-10-16T14:24:15.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.218+0000 7feaebfff700 1 -- 172.21.15.178:0/2120513505 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7feaf4024a60 con 0x7feb0407be90 2023-10-16T14:24:15.771 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 msgr2=0x7feb0419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 --2- 172.21.15.178:0/2120513505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7feaf4006af0 tx=0x7feaf4011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 shutdown_connections 2023-10-16T14:24:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 --2- 172.21.15.178:0/2120513505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb0407be90 0x7feb0419c440 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 >> 172.21.15.178:0/2120513505 conn(0x7feb04101b70 msgr2=0x7feb04103260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:15.772 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 shutdown_connections 2023-10-16T14:24:15.773 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:15.219+0000 7feb09449700 1 -- 172.21.15.178:0/2120513505 wait complete. 2023-10-16T14:24:15.773 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote config to /etc/ceph/ceph.conf 2023-10-16T14:24:15.803 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:24:15.803 INFO:teuthology.orchestra.run.smithi178.stdout:Creating mgr... 2023-10-16T14:24:15.804 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying port 0.0.0.0:9283 ... 2023-10-16T14:24:15.804 INFO:teuthology.orchestra.run.smithi178.stdout:Verifying port 0.0.0.0:8765 ... 2023-10-16T14:24:16.164 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x 2023-10-16T14:24:16.165 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x.service: Unit ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x.service not loaded. 2023-10-16T14:24:16.373 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7.target.wants/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x.service → /etc/systemd/system/ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@.service. 2023-10-16T14:24:16.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:16 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2120513505' entity='client.admin' 2023-10-16T14:24:16.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:16 smithi178 systemd[1]: Starting Ceph mgr.x for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:24:17.167 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:16 smithi178 podman[98729]: 2023-10-16T14:24:17.167 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 bash[98729]: 8ff480ef2ed51c46823e9c8dd65f4ed215220063b73d351ba3eadfb7bc0b5d16 2023-10-16T14:24:17.190 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-16T14:24:17.191 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2023-10-16T14:24:17.207 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-16T14:24:17.207 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:24:17.207 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2023-10-16T14:24:17.207 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to enable service . firewalld.service is not available 2023-10-16T14:24:17.225 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-16T14:24:17.225 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2023-10-16T14:24:17.242 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-16T14:24:17.242 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:24:17.242 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2023-10-16T14:24:17.242 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-10-16T14:24:17.243 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr to start... 2023-10-16T14:24:17.243 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr... 2023-10-16T14:24:17.513 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 systemd[1]: Started Ceph mgr.x for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:24:17.513 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:17.280+0000 7f18cfe14200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T14:24:17.513 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:17.390+0000 7f18cfe14200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T14:24:17.514 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:17.510+0000 7f18cfe14200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T14:24:17.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:17.640+0000 7f18cfe14200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T14:24:18.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1245814590' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-16T14:24:18.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:17 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:17.887+0000 7f18cfe14200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T14:24:18.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:18.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:24:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:24:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "health": { 2023-10-16T14:24:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-16T14:24:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-16T14:24:18.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-16T14:24:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-16T14:24:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-16T14:24:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 0 2023-10-16T14:24:18.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:18.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-16T14:24:18.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "a" 2023-10-16T14:24:18.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:18.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-10-16T14:24:18.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-16T14:24:18.433 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:18.433 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-16T14:24:18.433 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-16T14:24:18.433 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.434 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-16T14:24:18.434 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:18.434 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-16T14:24:18.434 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-16T14:24:18.434 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-16T14:24:18.441 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-16T14:24:18.442 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-16T14:24:18.442 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-16T14:24:18.442 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.442 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-16T14:24:18.442 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-16T14:24:18.443 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-16T14:24:18.443 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-16T14:24:18.443 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-16T14:24:18.443 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-16T14:24:18.444 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-16T14:24:18.444 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-16T14:24:18.444 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-16T14:24:18.444 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.444 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-16T14:24:18.445 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": false, 2023-10-16T14:24:18.446 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-16T14:24:18.446 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-16T14:24:18.446 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-16T14:24:18.446 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-16T14:24:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "restful" 2023-10-16T14:24:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-16T14:24:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.447 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modified": "2023-10-16T14:24:07.673545+0000", 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-16T14:24:18.448 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:24:18.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.814+0000 7f5c4f949700 1 Processor -- start 2023-10-16T14:24:18.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.815+0000 7f5c4f949700 1 -- start start 2023-10-16T14:24:18.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.815+0000 7f5c4f949700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:18.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.815+0000 7f5c4f949700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4807a8c0 con 0x7f5c4807be90 2023-10-16T14:24:18.449 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.815+0000 7f5c4d6e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:18.450 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c4d6e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45208/0 (socket says 172.21.15.178:45208) 2023-10-16T14:24:18.450 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c4d6e5700 1 -- 172.21.15.178:0/2110612945 learned_addr learned my addr 172.21.15.178:0/2110612945 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:18.450 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c4d6e5700 1 -- 172.21.15.178:0/2110612945 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4807aa00 con 0x7f5c4807be90 2023-10-16T14:24:18.451 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c4d6e5700 1 --2- 172.21.15.178:0/2110612945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93f83f98ce79dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:18.451 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c3ffff700 1 -- 172.21.15.178:0/2110612945 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44004160 con 0x7f5c4807be90 2023-10-16T14:24:18.451 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c3ffff700 1 -- 172.21.15.178:0/2110612945 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c440042c0 con 0x7f5c4807be90 2023-10-16T14:24:18.451 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.816+0000 7f5c3ffff700 1 -- 172.21.15.178:0/2110612945 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44011a00 con 0x7f5c4807be90 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 -- 172.21.15.178:0/2110612945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 msgr2=0x7f5c4807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 --2- 172.21.15.178:0/2110612945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c440098a0 tx=0x7f5c44009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 -- 172.21.15.178:0/2110612945 shutdown_connections 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 --2- 172.21.15.178:0/2110612945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4807a2f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 -- 172.21.15.178:0/2110612945 >> 172.21.15.178:0/2110612945 conn(0x7f5c48101b70 msgr2=0x7f5c48103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:18.452 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 -- 172.21.15.178:0/2110612945 shutdown_connections 2023-10-16T14:24:18.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.817+0000 7f5c4f949700 1 -- 172.21.15.178:0/2110612945 wait complete. 2023-10-16T14:24:18.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.818+0000 7f5c4f949700 1 Processor -- start 2023-10-16T14:24:18.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.818+0000 7f5c4f949700 1 -- start start 2023-10-16T14:24:18.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4f949700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:18.453 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4f949700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4819c970 con 0x7f5c4807be90 2023-10-16T14:24:18.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4d6e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:18.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4d6e5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45210/0 (socket says 172.21.15.178:45210) 2023-10-16T14:24:18.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4d6e5700 1 -- 172.21.15.178:0/1245814590 learned_addr learned my addr 172.21.15.178:0/1245814590 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:18.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.819+0000 7f5c4d6e5700 1 -- 172.21.15.178:0/1245814590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44009550 con 0x7f5c4807be90 2023-10-16T14:24:18.454 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c4d6e5700 1 --2- 172.21.15.178:0/1245814590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006af0 tx=0x7f5c44011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:18.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c44003a40 con 0x7f5c4807be90 2023-10-16T14:24:18.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c44003ba0 con 0x7f5c4807be90 2023-10-16T14:24:18.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4819cb70 con 0x7f5c4807be90 2023-10-16T14:24:18.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c4401a6e0 con 0x7f5c4807be90 2023-10-16T14:24:18.455 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4819cf90 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c4401ab40 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.820+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c44019700 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.821+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c480624e0 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.824+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5c4402de20 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.913+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5c4810cae0 con 0x7f5c4807be90 2023-10-16T14:24:18.456 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.914+0000 7f5c3e7fc700 1 -- 172.21.15.178:0/1245814590 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f5c44024a60 con 0x7f5c4807be90 2023-10-16T14:24:18.457 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 msgr2=0x7f5c4819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:18.457 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 --2- 172.21.15.178:0/1245814590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5c44006af0 tx=0x7f5c44011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:18.457 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 shutdown_connections 2023-10-16T14:24:18.457 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 --2- 172.21.15.178:0/1245814590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c4807be90 0x7f5c4819c430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:18.457 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 >> 172.21.15.178:0/1245814590 conn(0x7f5c48101b70 msgr2=0x7f5c481031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:18.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 shutdown_connections 2023-10-16T14:24:18.458 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:17.915+0000 7f5c4f949700 1 -- 172.21.15.178:0/1245814590 wait complete. 2023-10-16T14:24:18.458 INFO:teuthology.orchestra.run.smithi178.stdout:mgr not available, waiting (1/15)... 2023-10-16T14:24:18.674 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:18 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:18.382+0000 7f18cfe14200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T14:24:18.674 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:18 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:18.554+0000 7f18cfe14200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T14:24:19.053 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:18 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:18.671+0000 7f18cfe14200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T14:24:19.053 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:18 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:18.794+0000 7f18cfe14200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T14:24:19.802 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:19 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:19.343+0000 7f18cfe14200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T14:24:20.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:20 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:20.227+0000 7f18cfe14200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T14:24:20.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:20 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:20.332+0000 7f18cfe14200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T14:24:21.671 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:21.316+0000 7f18cfe14200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:24:21.672 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:21.438+0000 7f18cfe14200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T14:24:22.038 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:21.668+0000 7f18cfe14200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:24:22.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:22 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:22.035+0000 7f18cfe14200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T14:24:22.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:22 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:22.322+0000 7f18cfe14200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T14:24:22.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:22 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:22.513+0000 7f18cfe14200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T14:24:23.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:23 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:22.862+0000 7f18cfe14200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T14:24:24.230 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:23 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:23.802+0000 7f18cfe14200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T14:24:24.231 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:23 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:23.976+0000 7f18cfe14200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T14:24:24.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:24 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:24.228+0000 7f18cfe14200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T14:24:24.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:24 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:24.372+0000 7f18cfe14200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T14:24:25.052 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:24 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:24.653+0000 7f18cfe14200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T14:24:26.053 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:25.755+0000 7f18cfe14200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T14:24:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: Activating manager daemon x 2023-10-16T14:24:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: mgrmap e2: x(active, starting, since 0.00425053s) 2023-10-16T14:24:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-16T14:24:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-16T14:24:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-16T14:24:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:24:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-16T14:24:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: Manager daemon x is now available 2023-10-16T14:24:26.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-16T14:24:26.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' 2023-10-16T14:24:26.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-16T14:24:26.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' 2023-10-16T14:24:26.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:25 smithi178 ceph-mon[98495]: from='mgr.14100 172.21.15.178:0/3139614412' entity='mgr.x' 2023-10-16T14:24:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:27 smithi178 ceph-mon[98495]: mgrmap e3: x(active, since 1.00917s) 2023-10-16T14:24:28.735 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:28.736 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:24:28.736 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:24:28.736 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "health": { 2023-10-16T14:24:28.736 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-10-16T14:24:28.736 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-10-16T14:24:28.737 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-10-16T14:24:28.737 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.737 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-10-16T14:24:28.737 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-10-16T14:24:28.737 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 0 2023-10-16T14:24:28.738 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:28.738 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-10-16T14:24:28.738 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "a" 2023-10-16T14:24:28.738 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:28.738 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-10-16T14:24:28.739 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "monmap": { 2023-10-16T14:24:28.739 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:28.739 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-10-16T14:24:28.739 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-10-16T14:24:28.739 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.740 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-10-16T14:24:28.740 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:28.740 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-10-16T14:24:28.740 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-10-16T14:24:28.740 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-10-16T14:24:28.748 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-10-16T14:24:28.748 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-10-16T14:24:28.748 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-10-16T14:24:28.748 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.748 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-10-16T14:24:28.749 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-10-16T14:24:28.749 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-10-16T14:24:28.749 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-10-16T14:24:28.749 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-10-16T14:24:28.749 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-10-16T14:24:28.750 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-10-16T14:24:28.750 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-10-16T14:24:28.750 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-10-16T14:24:28.750 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.750 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-10-16T14:24:28.751 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:28.751 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-10-16T14:24:28.751 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-10-16T14:24:28.751 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.751 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-10-16T14:24:28.752 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-16T14:24:28.752 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-10-16T14:24:28.752 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modules": [ 2023-10-16T14:24:28.752 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "iostat", 2023-10-16T14:24:28.752 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "nfs", 2023-10-16T14:24:28.753 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "restful" 2023-10-16T14:24:28.753 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ], 2023-10-16T14:24:28.753 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-16T14:24:28.753 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.753 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-10-16T14:24:28.754 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-10-16T14:24:28.754 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "modified": "2023-10-16T14:24:07.673545+0000", 2023-10-16T14:24:28.754 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "services": {} 2023-10-16T14:24:28.754 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout }, 2023-10-16T14:24:28.754 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.951+0000 7f0a2c014700 1 Processor -- start 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.951+0000 7f0a2c014700 1 -- start start 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a2c014700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a2c014700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a24106fa0 con 0x7f0a241065b0 2023-10-16T14:24:28.755 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a29db0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a29db0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36282/0 (socket says 172.21.15.178:36282) 2023-10-16T14:24:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a29db0700 1 -- 172.21.15.178:0/1445187584 learned_addr learned my addr 172.21.15.178:0/1445187584 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a29db0700 1 -- 172.21.15.178:0/1445187584 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a241077b0 con 0x7f0a241065b0 2023-10-16T14:24:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.952+0000 7f0a29db0700 1 --2- 172.21.15.178:0/1445187584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a200098a0 tx=0x7f0a20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b351fd5ee80f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:28.756 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a28dae700 1 -- 172.21.15.178:0/1445187584 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a20004160 con 0x7f0a241065b0 2023-10-16T14:24:28.757 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a28dae700 1 -- 172.21.15.178:0/1445187584 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a200042c0 con 0x7f0a241065b0 2023-10-16T14:24:28.757 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a28dae700 1 -- 172.21.15.178:0/1445187584 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a20011a00 con 0x7f0a241065b0 2023-10-16T14:24:28.757 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 -- 172.21.15.178:0/1445187584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 msgr2=0x7f0a241069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:28.757 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1445187584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0a200098a0 tx=0x7f0a20009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:28.757 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 -- 172.21.15.178:0/1445187584 shutdown_connections 2023-10-16T14:24:28.758 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1445187584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a241069d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:28.758 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 -- 172.21.15.178:0/1445187584 >> 172.21.15.178:0/1445187584 conn(0x7f0a24101b70 msgr2=0x7f0a24103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:28.758 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 -- 172.21.15.178:0/1445187584 shutdown_connections 2023-10-16T14:24:28.758 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.953+0000 7f0a2c014700 1 -- 172.21.15.178:0/1445187584 wait complete. 2023-10-16T14:24:28.758 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.954+0000 7f0a2c014700 1 Processor -- start 2023-10-16T14:24:28.759 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.954+0000 7f0a2c014700 1 -- start start 2023-10-16T14:24:28.759 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a2c014700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:28.759 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a2c014700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2419ca30 con 0x7f0a241065b0 2023-10-16T14:24:28.759 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a29db0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:28.759 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a29db0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36288/0 (socket says 172.21.15.178:36288) 2023-10-16T14:24:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a29db0700 1 -- 172.21.15.178:0/1040516869 learned_addr learned my addr 172.21.15.178:0/1040516869 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a29db0700 1 -- 172.21.15.178:0/1040516869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20009550 con 0x7f0a241065b0 2023-10-16T14:24:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.955+0000 7f0a29db0700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009130 tx=0x7f0a20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.956+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a20003ab0 con 0x7f0a241065b0 2023-10-16T14:24:28.760 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.956+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0a20003c10 con 0x7f0a241065b0 2023-10-16T14:24:28.761 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.956+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a200247d0 con 0x7f0a241065b0 2023-10-16T14:24:28.761 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.956+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2419cc30 con 0x7f0a241065b0 2023-10-16T14:24:28.761 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.956+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2419d050 con 0x7f0a241065b0 2023-10-16T14:24:28.761 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.957+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f0a2001abb0 con 0x7f0a241065b0 2023-10-16T14:24:28.761 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.957+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240624e0 con 0x7f0a241065b0 2023-10-16T14:24:28.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.957+0000 7f0a1affd700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 0x7f0a1003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:28.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.957+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0a2004c540 con 0x7f0a241065b0 2023-10-16T14:24:28.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.958+0000 7f0a295af700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 0x7f0a1003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:28.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.958+0000 7f0a295af700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 0x7f0a1003ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a14006fd0 tx=0x7f0a14006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:28.762 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:27.963+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a20018710 con 0x7f0a241065b0 2023-10-16T14:24:28.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.238+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0a2410a9e0 con 0x7f0a241065b0 2023-10-16T14:24:28.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.239+0000 7f0a1affd700 1 -- 172.21.15.178:0/1040516869 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0a20018350 con 0x7f0a241065b0 2023-10-16T14:24:28.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 msgr2=0x7f0a1003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:28.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 0x7f0a1003ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0a14006fd0 tx=0x7f0a14006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:28.763 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 msgr2=0x7f0a2419c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:28.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009130 tx=0x7f0a20011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:28.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 shutdown_connections 2023-10-16T14:24:28.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f0a10038820 0x7f0a1003ace0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:28.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 --2- 172.21.15.178:0/1040516869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a241065b0 0x7f0a2419c4f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:28.764 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 >> 172.21.15.178:0/1040516869 conn(0x7f0a24101b70 msgr2=0x7f0a24103270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:28.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 shutdown_connections 2023-10-16T14:24:28.765 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:28.241+0000 7f0a2c014700 1 -- 172.21.15.178:0/1040516869 wait complete. 2023-10-16T14:24:28.765 INFO:teuthology.orchestra.run.smithi178.stdout:mgr is available 2023-10-16T14:24:29.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:28 smithi178 ceph-mon[98495]: mgrmap e4: x(active, since 2s) 2023-10-16T14:24:29.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:28 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1040516869' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-10-16T14:24:29.921 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:29.921 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [global] 2023-10-16T14:24:29.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout fsid = 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:29.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-10-16T14:24:29.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-10-16T14:24:29.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-10-16T14:24:29.922 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-10-16T14:24:29.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:29.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [mgr] 2023-10-16T14:24:29.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-10-16T14:24:29.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 2023-10-16T14:24:29.923 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout [osd] 2023-10-16T14:24:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-10-16T14:24:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-10-16T14:24:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-10-16T14:24:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-10-16T14:24:29.924 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-10-16T14:24:29.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.264+0000 7fb7a6452700 1 Processor -- start 2023-10-16T14:24:29.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb7a6452700 1 -- start start 2023-10-16T14:24:29.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb7a6452700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:29.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb7a6452700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a007a8a0 con 0x7fb7a007be70 2023-10-16T14:24:29.925 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb79ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:29.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb79ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47434/0 (socket says 172.21.15.178:47434) 2023-10-16T14:24:29.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.265+0000 7fb79ffff700 1 -- 172.21.15.178:0/3912279901 learned_addr learned my addr 172.21.15.178:0/3912279901 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:29.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.266+0000 7fb79ffff700 1 -- 172.21.15.178:0/3912279901 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a007a9e0 con 0x7fb7a007be70 2023-10-16T14:24:29.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.266+0000 7fb79ffff700 1 --2- 172.21.15.178:0/3912279901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7900098a0 tx=0x7fb790009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a7f1c37d45d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:29.926 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.266+0000 7fb79effd700 1 -- 172.21.15.178:0/3912279901 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb790004160 con 0x7fb7a007be70 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.266+0000 7fb79effd700 1 -- 172.21.15.178:0/3912279901 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb7900042c0 con 0x7fb7a007be70 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.266+0000 7fb79effd700 1 -- 172.21.15.178:0/3912279901 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb790011a00 con 0x7fb7a007be70 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 -- 172.21.15.178:0/3912279901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 msgr2=0x7fb7a007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 --2- 172.21.15.178:0/3912279901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb7900098a0 tx=0x7fb790009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 -- 172.21.15.178:0/3912279901 shutdown_connections 2023-10-16T14:24:29.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 --2- 172.21.15.178:0/3912279901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a007a2d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:29.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 -- 172.21.15.178:0/3912279901 >> 172.21.15.178:0/3912279901 conn(0x7fb7a0101b70 msgr2=0x7fb7a0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:29.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 -- 172.21.15.178:0/3912279901 shutdown_connections 2023-10-16T14:24:29.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.267+0000 7fb7a6452700 1 -- 172.21.15.178:0/3912279901 wait complete. 2023-10-16T14:24:29.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.268+0000 7fb7a6452700 1 Processor -- start 2023-10-16T14:24:29.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.268+0000 7fb7a6452700 1 -- start start 2023-10-16T14:24:29.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.268+0000 7fb7a6452700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:29.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.268+0000 7fb7a6452700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a019c9a0 con 0x7fb7a007be70 2023-10-16T14:24:29.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:29.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47438/0 (socket says 172.21.15.178:47438) 2023-10-16T14:24:29.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79ffff700 1 -- 172.21.15.178:0/48474178 learned_addr learned my addr 172.21.15.178:0/48474178 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79ffff700 1 -- 172.21.15.178:0/48474178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009550 con 0x7fb7a007be70 2023-10-16T14:24:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79ffff700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb790006af0 tx=0x7fb790011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb790003a40 con 0x7fb7a007be70 2023-10-16T14:24:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb790003ba0 con 0x7fb7a007be70 2023-10-16T14:24:29.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb79001a6e0 con 0x7fb7a007be70 2023-10-16T14:24:29.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.269+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a019cba0 con 0x7fb7a007be70 2023-10-16T14:24:29.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.270+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a019cfc0 con 0x7fb7a007be70 2023-10-16T14:24:29.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.271+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7fb790024a60 con 0x7fb7a007be70 2023-10-16T14:24:29.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.271+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a00624e0 con 0x7fb7a007be70 2023-10-16T14:24:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.271+0000 7fb79d7fa700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 0x7fb78803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.271+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb79004c830 con 0x7fb7a007be70 2023-10-16T14:24:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.271+0000 7fb79f7fe700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 0x7fb78803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.272+0000 7fb79f7fe700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 0x7fb78803ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb794006fd0 tx=0x7fb794006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:29.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.276+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb790024430 con 0x7fb7a007be70 2023-10-16T14:24:29.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.474+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb7a010c940 con 0x7fb7a007be70 2023-10-16T14:24:29.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.475+0000 7fb79d7fa700 1 -- 172.21.15.178:0/48474178 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fb790018350 con 0x7fb7a007be70 2023-10-16T14:24:29.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.477+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 msgr2=0x7fb78803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:29.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.477+0000 7fb7a6452700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 0x7fb78803ac90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb794006fd0 tx=0x7fb794006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:29.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.477+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 msgr2=0x7fb7a019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:29.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.477+0000 7fb7a6452700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb790006af0 tx=0x7fb790011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:29.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 shutdown_connections 2023-10-16T14:24:29.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7fb7880387d0 0x7fb78803ac90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:29.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 --2- 172.21.15.178:0/48474178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a007be70 0x7fb7a019c460 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:29.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 >> 172.21.15.178:0/48474178 conn(0x7fb7a0101b70 msgr2=0x7fb7a0103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:29.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 shutdown_connections 2023-10-16T14:24:29.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:29.478+0000 7fb7a6452700 1 -- 172.21.15.178:0/48474178 wait complete. 2023-10-16T14:24:29.935 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling cephadm module... 2023-10-16T14:24:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:29 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/48474178' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-10-16T14:24:31.048 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:30 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/931002452' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-10-16T14:24:31.048 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:30 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: ignoring --setuser ceph since I am not root 2023-10-16T14:24:31.049 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:30 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: ignoring --setgroup ceph since I am not root 2023-10-16T14:24:31.049 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:31.044+0000 7efec8605200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T14:24:31.201 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.439+0000 7f8e543b7700 1 Processor -- start 2023-10-16T14:24:31.201 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.439+0000 7f8e543b7700 1 -- start start 2023-10-16T14:24:31.201 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.439+0000 7f8e543b7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:31.202 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.439+0000 7f8e543b7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c07a900 con 0x7f8e4c07bed0 2023-10-16T14:24:31.202 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.440+0000 7f8e52153700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:31.202 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.440+0000 7f8e52153700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47452/0 (socket says 172.21.15.178:47452) 2023-10-16T14:24:31.202 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.440+0000 7f8e52153700 1 -- 172.21.15.178:0/265048663 learned_addr learned my addr 172.21.15.178:0/265048663 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:31.202 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.440+0000 7f8e52153700 1 -- 172.21.15.178:0/265048663 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c07aa40 con 0x7f8e4c07bed0 2023-10-16T14:24:31.203 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.440+0000 7f8e52153700 1 --2- 172.21.15.178:0/265048663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e480098a0 tx=0x7f8e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10ac1642e15dc4a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:31.207 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e51151700 1 -- 172.21.15.178:0/265048663 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e48004160 con 0x7f8e4c07bed0 2023-10-16T14:24:31.207 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e51151700 1 -- 172.21.15.178:0/265048663 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e480042c0 con 0x7f8e4c07bed0 2023-10-16T14:24:31.207 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e51151700 1 -- 172.21.15.178:0/265048663 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e48011a00 con 0x7f8e4c07bed0 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 -- 172.21.15.178:0/265048663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 msgr2=0x7f8e4c07a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 --2- 172.21.15.178:0/265048663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8e480098a0 tx=0x7f8e48009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 -- 172.21.15.178:0/265048663 shutdown_connections 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 --2- 172.21.15.178:0/265048663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c07a330 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 -- 172.21.15.178:0/265048663 >> 172.21.15.178:0/265048663 conn(0x7f8e4c101b70 msgr2=0x7f8e4c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:31.208 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 -- 172.21.15.178:0/265048663 shutdown_connections 2023-10-16T14:24:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.441+0000 7f8e543b7700 1 -- 172.21.15.178:0/265048663 wait complete. 2023-10-16T14:24:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.442+0000 7f8e543b7700 1 Processor -- start 2023-10-16T14:24:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.442+0000 7f8e543b7700 1 -- start start 2023-10-16T14:24:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e543b7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:31.209 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e543b7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19c9b0 con 0x7f8e4c07bed0 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e52153700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e52153700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47462/0 (socket says 172.21.15.178:47462) 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e52153700 1 -- 172.21.15.178:0/931002452 learned_addr learned my addr 172.21.15.178:0/931002452 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.443+0000 7f8e52153700 1 -- 172.21.15.178:0/931002452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e48009550 con 0x7f8e4c07bed0 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e52153700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006af0 tx=0x7f8e48011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:31.210 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e48003a40 con 0x7f8e4c07bed0 2023-10-16T14:24:31.211 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8e48003ba0 con 0x7f8e4c07bed0 2023-10-16T14:24:31.211 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c19cbb0 con 0x7f8e4c07bed0 2023-10-16T14:24:31.211 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e4801a700 con 0x7f8e4c07bed0 2023-10-16T14:24:31.211 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.444+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c19cfd0 con 0x7f8e4c07bed0 2023-10-16T14:24:31.211 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.445+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 4) v1 ==== 45568+0+0 (secure 0 0 0) 0x7f8e48024430 con 0x7f8e4c07bed0 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.445+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c10c940 con 0x7f8e4c07bed0 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.445+0000 7f8e437fe700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 0x7f8e3803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.445+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8e4804c790 con 0x7f8e4c07bed0 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.446+0000 7f8e51952700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 0x7f8e3803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.446+0000 7f8e51952700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 0x7f8e3803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c006fd0 tx=0x7f8e3c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:31.212 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.451+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8e48018530 con 0x7f8e4c07bed0 2023-10-16T14:24:31.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.701+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f8e4c10cb80 con 0x7f8e4c07bed0 2023-10-16T14:24:31.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.771+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f8e48018350 con 0x7f8e4c07bed0 2023-10-16T14:24:31.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.772+0000 7f8e437fe700 1 -- 172.21.15.178:0/931002452 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f8e4802ad60 con 0x7f8e4c07bed0 2023-10-16T14:24:31.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 msgr2=0x7f8e3803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:31.213 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 0x7f8e3803ac90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c006fd0 tx=0x7f8e3c006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 msgr2=0x7f8e4c19c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e48006af0 tx=0x7f8e48011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 shutdown_connections 2023-10-16T14:24:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f8e380387d0 0x7f8e3803ac90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:31.214 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 --2- 172.21.15.178:0/931002452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e4c07bed0 0x7f8e4c19c470 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:31.215 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.774+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 >> 172.21.15.178:0/931002452 conn(0x7f8e4c101b70 msgr2=0x7f8e4c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:31.215 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.775+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 shutdown_connections 2023-10-16T14:24:31.215 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:30.775+0000 7f8e543b7700 1 -- 172.21.15.178:0/931002452 wait complete. 2023-10-16T14:24:31.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:31.149+0000 7efec8605200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T14:24:31.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:31.276+0000 7efec8605200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T14:24:31.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:31.400+0000 7efec8605200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T14:24:31.776 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:31.639+0000 7efec8605200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T14:24:32.045 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/931002452' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-10-16T14:24:32.045 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:31 smithi178 ceph-mon[98495]: mgrmap e5: x(active, since 5s) 2023-10-16T14:24:32.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:32.114+0000 7efec8605200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T14:24:32.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:32.289+0000 7efec8605200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T14:24:32.495 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:24:32.496 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-10-16T14:24:32.496 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-16T14:24:32.496 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-16T14:24:32.496 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-16T14:24:32.496 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f993ab4e700 1 Processor -- start 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f993ab4e700 1 -- start start 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f993ab4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f993ab4e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993407a8a0 con 0x7f993407be70 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f99388ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:32.497 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f99388ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47494/0 (socket says 172.21.15.178:47494) 2023-10-16T14:24:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.743+0000 7f99388ea700 1 -- 172.21.15.178:0/2092435555 learned_addr learned my addr 172.21.15.178:0/2092435555 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.744+0000 7f99388ea700 1 -- 172.21.15.178:0/2092435555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993407a9e0 con 0x7f993407be70 2023-10-16T14:24:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.744+0000 7f99388ea700 1 --2- 172.21.15.178:0/2092435555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99280098a0 tx=0x7f9928009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecc3a6088e8f4b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.744+0000 7f99337fe700 1 -- 172.21.15.178:0/2092435555 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9928004160 con 0x7f993407be70 2023-10-16T14:24:32.498 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.744+0000 7f99337fe700 1 -- 172.21.15.178:0/2092435555 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f99280042c0 con 0x7f993407be70 2023-10-16T14:24:32.499 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.744+0000 7f99337fe700 1 -- 172.21.15.178:0/2092435555 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9928011a00 con 0x7f993407be70 2023-10-16T14:24:32.499 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 -- 172.21.15.178:0/2092435555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 msgr2=0x7f993407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:32.499 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 --2- 172.21.15.178:0/2092435555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f99280098a0 tx=0x7f9928009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.499 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 -- 172.21.15.178:0/2092435555 shutdown_connections 2023-10-16T14:24:32.499 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 --2- 172.21.15.178:0/2092435555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993407a2d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 -- 172.21.15.178:0/2092435555 >> 172.21.15.178:0/2092435555 conn(0x7f9934101b70 msgr2=0x7f9934103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 -- 172.21.15.178:0/2092435555 shutdown_connections 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.745+0000 7f993ab4e700 1 -- 172.21.15.178:0/2092435555 wait complete. 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.746+0000 7f993ab4e700 1 Processor -- start 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.746+0000 7f993ab4e700 1 -- start start 2023-10-16T14:24:32.500 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f993ab4e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:32.501 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f993ab4e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993419c9a0 con 0x7f993407be70 2023-10-16T14:24:32.501 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f99388ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:32.501 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f99388ea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47504/0 (socket says 172.21.15.178:47504) 2023-10-16T14:24:32.501 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f99388ea700 1 -- 172.21.15.178:0/278407646 learned_addr learned my addr 172.21.15.178:0/278407646 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:32.501 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.747+0000 7f99388ea700 1 -- 172.21.15.178:0/278407646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009550 con 0x7f993407be70 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f99388ea700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9928006af0 tx=0x7f9928011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9928003a40 con 0x7f993407be70 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9928003ba0 con 0x7f993407be70 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f992801a6e0 con 0x7f993407be70 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993419cba0 con 0x7f993407be70 2023-10-16T14:24:32.502 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.748+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993419cfc0 con 0x7f993407be70 2023-10-16T14:24:32.503 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.749+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f992801a860 con 0x7f993407be70 2023-10-16T14:24:32.503 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.749+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99340624e0 con 0x7f993407be70 2023-10-16T14:24:32.503 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.750+0000 7f9931ffb700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 0x7f992403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:32.503 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.750+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f992804c810 con 0x7f993407be70 2023-10-16T14:24:32.503 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.750+0000 7f9933fff700 1 -- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 msgr2=0x7f992403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.750+0000 7f9933fff700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 0x7f992403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:24:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.757+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f992801ab10 con 0x7f993407be70 2023-10-16T14:24:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.951+0000 7f9933fff700 1 -- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 msgr2=0x7f992403ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:31.951+0000 7f9933fff700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 0x7f992403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T14:24:32.504 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.017+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f993410c940 con 0x7f993407be70 2023-10-16T14:24:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.017+0000 7f9931ffb700 1 -- 172.21.15.178:0/278407646 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9928029210 con 0x7f993407be70 2023-10-16T14:24:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 msgr2=0x7f992403ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:24:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 0x7f992403ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 msgr2=0x7f993419c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9928006af0 tx=0x7f9928011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.506 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 shutdown_connections 2023-10-16T14:24:32.506 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f9924038870 0x7f992403ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.506 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 --2- 172.21.15.178:0/278407646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f993407be70 0x7f993419c460 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:32.506 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 >> 172.21.15.178:0/278407646 conn(0x7f9934101b70 msgr2=0x7f9934103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:32.506 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 shutdown_connections 2023-10-16T14:24:32.507 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:32.020+0000 7f993ab4e700 1 -- 172.21.15.178:0/278407646 wait complete. 2023-10-16T14:24:32.507 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for the mgr to restart... 2023-10-16T14:24:32.507 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr epoch 5... 2023-10-16T14:24:32.759 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:32.411+0000 7efec8605200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T14:24:32.759 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:32.547+0000 7efec8605200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T14:24:33.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:32 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/278407646' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-16T14:24:33.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:33 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:33.113+0000 7efec8605200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T14:24:34.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:33 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:33.959+0000 7efec8605200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T14:24:34.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:34.064+0000 7efec8605200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T14:24:35.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:35.016+0000 7efec8605200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:24:35.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:35.135+0000 7efec8605200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T14:24:35.731 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:35.359+0000 7efec8605200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:24:36.012 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:35.728+0000 7efec8605200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T14:24:36.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:36.008+0000 7efec8605200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T14:24:36.304 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:36.199+0000 7efec8605200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T14:24:36.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:36.544+0000 7efec8605200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T14:24:37.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:37.474+0000 7efec8605200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T14:24:37.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:37.645+0000 7efec8605200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T14:24:38.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:37.893+0000 7efec8605200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T14:24:38.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:38 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:38.042+0000 7efec8605200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T14:24:38.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:38 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:38.321+0000 7efec8605200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T14:24:39.803 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:39.400+0000 7efec8605200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T14:24:39.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: Active manager daemon x restarted 2023-10-16T14:24:39.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: Activating manager daemon x 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: osdmap e2: 0 total, 0 up, 0 in 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: mgrmap e6: x(active, starting, since 0.00372463s) 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-16T14:24:39.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-16T14:24:39.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-16T14:24:39.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: Manager daemon x is now available 2023-10-16T14:24:39.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:39.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:39.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:39.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:39.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:39 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:40.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:40 smithi178 ceph-mon[98495]: Found migration_current of "None". Setting to last migration. 2023-10-16T14:24:40.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:40 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-16T14:24:40.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:40 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-16T14:24:40.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:40 smithi178 ceph-mon[98495]: mgrmap e7: x(active, since 1.0073s) 2023-10-16T14:24:40.889 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:24:40.896 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-10-16T14:24:40.896 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-16T14:24:40.897 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:24:40.897 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.051+0000 7f51e6517700 1 Processor -- start 2023-10-16T14:24:40.897 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.051+0000 7f51e6517700 1 -- start start 2023-10-16T14:24:40.897 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.052+0000 7f51e6517700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:40.897 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.052+0000 7f51e6517700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e007a8c0 con 0x7f51e007be90 2023-10-16T14:24:40.898 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.052+0000 7f51dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:40.898 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.052+0000 7f51dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47512/0 (socket says 172.21.15.178:47512) 2023-10-16T14:24:40.898 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.052+0000 7f51dffff700 1 -- 172.21.15.178:0/1097315666 learned_addr learned my addr 172.21.15.178:0/1097315666 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:40.898 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.053+0000 7f51dffff700 1 -- 172.21.15.178:0/1097315666 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e007aa00 con 0x7f51e007be90 2023-10-16T14:24:40.898 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.053+0000 7f51dffff700 1 --2- 172.21.15.178:0/1097315666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098a0 tx=0x7f51d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6a2f1faacfc399 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.053+0000 7f51deffd700 1 -- 172.21.15.178:0/1097315666 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d0004160 con 0x7f51e007be90 2023-10-16T14:24:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.053+0000 7f51deffd700 1 -- 172.21.15.178:0/1097315666 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51d00042c0 con 0x7f51e007be90 2023-10-16T14:24:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.053+0000 7f51deffd700 1 -- 172.21.15.178:0/1097315666 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d0011a00 con 0x7f51e007be90 2023-10-16T14:24:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 -- 172.21.15.178:0/1097315666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 msgr2=0x7f51e007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:40.899 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 --2- 172.21.15.178:0/1097315666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f51d00098a0 tx=0x7f51d0009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.900 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 -- 172.21.15.178:0/1097315666 shutdown_connections 2023-10-16T14:24:40.900 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 --2- 172.21.15.178:0/1097315666 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e007a2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.900 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 -- 172.21.15.178:0/1097315666 >> 172.21.15.178:0/1097315666 conn(0x7f51e0101b70 msgr2=0x7f51e0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:40.900 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 -- 172.21.15.178:0/1097315666 shutdown_connections 2023-10-16T14:24:40.900 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.054+0000 7f51e6517700 1 -- 172.21.15.178:0/1097315666 wait complete. 2023-10-16T14:24:40.901 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.055+0000 7f51e6517700 1 Processor -- start 2023-10-16T14:24:40.901 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.055+0000 7f51e6517700 1 -- start start 2023-10-16T14:24:40.901 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51e6517700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:40.901 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51e6517700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e019c970 con 0x7f51e007be90 2023-10-16T14:24:40.901 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47516/0 (socket says 172.21.15.178:47516) 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51dffff700 1 -- 172.21.15.178:0/1244597475 learned_addr learned my addr 172.21.15.178:0/1244597475 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.056+0000 7f51dffff700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0009550 con 0x7f51e007be90 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51dffff700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006af0 tx=0x7f51d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d0003a40 con 0x7f51e007be90 2023-10-16T14:24:40.902 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f51d0003ba0 con 0x7f51e007be90 2023-10-16T14:24:40.903 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e019cb70 con 0x7f51e007be90 2023-10-16T14:24:40.903 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51d001a6e0 con 0x7f51e007be90 2023-10-16T14:24:40.903 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.057+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e019cf90 con 0x7f51e007be90 2023-10-16T14:24:40.903 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.058+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f51d001a860 con 0x7f51e007be90 2023-10-16T14:24:40.903 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.058+0000 7f51dd7fa700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:40.904 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.058+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51c803b440 con 0x7f51c8038870 2023-10-16T14:24:40.904 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.058+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f51d004c940 con 0x7f51e007be90 2023-10-16T14:24:40.904 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.058+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.904 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.059+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:24:40.904 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.260+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.260+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.661+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:33.661+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:34.462+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:34.462+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T14:24:40.905 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:36.064+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.906 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:36.064+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T14:24:40.906 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:39.267+0000 7f51df7fe700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/2960529415 2023-10-16T14:24:40.906 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:39.267+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-10-16T14:24:40.906 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:39.405+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mgrmap(e 6) v1 ==== 45346+0+0 (secure 0 0 0) 0x7f51d001acf0 con 0x7f51e007be90 2023-10-16T14:24:40.906 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:39.405+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 msgr2=0x7f51c803ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:24:40.907 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:39.405+0000 7f51dd7fa700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.907 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.407+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7f51d0016d40 con 0x7f51e007be90 2023-10-16T14:24:40.907 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.408+0000 7f51dd7fa700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 0x7f51c803e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:40.907 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.408+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51c803b440 con 0x7f51c803bfd0 2023-10-16T14:24:40.907 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.408+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 0x7f51c803e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:40.908 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.409+0000 7f51df7fe700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 0x7f51c803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f51d4003a10 tx=0x7f51d40092b0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:40.908 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.410+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f51c803b440 con 0x7f51c803bfd0 2023-10-16T14:24:40.908 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.421+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f51e019d770 con 0x7f51c803bfd0 2023-10-16T14:24:40.908 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.421+0000 7f51dd7fa700 1 -- 172.21.15.178:0/1244597475 <== mgr.14116 v2:172.21.15.178:6800/3157107959 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f51e019d770 con 0x7f51c803bfd0 2023-10-16T14:24:40.908 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 msgr2=0x7f51c803e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 0x7f51c803e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f51d4003a10 tx=0x7f51d40092b0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 msgr2=0x7f51e019c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f51d0006af0 tx=0x7f51d0011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 shutdown_connections 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f51c803bfd0 0x7f51c803e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.909 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51e007be90 0x7f51e019c430 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.910 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 --2- 172.21.15.178:0/1244597475 >> [v2:172.21.15.178:6800/2960529415,v1:172.21.15.178:6801/2960529415] conn(0x7f51c8038870 0x7f51c803ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:40.910 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 >> 172.21.15.178:0/1244597475 conn(0x7f51e0101b70 msgr2=0x7f51e01031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:40.910 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 shutdown_connections 2023-10-16T14:24:40.910 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:40.422+0000 7f51e6517700 1 -- 172.21.15.178:0/1244597475 wait complete. 2023-10-16T14:24:40.910 INFO:teuthology.orchestra.run.smithi178.stdout:mgr epoch 5 is available 2023-10-16T14:24:40.911 INFO:teuthology.orchestra.run.smithi178.stdout:Setting orchestrator backend to cephadm... 2023-10-16T14:24:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:41 smithi178 ceph-mon[98495]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-16T14:24:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:41 smithi178 ceph-mon[98495]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-16T14:24:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:41 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:41 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:41 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:42.076 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.449+0000 7f4b90a84700 1 Processor -- start 2023-10-16T14:24:42.077 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.449+0000 7f4b90a84700 1 -- start start 2023-10-16T14:24:42.077 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.449+0000 7f4b90a84700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:42.077 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.450+0000 7f4b90a84700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c07a8e0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.077 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.450+0000 7f4b8a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:42.077 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.450+0000 7f4b8a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48126/0 (socket says 172.21.15.178:48126) 2023-10-16T14:24:42.078 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.450+0000 7f4b8a59c700 1 -- 172.21.15.178:0/3257290592 learned_addr learned my addr 172.21.15.178:0/3257290592 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:42.078 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.450+0000 7f4b8a59c700 1 -- 172.21.15.178:0/3257290592 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c07aa20 con 0x7f4b8c07beb0 2023-10-16T14:24:42.078 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b8a59c700 1 --2- 172.21.15.178:0/3257290592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0098a0 tx=0x7f4b7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d539696303898ad2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:42.078 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b8959a700 1 -- 172.21.15.178:0/3257290592 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c004160 con 0x7f4b8c07beb0 2023-10-16T14:24:42.078 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b8959a700 1 -- 172.21.15.178:0/3257290592 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b7c0042c0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.079 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b8959a700 1 -- 172.21.15.178:0/3257290592 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c011a00 con 0x7f4b8c07beb0 2023-10-16T14:24:42.079 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b90a84700 1 -- 172.21.15.178:0/3257290592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 msgr2=0x7f4b8c07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:42.079 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b90a84700 1 --2- 172.21.15.178:0/3257290592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0098a0 tx=0x7f4b7c009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:42.079 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b90a84700 1 -- 172.21.15.178:0/3257290592 shutdown_connections 2023-10-16T14:24:42.079 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b90a84700 1 --2- 172.21.15.178:0/3257290592 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c07a310 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.451+0000 7f4b90a84700 1 -- 172.21.15.178:0/3257290592 >> 172.21.15.178:0/3257290592 conn(0x7f4b8c101b70 msgr2=0x7f4b8c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.452+0000 7f4b90a84700 1 -- 172.21.15.178:0/3257290592 shutdown_connections 2023-10-16T14:24:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.452+0000 7f4b90a84700 1 -- 172.21.15.178:0/3257290592 wait complete. 2023-10-16T14:24:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.453+0000 7f4b90a84700 1 Processor -- start 2023-10-16T14:24:42.080 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.453+0000 7f4b90a84700 1 -- start start 2023-10-16T14:24:42.081 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b90a84700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:42.081 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b90a84700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c19c9d0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.081 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b8a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:42.081 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b8a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48136/0 (socket says 172.21.15.178:48136) 2023-10-16T14:24:42.081 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b8a59c700 1 -- 172.21.15.178:0/2017243014 learned_addr learned my addr 172.21.15.178:0/2017243014 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:42.082 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b8a59c700 1 -- 172.21.15.178:0/2017243014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7c009550 con 0x7f4b8c07beb0 2023-10-16T14:24:42.082 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.454+0000 7f4b8a59c700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006af0 tx=0x7f4b7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:42.082 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.455+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c003a40 con 0x7f4b8c07beb0 2023-10-16T14:24:42.082 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.455+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4b7c003ba0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.082 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.455+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8c19cbd0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.455+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c01a700 con 0x7f4b8c07beb0 2023-10-16T14:24:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.455+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c19cff0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.456+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 7) v1 ==== 45473+0+0 (secure 0 0 0) 0x7f4b7c024430 con 0x7f4b8c07beb0 2023-10-16T14:24:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.456+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8c0624e0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.083 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.456+0000 7f4b7b7fe700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 0x7f4b7403ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.456+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4b7c04c650 con 0x7f4b8c07beb0 2023-10-16T14:24:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.457+0000 7f4b89d9b700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 0x7f4b7403ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.457+0000 7f4b89d9b700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 0x7f4b7403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b80006fd0 tx=0x7f4b80006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.461+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4b7c0189a0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.084 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.526+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f4b7c0246e0 con 0x7f4b8c07beb0 2023-10-16T14:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.679+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b8c103e70 con 0x7f4b74038740 2023-10-16T14:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.685+0000 7f4b7b7fe700 1 -- 172.21.15.178:0/2017243014 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4b8c103e70 con 0x7f4b74038740 2023-10-16T14:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 msgr2=0x7f4b7403ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 0x7f4b7403ac00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b80006fd0 tx=0x7f4b80006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:42.085 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 msgr2=0x7f4b8c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006af0 tx=0x7f4b7c011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 shutdown_connections 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4b74038740 0x7f4b7403ac00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 --2- 172.21.15.178:0/2017243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b8c07beb0 0x7f4b8c19c490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 >> 172.21.15.178:0/2017243014 conn(0x7f4b8c101b70 msgr2=0x7f4b8c103250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:42.086 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.688+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 shutdown_connections 2023-10-16T14:24:42.087 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:41.689+0000 7f4b90a84700 1 -- 172.21.15.178:0/2017243014 wait complete. 2023-10-16T14:24:42.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: [16/Oct/2023:14:24:40] ENGINE Bus STARTING 2023-10-16T14:24:42.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: [16/Oct/2023:14:24:41] ENGINE Serving on http://172.21.15.178:8765 2023-10-16T14:24:42.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: [16/Oct/2023:14:24:41] ENGINE Serving on https://172.21.15.178:7150 2023-10-16T14:24:42.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: [16/Oct/2023:14:24:41] ENGINE Bus STARTED 2023-10-16T14:24:42.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: mgrmap e8: x(active, since 2s) 2023-10-16T14:24:42.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:42.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:42.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:42 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout value unchanged 2023-10-16T14:24:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.605+0000 7fc73e58a700 1 Processor -- start 2023-10-16T14:24:43.359 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.605+0000 7fc73e58a700 1 -- start start 2023-10-16T14:24:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc73e58a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc73e58a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73807a8c0 con 0x7fc73807be90 2023-10-16T14:24:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc737fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:43.360 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc737fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48146/0 (socket says 172.21.15.178:48146) 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc737fff700 1 -- 172.21.15.178:0/2856355803 learned_addr learned my addr 172.21.15.178:0/2856355803 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc737fff700 1 -- 172.21.15.178:0/2856355803 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73807aa00 con 0x7fc73807be90 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.606+0000 7fc737fff700 1 --2- 172.21.15.178:0/2856355803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098a0 tx=0x7fc728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117bcac3e68b3c3e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc736ffd700 1 -- 172.21.15.178:0/2856355803 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc728004160 con 0x7fc73807be90 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc736ffd700 1 -- 172.21.15.178:0/2856355803 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc7280042c0 con 0x7fc73807be90 2023-10-16T14:24:43.361 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc736ffd700 1 -- 172.21.15.178:0/2856355803 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc728011a00 con 0x7fc73807be90 2023-10-16T14:24:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc73e58a700 1 -- 172.21.15.178:0/2856355803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 msgr2=0x7fc73807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc73e58a700 1 --2- 172.21.15.178:0/2856355803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098a0 tx=0x7fc728009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc73e58a700 1 -- 172.21.15.178:0/2856355803 shutdown_connections 2023-10-16T14:24:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc73e58a700 1 --2- 172.21.15.178:0/2856355803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73807a2f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:43.362 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.607+0000 7fc73e58a700 1 -- 172.21.15.178:0/2856355803 >> 172.21.15.178:0/2856355803 conn(0x7fc738101b70 msgr2=0x7fc738103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.608+0000 7fc73e58a700 1 -- 172.21.15.178:0/2856355803 shutdown_connections 2023-10-16T14:24:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.608+0000 7fc73e58a700 1 -- 172.21.15.178:0/2856355803 wait complete. 2023-10-16T14:24:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.609+0000 7fc73e58a700 1 Processor -- start 2023-10-16T14:24:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.609+0000 7fc73e58a700 1 -- start start 2023-10-16T14:24:43.363 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc73e58a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc73e58a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73819c970 con 0x7fc73807be90 2023-10-16T14:24:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc737fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc737fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48148/0 (socket says 172.21.15.178:48148) 2023-10-16T14:24:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc737fff700 1 -- 172.21.15.178:0/3223711396 learned_addr learned my addr 172.21.15.178:0/3223711396 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:43.364 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.610+0000 7fc737fff700 1 -- 172.21.15.178:0/3223711396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728009550 con 0x7fc73807be90 2023-10-16T14:24:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc737fff700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc728006af0 tx=0x7fc728011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc728003a40 con 0x7fc73807be90 2023-10-16T14:24:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc728003ba0 con 0x7fc73807be90 2023-10-16T14:24:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73819cb70 con 0x7fc73807be90 2023-10-16T14:24:43.365 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc72801a6e0 con 0x7fc73807be90 2023-10-16T14:24:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.611+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73819cf90 con 0x7fc73807be90 2023-10-16T14:24:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.612+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7380624e0 con 0x7fc73807be90 2023-10-16T14:24:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.612+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7fc728024a60 con 0x7fc73807be90 2023-10-16T14:24:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.613+0000 7fc7357fa700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 0x7fc72003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:43.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.613+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc72804c930 con 0x7fc73807be90 2023-10-16T14:24:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.613+0000 7fc7377fe700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 0x7fc72003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.613+0000 7fc7377fe700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 0x7fc72003ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc72c006fd0 tx=0x7fc72c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.618+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc728024430 con 0x7fc73807be90 2023-10-16T14:24:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.816+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fc738103e30 con 0x7fc720038870 2023-10-16T14:24:43.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.817+0000 7fc7357fa700 1 -- 172.21.15.178:0/3223711396 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fc738103e30 con 0x7fc720038870 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 msgr2=0x7fc72003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 0x7fc72003ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc72c006fd0 tx=0x7fc72c006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 msgr2=0x7fc73819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc728006af0 tx=0x7fc728011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 shutdown_connections 2023-10-16T14:24:43.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7fc720038870 0x7fc72003ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 --2- 172.21.15.178:0/3223711396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc73807be90 0x7fc73819c430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 >> 172.21.15.178:0/3223711396 conn(0x7fc738101b70 msgr2=0x7fc7381031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.819+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 shutdown_connections 2023-10-16T14:24:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:42.820+0000 7fc73e58a700 1 -- 172.21.15.178:0/3223711396 wait complete. 2023-10-16T14:24:43.369 INFO:teuthology.orchestra.run.smithi178.stdout:Generating ssh key... 2023-10-16T14:24:43.626 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:43 smithi178 ceph-mon[98495]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:44.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: Generating public/private rsa key pair. 2023-10-16T14:24:44.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: Your identification has been saved in /tmp/tmptra0z9dq/key. 2023-10-16T14:24:44.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: Your public key has been saved in /tmp/tmptra0z9dq/key.pub. 2023-10-16T14:24:44.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: The key fingerprint is: 2023-10-16T14:24:44.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: SHA256:I0NBFq+41k+tmnrVNhx+oNXuTiXjWsx+l+OyKMzxF1c ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:44.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: The key's randomart image is: 2023-10-16T14:24:44.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: +---[RSA 3072]----+ 2023-10-16T14:24:44.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | .=. | 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | . o | 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | . . . | 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | o . + . E| 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | . + S* +o . .| 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | o oo+*+o= . | 2023-10-16T14:24:44.555 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | o ..+.++* o .| 2023-10-16T14:24:44.556 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | . .+ = *oo.o.| 2023-10-16T14:24:44.556 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: | .oo.o oo+o+o.| 2023-10-16T14:24:44.556 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: +----[SHA256]-----+ 2023-10-16T14:24:44.640 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.911+0000 7f3f6efbd700 1 Processor -- start 2023-10-16T14:24:44.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6efbd700 1 -- start start 2023-10-16T14:24:44.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6efbd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:44.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6efbd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6807a8c0 con 0x7f3f6807be90 2023-10-16T14:24:44.641 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6cd59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:44.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6cd59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48156/0 (socket says 172.21.15.178:48156) 2023-10-16T14:24:44.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.912+0000 7f3f6cd59700 1 -- 172.21.15.178:0/228243014 learned_addr learned my addr 172.21.15.178:0/228243014 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:44.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.913+0000 7f3f6cd59700 1 -- 172.21.15.178:0/228243014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6807aa00 con 0x7f3f6807be90 2023-10-16T14:24:44.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.913+0000 7f3f6cd59700 1 --2- 172.21.15.178:0/228243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0098a0 tx=0x7f3f5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aa2a6cd6e9762a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:44.642 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.913+0000 7f3f677fe700 1 -- 172.21.15.178:0/228243014 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c004160 con 0x7f3f6807be90 2023-10-16T14:24:44.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.913+0000 7f3f677fe700 1 -- 172.21.15.178:0/228243014 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f5c0042c0 con 0x7f3f6807be90 2023-10-16T14:24:44.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.913+0000 7f3f677fe700 1 -- 172.21.15.178:0/228243014 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c011a00 con 0x7f3f6807be90 2023-10-16T14:24:44.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 -- 172.21.15.178:0/228243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 msgr2=0x7f3f6807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:44.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/228243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0098a0 tx=0x7f3f5c009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:44.643 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 -- 172.21.15.178:0/228243014 shutdown_connections 2023-10-16T14:24:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/228243014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6807a2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 -- 172.21.15.178:0/228243014 >> 172.21.15.178:0/228243014 conn(0x7f3f68101b70 msgr2=0x7f3f68103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 -- 172.21.15.178:0/228243014 shutdown_connections 2023-10-16T14:24:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.914+0000 7f3f6efbd700 1 -- 172.21.15.178:0/228243014 wait complete. 2023-10-16T14:24:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.915+0000 7f3f6efbd700 1 Processor -- start 2023-10-16T14:24:44.645 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6efbd700 1 -- start start 2023-10-16T14:24:44.645 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6efbd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:44.645 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6efbd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6819c970 con 0x7f3f6807be90 2023-10-16T14:24:44.645 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6cd59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:44.645 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6cd59700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48158/0 (socket says 172.21.15.178:48158) 2023-10-16T14:24:44.646 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.916+0000 7f3f6cd59700 1 -- 172.21.15.178:0/3432974981 learned_addr learned my addr 172.21.15.178:0/3432974981 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:44.646 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f6cd59700 1 -- 172.21.15.178:0/3432974981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c009550 con 0x7f3f6807be90 2023-10-16T14:24:44.646 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f6cd59700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006af0 tx=0x7f3f5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:44.646 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c003a40 con 0x7f3f6807be90 2023-10-16T14:24:44.646 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3f5c003ba0 con 0x7f3f6807be90 2023-10-16T14:24:44.647 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6819cb70 con 0x7f3f6807be90 2023-10-16T14:24:44.647 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f5c01a6e0 con 0x7f3f6807be90 2023-10-16T14:24:44.647 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.917+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6819cf90 con 0x7f3f6807be90 2023-10-16T14:24:44.647 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.918+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f3f5c024a60 con 0x7f3f6807be90 2023-10-16T14:24:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.918+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f680624e0 con 0x7f3f6807be90 2023-10-16T14:24:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.919+0000 7f3f65ffb700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 0x7f3f5803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.919+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3f5c04c930 con 0x7f3f6807be90 2023-10-16T14:24:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.919+0000 7f3f67fff700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 0x7f3f5803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:44.648 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.919+0000 7f3f67fff700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 0x7f3f5803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006fd0 tx=0x7f3f54006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:43.923+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f5c024430 con 0x7f3f6807be90 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.122+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f68103e30 con 0x7f3f58038870 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.241+0000 7f3f65ffb700 1 -- 172.21.15.178:0/3432974981 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3f68103e30 con 0x7f3f58038870 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.243+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 msgr2=0x7f3f5803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.243+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 0x7f3f5803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f54006fd0 tx=0x7f3f54006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:44.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.243+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 msgr2=0x7f3f6819c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.243+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006af0 tx=0x7f3f5c011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 shutdown_connections 2023-10-16T14:24:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f3f58038870 0x7f3f5803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 --2- 172.21.15.178:0/3432974981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f6807be90 0x7f3f6819c430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:44.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 >> 172.21.15.178:0/3432974981 conn(0x7f3f68101b70 msgr2=0x7f3f681031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:44.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 shutdown_connections 2023-10-16T14:24:44.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:44.244+0000 7f3f6efbd700 1 -- 172.21.15.178:0/3432974981 wait complete. 2023-10-16T14:24:45.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:45 smithi178 ceph-mon[98495]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:45.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:45 smithi178 ceph-mon[98495]: Generating ssh key... 2023-10-16T14:24:45.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:45 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:45.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:45 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:45.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:24:45.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.201+0000 7f449ae51700 1 Processor -- start 2023-10-16T14:24:45.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f449ae51700 1 -- start start 2023-10-16T14:24:45.927 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f449ae51700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:45.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f449ae51700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449407a8c0 con 0x7f449407be90 2023-10-16T14:24:45.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f4498bed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:45.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f4498bed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48172/0 (socket says 172.21.15.178:48172) 2023-10-16T14:24:45.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.202+0000 7f4498bed700 1 -- 172.21.15.178:0/3362182400 learned_addr learned my addr 172.21.15.178:0/3362182400 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:45.928 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.203+0000 7f4498bed700 1 -- 172.21.15.178:0/3362182400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449407aa00 con 0x7f449407be90 2023-10-16T14:24:45.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.203+0000 7f4498bed700 1 --2- 172.21.15.178:0/3362182400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44880098a0 tx=0x7f4488009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970095b772ca93fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:45.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.203+0000 7f44937fe700 1 -- 172.21.15.178:0/3362182400 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4488004160 con 0x7f449407be90 2023-10-16T14:24:45.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.203+0000 7f44937fe700 1 -- 172.21.15.178:0/3362182400 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f44880042c0 con 0x7f449407be90 2023-10-16T14:24:45.929 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.203+0000 7f44937fe700 1 -- 172.21.15.178:0/3362182400 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4488011a00 con 0x7f449407be90 2023-10-16T14:24:45.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 -- 172.21.15.178:0/3362182400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 msgr2=0x7f449407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:45.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 --2- 172.21.15.178:0/3362182400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44880098a0 tx=0x7f4488009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:45.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 -- 172.21.15.178:0/3362182400 shutdown_connections 2023-10-16T14:24:45.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 --2- 172.21.15.178:0/3362182400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449407a2f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:45.930 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 -- 172.21.15.178:0/3362182400 >> 172.21.15.178:0/3362182400 conn(0x7f4494101b70 msgr2=0x7f4494103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:45.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 -- 172.21.15.178:0/3362182400 shutdown_connections 2023-10-16T14:24:45.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.204+0000 7f449ae51700 1 -- 172.21.15.178:0/3362182400 wait complete. 2023-10-16T14:24:45.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.205+0000 7f449ae51700 1 Processor -- start 2023-10-16T14:24:45.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.205+0000 7f449ae51700 1 -- start start 2023-10-16T14:24:45.931 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.205+0000 7f449ae51700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.205+0000 7f449ae51700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449419c970 con 0x7f449407be90 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.205+0000 7f4498bed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4498bed700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48188/0 (socket says 172.21.15.178:48188) 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4498bed700 1 -- 172.21.15.178:0/684642578 learned_addr learned my addr 172.21.15.178:0/684642578 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4498bed700 1 -- 172.21.15.178:0/684642578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009550 con 0x7f449407be90 2023-10-16T14:24:45.932 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4498bed700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4488006af0 tx=0x7f4488011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:45.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4488003a40 con 0x7f449407be90 2023-10-16T14:24:45.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4488003ba0 con 0x7f449407be90 2023-10-16T14:24:45.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f448801a6e0 con 0x7f449407be90 2023-10-16T14:24:45.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449419cb70 con 0x7f449407be90 2023-10-16T14:24:45.933 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.206+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449419cf90 con 0x7f449407be90 2023-10-16T14:24:45.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.208+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f4488024430 con 0x7f449407be90 2023-10-16T14:24:45.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.208+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44940624e0 con 0x7f449407be90 2023-10-16T14:24:45.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.208+0000 7f4491ffb700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 0x7f448403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:45.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.208+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f448804c870 con 0x7f449407be90 2023-10-16T14:24:45.934 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.209+0000 7f4493fff700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 0x7f448403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.209+0000 7f4493fff700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 0x7f448403ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4480006fd0 tx=0x7f4480006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.214+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f44880246e0 con 0x7f449407be90 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.412+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4494103e30 con 0x7f4484038820 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.413+0000 7f4491ffb700 1 -- 172.21.15.178:0/684642578 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4494103e30 con 0x7f4484038820 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 msgr2=0x7f448403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:45.935 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 0x7f448403ace0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4480006fd0 tx=0x7f4480006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:45.936 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 msgr2=0x7f449419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:45.936 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4488006af0 tx=0x7f4488011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:45.936 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 shutdown_connections 2023-10-16T14:24:45.936 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f4484038820 0x7f448403ace0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:45.936 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 --2- 172.21.15.178:0/684642578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f449407be90 0x7f449419c430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:45.937 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 >> 172.21.15.178:0/684642578 conn(0x7f4494101b70 msgr2=0x7f44941031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:45.937 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 shutdown_connections 2023-10-16T14:24:45.937 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:45.415+0000 7f449ae51700 1 -- 172.21.15.178:0/684642578 wait complete. 2023-10-16T14:24:45.937 INFO:teuthology.orchestra.run.smithi178.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-10-16T14:24:45.937 INFO:teuthology.orchestra.run.smithi178.stdout:Adding key to root@localhost authorized_keys... 2023-10-16T14:24:45.938 INFO:teuthology.orchestra.run.smithi178.stdout:Adding host smithi178... 2023-10-16T14:24:46.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:46 smithi178 ceph-mon[98495]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:47.419 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:47 smithi178 ceph-mon[98495]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "addr": "172.21.15.178", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:48 smithi178 ceph-mon[98495]: Deploying cephadm binary to smithi178 2023-10-16T14:24:50.416 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Added host 'smithi178' with addr '172.21.15.178' 2023-10-16T14:24:50.417 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.490+0000 7f3779ee2700 1 Processor -- start 2023-10-16T14:24:50.417 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.490+0000 7f3779ee2700 1 -- start start 2023-10-16T14:24:50.417 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.490+0000 7f3779ee2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:50.417 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.490+0000 7f3779ee2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377407a8a0 con 0x7f377407be70 2023-10-16T14:24:50.418 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.491+0000 7f37737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:50.418 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.491+0000 7f37737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48194/0 (socket says 172.21.15.178:48194) 2023-10-16T14:24:50.418 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.491+0000 7f37737fe700 1 -- 172.21.15.178:0/2702961553 learned_addr learned my addr 172.21.15.178:0/2702961553 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:50.418 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.491+0000 7f37737fe700 1 -- 172.21.15.178:0/2702961553 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377407a9e0 con 0x7f377407be70 2023-10-16T14:24:50.418 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.491+0000 7f37737fe700 1 --2- 172.21.15.178:0/2702961553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37640098a0 tx=0x7f3764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a507992a17957a9c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:50.423 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f37727fc700 1 -- 172.21.15.178:0/2702961553 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764004160 con 0x7f377407be70 2023-10-16T14:24:50.423 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f37727fc700 1 -- 172.21.15.178:0/2702961553 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f37640042c0 con 0x7f377407be70 2023-10-16T14:24:50.423 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f37727fc700 1 -- 172.21.15.178:0/2702961553 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764011a00 con 0x7f377407be70 2023-10-16T14:24:50.423 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 -- 172.21.15.178:0/2702961553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 msgr2=0x7f377407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:50.424 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 --2- 172.21.15.178:0/2702961553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f37640098a0 tx=0x7f3764009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:50.424 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 -- 172.21.15.178:0/2702961553 shutdown_connections 2023-10-16T14:24:50.424 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 --2- 172.21.15.178:0/2702961553 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377407a2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:50.424 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 -- 172.21.15.178:0/2702961553 >> 172.21.15.178:0/2702961553 conn(0x7f3774101b70 msgr2=0x7f3774103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:50.424 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 -- 172.21.15.178:0/2702961553 shutdown_connections 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.492+0000 7f3779ee2700 1 -- 172.21.15.178:0/2702961553 wait complete. 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.493+0000 7f3779ee2700 1 Processor -- start 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.493+0000 7f3779ee2700 1 -- start start 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f3779ee2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f3779ee2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377419c980 con 0x7f377407be70 2023-10-16T14:24:50.425 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f37737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:50.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f37737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48210/0 (socket says 172.21.15.178:48210) 2023-10-16T14:24:50.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f37737fe700 1 -- 172.21.15.178:0/3487817823 learned_addr learned my addr 172.21.15.178:0/3487817823 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:50.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f37737fe700 1 -- 172.21.15.178:0/3487817823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764009550 con 0x7f377407be70 2023-10-16T14:24:50.426 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.494+0000 7f37737fe700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3764006af0 tx=0x7f3764011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:50.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.495+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3764003a40 con 0x7f377407be70 2023-10-16T14:24:50.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.495+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3764003ba0 con 0x7f377407be70 2023-10-16T14:24:50.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.495+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377419cb80 con 0x7f377407be70 2023-10-16T14:24:50.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.495+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f376401a6e0 con 0x7f377407be70 2023-10-16T14:24:50.427 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.495+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377419cfa0 con 0x7f377407be70 2023-10-16T14:24:50.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.496+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f376401a860 con 0x7f377407be70 2023-10-16T14:24:50.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.496+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37740624e0 con 0x7f377407be70 2023-10-16T14:24:50.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.496+0000 7f3770ff9700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 0x7f375c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:50.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.496+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f376404c8a0 con 0x7f377407be70 2023-10-16T14:24:50.428 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.497+0000 7f3772ffd700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 0x7f375c03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.497+0000 7f3772ffd700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 0x7f375c03ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.501+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3764024a60 con 0x7f377407be70 2023-10-16T14:24:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:46.703+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "addr": "172.21.15.178", "target": ["mon-mgr", ""]}) v1 -- 0x7f3774103e40 con 0x7f375c038870 2023-10-16T14:24:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.974+0000 7f3770ff9700 1 -- 172.21.15.178:0/3487817823 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3774103e40 con 0x7f375c038870 2023-10-16T14:24:50.429 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 msgr2=0x7f375c03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:50.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 0x7f375c03ad30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3768006fd0 tx=0x7f3768006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:50.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 msgr2=0x7f377419c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:50.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3764006af0 tx=0x7f3764011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:50.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 shutdown_connections 2023-10-16T14:24:50.430 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f375c038870 0x7f375c03ad30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:50.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 --2- 172.21.15.178:0/3487817823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f377407be70 0x7f377419c440 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:50.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 >> 172.21.15.178:0/3487817823 conn(0x7f3774101b70 msgr2=0x7f3774103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:50.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 shutdown_connections 2023-10-16T14:24:50.431 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:49.977+0000 7f3779ee2700 1 -- 172.21.15.178:0/3487817823 wait complete. 2023-10-16T14:24:50.431 INFO:teuthology.orchestra.run.smithi178.stdout:Deploying unmanaged mon service... 2023-10-16T14:24:51.240 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:50 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:51.241 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:50 smithi178 ceph-mon[98495]: Added host smithi178 2023-10-16T14:24:51.241 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:50 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:24:51.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-10-16T14:24:51.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.951+0000 7f90d2470700 1 Processor -- start 2023-10-16T14:24:51.649 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.951+0000 7f90d2470700 1 -- start start 2023-10-16T14:24:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90d2470700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90d2470700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc07a8c0 con 0x7f90cc07be90 2023-10-16T14:24:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38548/0 (socket says 172.21.15.178:38548) 2023-10-16T14:24:51.650 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90cbfff700 1 -- 172.21.15.178:0/1931561587 learned_addr learned my addr 172.21.15.178:0/1931561587 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.952+0000 7f90cbfff700 1 -- 172.21.15.178:0/1931561587 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc07aa00 con 0x7f90cc07be90 2023-10-16T14:24:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90cbfff700 1 --2- 172.21.15.178:0/1931561587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36769dca7142aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90caffd700 1 -- 172.21.15.178:0/1931561587 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90bc004160 con 0x7f90cc07be90 2023-10-16T14:24:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90caffd700 1 -- 172.21.15.178:0/1931561587 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90bc0042c0 con 0x7f90cc07be90 2023-10-16T14:24:51.651 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90caffd700 1 -- 172.21.15.178:0/1931561587 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90bc011a00 con 0x7f90cc07be90 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90d2470700 1 -- 172.21.15.178:0/1931561587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 msgr2=0x7f90cc07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.953+0000 7f90d2470700 1 --2- 172.21.15.178:0/1931561587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0098a0 tx=0x7f90bc009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.954+0000 7f90d2470700 1 -- 172.21.15.178:0/1931561587 shutdown_connections 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.954+0000 7f90d2470700 1 --2- 172.21.15.178:0/1931561587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc07a2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.954+0000 7f90d2470700 1 -- 172.21.15.178:0/1931561587 >> 172.21.15.178:0/1931561587 conn(0x7f90cc101b70 msgr2=0x7f90cc103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:51.652 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.954+0000 7f90d2470700 1 -- 172.21.15.178:0/1931561587 shutdown_connections 2023-10-16T14:24:51.653 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.954+0000 7f90d2470700 1 -- 172.21.15.178:0/1931561587 wait complete. 2023-10-16T14:24:51.653 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90d2470700 1 Processor -- start 2023-10-16T14:24:51.653 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90d2470700 1 -- start start 2023-10-16T14:24:51.653 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90d2470700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:51.653 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90d2470700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90cc19c980 con 0x7f90cc07be90 2023-10-16T14:24:51.654 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:51.654 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38554/0 (socket says 172.21.15.178:38554) 2023-10-16T14:24:51.654 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.955+0000 7f90cbfff700 1 -- 172.21.15.178:0/2089557772 learned_addr learned my addr 172.21.15.178:0/2089557772 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:51.654 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90cbfff700 1 -- 172.21.15.178:0/2089557772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009550 con 0x7f90cc07be90 2023-10-16T14:24:51.654 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90cbfff700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90bc006af0 tx=0x7f90bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90bc003a40 con 0x7f90cc07be90 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f90bc003ba0 con 0x7f90cc07be90 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90cc19cb80 con 0x7f90cc07be90 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f90bc01a6e0 con 0x7f90cc07be90 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.956+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90cc19cfa0 con 0x7f90cc07be90 2023-10-16T14:24:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.957+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f90bc024a60 con 0x7f90cc07be90 2023-10-16T14:24:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.958+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90cc0624e0 con 0x7f90cc07be90 2023-10-16T14:24:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.958+0000 7f90c97fa700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 0x7f90b403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.958+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f90bc04c930 con 0x7f90cc07be90 2023-10-16T14:24:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.958+0000 7f90cb7fe700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 0x7f90b403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:51.656 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.959+0000 7f90cb7fe700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 0x7f90b403ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:50.963+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f90bc024430 con 0x7f90cc07be90 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.208+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f90cc103e80 con 0x7f90b40388c0 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.213+0000 7f90c97fa700 1 -- 172.21.15.178:0/2089557772 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f90cc103e80 con 0x7f90b40388c0 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 msgr2=0x7f90b403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 0x7f90b403ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f90c0006fd0 tx=0x7f90c0006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:51.657 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 msgr2=0x7f90cc19c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:51.658 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90bc006af0 tx=0x7f90bc011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:51.658 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 shutdown_connections 2023-10-16T14:24:51.658 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f90b40388c0 0x7f90b403ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:51.658 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 --2- 172.21.15.178:0/2089557772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f90cc07be90 0x7f90cc19c440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:51.658 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 >> 172.21.15.178:0/2089557772 conn(0x7f90cc101b70 msgr2=0x7f90cc103260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:51.659 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 shutdown_connections 2023-10-16T14:24:51.659 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:51.216+0000 7f90d2470700 1 -- 172.21.15.178:0/2089557772 wait complete. 2023-10-16T14:24:51.659 INFO:teuthology.orchestra.run.smithi178.stdout:Deploying unmanaged mgr service... 2023-10-16T14:24:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:52 smithi178 ceph-mon[98495]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:52 smithi178 ceph-mon[98495]: Saving service mon spec with placement count:5 2023-10-16T14:24:52.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:52 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:53.280 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-10-16T14:24:53.280 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.527+0000 7ff133719700 1 Processor -- start 2023-10-16T14:24:53.280 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff133719700 1 -- start start 2023-10-16T14:24:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff133719700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff133719700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c079de0 con 0x7ff12c1070e0 2023-10-16T14:24:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff1314b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff1314b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38564/0 (socket says 172.21.15.178:38564) 2023-10-16T14:24:53.281 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.528+0000 7ff1314b5700 1 -- 172.21.15.178:0/2219785841 learned_addr learned my addr 172.21.15.178:0/2219785841 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:53.282 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.529+0000 7ff1314b5700 1 -- 172.21.15.178:0/2219785841 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c079f20 con 0x7ff12c1070e0 2023-10-16T14:24:53.282 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.529+0000 7ff1314b5700 1 --2- 172.21.15.178:0/2219785841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098a0 tx=0x7ff128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8333766f42b8980 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:53.282 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.529+0000 7ff123fff700 1 -- 172.21.15.178:0/2219785841 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff128004160 con 0x7ff12c1070e0 2023-10-16T14:24:53.282 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.529+0000 7ff123fff700 1 -- 172.21.15.178:0/2219785841 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff1280042c0 con 0x7ff12c1070e0 2023-10-16T14:24:53.282 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.529+0000 7ff123fff700 1 -- 172.21.15.178:0/2219785841 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff128011a00 con 0x7ff12c1070e0 2023-10-16T14:24:53.283 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 -- 172.21.15.178:0/2219785841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 msgr2=0x7ff12c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:53.283 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 --2- 172.21.15.178:0/2219785841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff1280098a0 tx=0x7ff128009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:53.283 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 -- 172.21.15.178:0/2219785841 shutdown_connections 2023-10-16T14:24:53.283 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 --2- 172.21.15.178:0/2219785841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c1094d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:53.283 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 -- 172.21.15.178:0/2219785841 >> 172.21.15.178:0/2219785841 conn(0x7ff12c100a50 msgr2=0x7ff12c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:53.284 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 -- 172.21.15.178:0/2219785841 shutdown_connections 2023-10-16T14:24:53.284 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.530+0000 7ff133719700 1 -- 172.21.15.178:0/2219785841 wait complete. 2023-10-16T14:24:53.284 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.531+0000 7ff133719700 1 Processor -- start 2023-10-16T14:24:53.284 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.531+0000 7ff133719700 1 -- start start 2023-10-16T14:24:53.284 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.531+0000 7ff133719700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:53.285 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.531+0000 7ff133719700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c19c950 con 0x7ff12c1070e0 2023-10-16T14:24:53.285 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1314b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:53.285 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1314b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38576/0 (socket says 172.21.15.178:38576) 2023-10-16T14:24:53.285 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1314b5700 1 -- 172.21.15.178:0/3926773384 learned_addr learned my addr 172.21.15.178:0/3926773384 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:53.285 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1314b5700 1 -- 172.21.15.178:0/3926773384 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009550 con 0x7ff12c1070e0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1314b5700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff128006af0 tx=0x7ff128011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff128003a40 con 0x7ff12c1070e0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff128003ba0 con 0x7ff12c1070e0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff12801a6e0 con 0x7ff12c1070e0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.532+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c19cb50 con 0x7ff12c1070e0 2023-10-16T14:24:53.286 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.533+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c19cf70 con 0x7ff12c1070e0 2023-10-16T14:24:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.534+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7ff128024a60 con 0x7ff12c1070e0 2023-10-16T14:24:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.534+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c10a740 con 0x7ff12c1070e0 2023-10-16T14:24:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.534+0000 7ff1227fc700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 0x7ff11803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.534+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff12804c960 con 0x7ff12c1070e0 2023-10-16T14:24:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.534+0000 7ff130cb4700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 0x7ff11803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:53.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.535+0000 7ff130cb4700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 0x7ff11803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff11c006fd0 tx=0x7ff11c006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:53.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.539+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff128024430 con 0x7ff12c1070e0 2023-10-16T14:24:53.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.546+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mon.0 v2:172.21.15.178:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12804c2b0 con 0x7ff12c1070e0 2023-10-16T14:24:53.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.767+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff12c19f9b0 con 0x7ff118038870 2023-10-16T14:24:53.288 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.772+0000 7ff1227fc700 1 -- 172.21.15.178:0/3926773384 <== mgr.14116 v2:172.21.15.178:6800/3157107959 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff12c19f9b0 con 0x7ff118038870 2023-10-16T14:24:53.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.774+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 msgr2=0x7ff11803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:53.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.774+0000 7ff133719700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 0x7ff11803ad30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff11c006fd0 tx=0x7ff11c006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:53.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.774+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 msgr2=0x7ff12c19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:53.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.774+0000 7ff133719700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff128006af0 tx=0x7ff128011ad0 comp rx=0 tx=0).stop 2023-10-16T14:24:53.289 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 shutdown_connections 2023-10-16T14:24:53.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7ff118038870 0x7ff11803ad30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:53.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 --2- 172.21.15.178:0/3926773384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c1070e0 0x7ff12c19c410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:53.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 >> 172.21.15.178:0/3926773384 conn(0x7ff12c100a50 msgr2=0x7ff12c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:53.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 shutdown_connections 2023-10-16T14:24:53.290 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:52.775+0000 7ff133719700 1 -- 172.21.15.178:0/3926773384 wait complete. 2023-10-16T14:24:53.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:53 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:53.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:53 smithi178 ceph-mon[98495]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:24:53.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:53 smithi178 ceph-mon[98495]: Saving service mgr spec with placement count:2 2023-10-16T14:24:53.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:53 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:53.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:53 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:54.673 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.877+0000 7f60ca305700 1 Processor -- start 2023-10-16T14:24:54.674 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.877+0000 7f60ca305700 1 -- start start 2023-10-16T14:24:54.678 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60ca305700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:54.678 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60ca305700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c407a8c0 con 0x7f60c407be90 2023-10-16T14:24:54.678 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:54.679 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38592/0 (socket says 172.21.15.178:38592) 2023-10-16T14:24:54.679 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60c3fff700 1 -- 172.21.15.178:0/4231659216 learned_addr learned my addr 172.21.15.178:0/4231659216 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:54.679 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.878+0000 7f60c3fff700 1 -- 172.21.15.178:0/4231659216 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c407aa00 con 0x7f60c407be90 2023-10-16T14:24:54.679 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.879+0000 7f60c3fff700 1 --2- 172.21.15.178:0/4231659216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60b40098a0 tx=0x7f60b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a863b97bb512dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:54.679 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.879+0000 7f60c2ffd700 1 -- 172.21.15.178:0/4231659216 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b4004160 con 0x7f60c407be90 2023-10-16T14:24:54.680 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.879+0000 7f60c2ffd700 1 -- 172.21.15.178:0/4231659216 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b40042c0 con 0x7f60c407be90 2023-10-16T14:24:54.680 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.879+0000 7f60c2ffd700 1 -- 172.21.15.178:0/4231659216 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b40119b0 con 0x7f60c407be90 2023-10-16T14:24:54.680 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 -- 172.21.15.178:0/4231659216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 msgr2=0x7f60c407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 --2- 172.21.15.178:0/4231659216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60b40098a0 tx=0x7f60b4009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 -- 172.21.15.178:0/4231659216 shutdown_connections 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 --2- 172.21.15.178:0/4231659216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c407a2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 -- 172.21.15.178:0/4231659216 >> 172.21.15.178:0/4231659216 conn(0x7f60c4101b70 msgr2=0x7f60c4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 -- 172.21.15.178:0/4231659216 shutdown_connections 2023-10-16T14:24:54.681 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.880+0000 7f60ca305700 1 -- 172.21.15.178:0/4231659216 wait complete. 2023-10-16T14:24:54.682 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.881+0000 7f60ca305700 1 Processor -- start 2023-10-16T14:24:54.682 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60ca305700 1 -- start start 2023-10-16T14:24:54.682 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60ca305700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:54.682 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60ca305700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c419c990 con 0x7f60c407be90 2023-10-16T14:24:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60c3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38608/0 (socket says 172.21.15.178:38608) 2023-10-16T14:24:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.882+0000 7f60c3fff700 1 -- 172.21.15.178:0/602195493 learned_addr learned my addr 172.21.15.178:0/602195493 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60c3fff700 1 -- 172.21.15.178:0/602195493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4009550 con 0x7f60c407be90 2023-10-16T14:24:54.683 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60c3fff700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f60b4011980 tx=0x7f60b4011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:54.684 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b4003a60 con 0x7f60c407be90 2023-10-16T14:24:54.684 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b4003bc0 con 0x7f60c407be90 2023-10-16T14:24:54.684 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c419cb90 con 0x7f60c407be90 2023-10-16T14:24:54.685 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b401a670 con 0x7f60c407be90 2023-10-16T14:24:54.685 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.883+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c419cfb0 con 0x7f60c407be90 2023-10-16T14:24:54.685 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.884+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f60b4024440 con 0x7f60c407be90 2023-10-16T14:24:54.685 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.884+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c40624e0 con 0x7f60c407be90 2023-10-16T14:24:54.685 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.885+0000 7f60c17fa700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 0x7f60ac03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:54.686 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.885+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f60b404c9b0 con 0x7f60c407be90 2023-10-16T14:24:54.686 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.885+0000 7f60c37fe700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 0x7f60ac03ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:54.686 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.886+0000 7f60c37fe700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 0x7f60ac03ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006fd0 tx=0x7f60b8006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:54.687 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:53.890+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f60b40246f0 con 0x7f60c407be90 2023-10-16T14:24:54.687 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.100+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f60c410c9d0 con 0x7f60c407be90 2023-10-16T14:24:54.687 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.103+0000 7f60c17fa700 1 -- 172.21.15.178:0/602195493 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f60b4018350 con 0x7f60c407be90 2023-10-16T14:24:54.687 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.106+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 msgr2=0x7f60ac03ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:54.688 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 0x7f60ac03ad30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006fd0 tx=0x7f60b8006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:54.688 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 msgr2=0x7f60c419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:54.688 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f60b4011980 tx=0x7f60b4011a60 comp rx=0 tx=0).stop 2023-10-16T14:24:54.688 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 shutdown_connections 2023-10-16T14:24:54.689 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f60ac038870 0x7f60ac03ad30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:54.689 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 --2- 172.21.15.178:0/602195493 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60c407be90 0x7f60c419c450 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:54.689 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 >> 172.21.15.178:0/602195493 conn(0x7f60c4101b70 msgr2=0x7f60c4102940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:54.689 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 shutdown_connections 2023-10-16T14:24:54.690 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:54.107+0000 7f60ca305700 1 -- 172.21.15.178:0/602195493 wait complete. 2023-10-16T14:24:55.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:55 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/602195493' entity='client.admin' 2023-10-16T14:24:56.059 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.397+0000 7f6d773a7700 1 Processor -- start 2023-10-16T14:24:56.060 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d773a7700 1 -- start start 2023-10-16T14:24:56.060 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d773a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:56.060 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d773a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7007a8c0 con 0x7f6d7007be90 2023-10-16T14:24:56.060 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d75143700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:56.061 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d75143700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38620/0 (socket says 172.21.15.178:38620) 2023-10-16T14:24:56.061 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.398+0000 7f6d75143700 1 -- 172.21.15.178:0/4020444552 learned_addr learned my addr 172.21.15.178:0/4020444552 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:56.061 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.399+0000 7f6d75143700 1 -- 172.21.15.178:0/4020444552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7007aa00 con 0x7f6d7007be90 2023-10-16T14:24:56.061 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.399+0000 7f6d75143700 1 --2- 172.21.15.178:0/4020444552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0098a0 tx=0x7f6d6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d510edbf437a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:56.061 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.399+0000 7f6d67fff700 1 -- 172.21.15.178:0/4020444552 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c004160 con 0x7f6d7007be90 2023-10-16T14:24:56.062 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.399+0000 7f6d67fff700 1 -- 172.21.15.178:0/4020444552 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c0042c0 con 0x7f6d7007be90 2023-10-16T14:24:56.062 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.399+0000 7f6d67fff700 1 -- 172.21.15.178:0/4020444552 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c0119b0 con 0x7f6d7007be90 2023-10-16T14:24:56.062 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 -- 172.21.15.178:0/4020444552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 msgr2=0x7f6d7007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:56.062 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 --2- 172.21.15.178:0/4020444552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0098a0 tx=0x7f6d6c009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:56.062 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 -- 172.21.15.178:0/4020444552 shutdown_connections 2023-10-16T14:24:56.063 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 --2- 172.21.15.178:0/4020444552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7007a2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:56.063 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 -- 172.21.15.178:0/4020444552 >> 172.21.15.178:0/4020444552 conn(0x7f6d70101b70 msgr2=0x7f6d70103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:56.063 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 -- 172.21.15.178:0/4020444552 shutdown_connections 2023-10-16T14:24:56.063 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.400+0000 7f6d773a7700 1 -- 172.21.15.178:0/4020444552 wait complete. 2023-10-16T14:24:56.063 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.401+0000 7f6d773a7700 1 Processor -- start 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.401+0000 7f6d773a7700 1 -- start start 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.401+0000 7f6d773a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.401+0000 7f6d773a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7019c990 con 0x7f6d7007be90 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d75143700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d75143700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38636/0 (socket says 172.21.15.178:38636) 2023-10-16T14:24:56.064 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d75143700 1 -- 172.21.15.178:0/1748932273 learned_addr learned my addr 172.21.15.178:0/1748932273 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:56.065 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d75143700 1 -- 172.21.15.178:0/1748932273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c009550 con 0x7f6d7007be90 2023-10-16T14:24:56.065 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d75143700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c011980 tx=0x7f6d6c011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:56.065 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c003a60 con 0x7f6d7007be90 2023-10-16T14:24:56.065 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d6c003bc0 con 0x7f6d7007be90 2023-10-16T14:24:56.065 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.402+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d6c01a670 con 0x7f6d7007be90 2023-10-16T14:24:56.066 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.403+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7019cb90 con 0x7f6d7007be90 2023-10-16T14:24:56.066 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.403+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7019cfb0 con 0x7f6d7007be90 2023-10-16T14:24:56.066 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.404+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f6d6c024a00 con 0x7f6d7007be90 2023-10-16T14:24:56.066 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.404+0000 7f6d667fc700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 0x7f6d5c03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:56.066 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.404+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d6c04c9c0 con 0x7f6d7007be90 2023-10-16T14:24:56.067 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.404+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d700624e0 con 0x7f6d7007be90 2023-10-16T14:24:56.067 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.404+0000 7f6d74942700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 0x7f6d5c03ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:56.067 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.405+0000 7f6d74942700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 0x7f6d5c03ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006fd0 tx=0x7f6d60006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:56.067 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.410+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6d6c018db0 con 0x7f6d7007be90 2023-10-16T14:24:56.068 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.621+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6d7010c9d0 con 0x7f6d7007be90 2023-10-16T14:24:56.068 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.624+0000 7f6d667fc700 1 -- 172.21.15.178:0/1748932273 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6d6c018350 con 0x7f6d7007be90 2023-10-16T14:24:56.068 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.626+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 msgr2=0x7f6d5c03ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:56.068 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 0x7f6d5c03ace0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6d60006fd0 tx=0x7f6d60006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:56.068 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 msgr2=0x7f6d7019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c011980 tx=0x7f6d6c011a60 comp rx=0 tx=0).stop 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 shutdown_connections 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f6d5c038820 0x7f6d5c03ace0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 --2- 172.21.15.178:0/1748932273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d7007be90 0x7f6d7019c450 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 >> 172.21.15.178:0/1748932273 conn(0x7f6d70101b70 msgr2=0x7f6d70103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:56.069 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 shutdown_connections 2023-10-16T14:24:56.070 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:55.627+0000 7f6d773a7700 1 -- 172.21.15.178:0/1748932273 wait complete. 2023-10-16T14:24:56.070 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling the dashboard module... 2023-10-16T14:24:56.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:56 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:56.337 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1748932273' entity='client.admin' 2023-10-16T14:24:58.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:57 smithi178 ceph-mon[98495]: from='mgr.14116 172.21.15.178:0/2064104863' entity='mgr.x' 2023-10-16T14:24:58.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:57 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3416488661' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-10-16T14:24:58.053 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: ignoring --setuser ceph since I am not root 2023-10-16T14:24:58.054 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: ignoring --setgroup ceph since I am not root 2023-10-16T14:24:58.357 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.605+0000 7f8db5d82700 1 Processor -- start 2023-10-16T14:24:58.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.605+0000 7f8db5d82700 1 -- start start 2023-10-16T14:24:58.366 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.605+0000 7f8db5d82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.605+0000 7f8db5d82700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db0106f80 con 0x7f8db0106590 2023-10-16T14:24:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:58.367 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38648/0 (socket says 172.21.15.178:38648) 2023-10-16T14:24:58.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8daf7fe700 1 -- 172.21.15.178:0/2566836978 learned_addr learned my addr 172.21.15.178:0/2566836978 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:58.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8daf7fe700 1 -- 172.21.15.178:0/2566836978 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0107790 con 0x7f8db0106590 2023-10-16T14:24:58.368 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8daf7fe700 1 --2- 172.21.15.178:0/2566836978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098a0 tx=0x7f8da0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c11efa6aafe668 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:58.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.606+0000 7f8dae7fc700 1 -- 172.21.15.178:0/2566836978 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8da0004160 con 0x7f8db0106590 2023-10-16T14:24:58.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8dae7fc700 1 -- 172.21.15.178:0/2566836978 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da00042c0 con 0x7f8db0106590 2023-10-16T14:24:58.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8dae7fc700 1 -- 172.21.15.178:0/2566836978 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8da00119b0 con 0x7f8db0106590 2023-10-16T14:24:58.369 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 -- 172.21.15.178:0/2566836978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 msgr2=0x7f8db01069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:58.370 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 --2- 172.21.15.178:0/2566836978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8da00098a0 tx=0x7f8da0009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:58.370 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 -- 172.21.15.178:0/2566836978 shutdown_connections 2023-10-16T14:24:58.370 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 --2- 172.21.15.178:0/2566836978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db01069b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:58.370 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 -- 172.21.15.178:0/2566836978 >> 172.21.15.178:0/2566836978 conn(0x7f8db0101b70 msgr2=0x7f8db0103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:58.371 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 -- 172.21.15.178:0/2566836978 shutdown_connections 2023-10-16T14:24:58.371 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.607+0000 7f8db5d82700 1 -- 172.21.15.178:0/2566836978 wait complete. 2023-10-16T14:24:58.371 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.608+0000 7f8db5d82700 1 Processor -- start 2023-10-16T14:24:58.372 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.608+0000 7f8db5d82700 1 -- start start 2023-10-16T14:24:58.372 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8db5d82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:58.372 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8db5d82700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db019ca20 con 0x7f8db0106590 2023-10-16T14:24:58.372 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:58.373 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8daf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38650/0 (socket says 172.21.15.178:38650) 2023-10-16T14:24:58.373 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8daf7fe700 1 -- 172.21.15.178:0/3416488661 learned_addr learned my addr 172.21.15.178:0/3416488661 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:58.373 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8daf7fe700 1 -- 172.21.15.178:0/3416488661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da0009550 con 0x7f8db0106590 2023-10-16T14:24:58.373 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.609+0000 7f8daf7fe700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:58.374 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.610+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8da00039f0 con 0x7f8db0106590 2023-10-16T14:24:58.374 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.610+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db019cc20 con 0x7f8db0106590 2023-10-16T14:24:58.374 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.610+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8da0003b50 con 0x7f8db0106590 2023-10-16T14:24:58.375 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.610+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8da001a780 con 0x7f8db0106590 2023-10-16T14:24:58.375 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.610+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db019d040 con 0x7f8db0106590 2023-10-16T14:24:58.375 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.611+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 8) v1 ==== 45579+0+0 (secure 0 0 0) 0x7f8da0024440 con 0x7f8db0106590 2023-10-16T14:24:58.375 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.611+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db00624e0 con 0x7f8db0106590 2023-10-16T14:24:58.376 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.611+0000 7f8dacff9700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 0x7f8d9803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:58.376 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.611+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8da004c840 con 0x7f8db0106590 2023-10-16T14:24:58.376 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.612+0000 7f8daeffd700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 0x7f8d9803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:58.377 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.612+0000 7f8daeffd700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 0x7f8d9803ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006fd0 tx=0x7f8da4006e40 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:58.377 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.616+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8da00246f0 con 0x7f8db0106590 2023-10-16T14:24:58.377 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:56.891+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8db010a980 con 0x7f8db0106590 2023-10-16T14:24:58.378 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.778+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8da0018350 con 0x7f8db0106590 2023-10-16T14:24:58.378 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.780+0000 7f8dacff9700 1 -- 172.21.15.178:0/3416488661 <== mon.0 v2:172.21.15.178:3300/0 8 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f8da004ed00 con 0x7f8db0106590 2023-10-16T14:24:58.378 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.782+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 msgr2=0x7f8d9803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:58.379 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.782+0000 7f8db5d82700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 0x7f8d9803ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8da4006fd0 tx=0x7f8da4006e40 comp rx=0 tx=0).stop 2023-10-16T14:24:58.379 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.782+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 msgr2=0x7f8db019c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:58.379 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.782+0000 7f8db5d82700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8da0000c00 tx=0x7f8da0011a30 comp rx=0 tx=0).stop 2023-10-16T14:24:58.380 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 shutdown_connections 2023-10-16T14:24:58.380 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f8d98038870 0x7f8d9803ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:58.380 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 --2- 172.21.15.178:0/3416488661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db0106590 0x7f8db019c4e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:58.381 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 >> 172.21.15.178:0/3416488661 conn(0x7f8db0101b70 msgr2=0x7f8db0103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:58.381 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 shutdown_connections 2023-10-16T14:24:58.381 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:57.783+0000 7f8db5d82700 1 -- 172.21.15.178:0/3416488661 wait complete. 2023-10-16T14:24:58.394 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:58.100+0000 7f068b617200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T14:24:58.395 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:58.204+0000 7f068b617200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T14:24:58.395 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:58.333+0000 7f068b617200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T14:24:58.667 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:58.477+0000 7f068b617200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T14:24:58.932 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:58.713+0000 7f068b617200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T14:24:58.932 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3416488661' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-10-16T14:24:58.932 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:58 smithi178 ceph-mon[98495]: mgrmap e9: x(active, since 18s) 2023-10-16T14:24:59.454 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:59.188+0000 7f068b617200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T14:24:59.454 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:59.358+0000 7f068b617200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T14:24:59.782 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:59.478+0000 7f068b617200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T14:24:59.783 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:24:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:24:59.625+0000 7f068b617200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T14:24:59.828 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:24:59.829 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-10-16T14:24:59.829 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "available": true, 2023-10-16T14:24:59.829 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-10-16T14:24:59.829 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-10-16T14:24:59.829 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:24:59.830 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.154+0000 7f9dda56a700 1 Processor -- start 2023-10-16T14:24:59.830 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.154+0000 7f9dda56a700 1 -- start start 2023-10-16T14:24:59.830 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.155+0000 7f9dda56a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:59.830 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.155+0000 7f9dda56a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd4106f80 con 0x7f9dd4106590 2023-10-16T14:24:59.830 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.155+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:59.831 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.155+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44654/0 (socket says 172.21.15.178:44654) 2023-10-16T14:24:59.831 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.155+0000 7f9dd3fff700 1 -- 172.21.15.178:0/1887626642 learned_addr learned my addr 172.21.15.178:0/1887626642 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:59.831 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.156+0000 7f9dd3fff700 1 -- 172.21.15.178:0/1887626642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4107790 con 0x7f9dd4106590 2023-10-16T14:24:59.831 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.156+0000 7f9dd3fff700 1 --2- 172.21.15.178:0/1887626642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40098a0 tx=0x7f9dc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aafd918b2f6e99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:59.831 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.156+0000 7f9dd2ffd700 1 -- 172.21.15.178:0/1887626642 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc4004160 con 0x7f9dd4106590 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.156+0000 7f9dd2ffd700 1 -- 172.21.15.178:0/1887626642 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc40042c0 con 0x7f9dd4106590 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.156+0000 7f9dd2ffd700 1 -- 172.21.15.178:0/1887626642 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc40119b0 con 0x7f9dd4106590 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 -- 172.21.15.178:0/1887626642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 msgr2=0x7f9dd41069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 --2- 172.21.15.178:0/1887626642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40098a0 tx=0x7f9dc4009bb0 comp rx=0 tx=0).stop 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 -- 172.21.15.178:0/1887626642 shutdown_connections 2023-10-16T14:24:59.832 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 --2- 172.21.15.178:0/1887626642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd41069b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 -- 172.21.15.178:0/1887626642 >> 172.21.15.178:0/1887626642 conn(0x7f9dd4101b70 msgr2=0x7f9dd4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 -- 172.21.15.178:0/1887626642 shutdown_connections 2023-10-16T14:24:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.157+0000 7f9dda56a700 1 -- 172.21.15.178:0/1887626642 wait complete. 2023-10-16T14:24:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.158+0000 7f9dda56a700 1 Processor -- start 2023-10-16T14:24:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.158+0000 7f9dda56a700 1 -- start start 2023-10-16T14:24:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dda56a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dda56a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd419a820 con 0x7f9dd4106590 2023-10-16T14:24:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:24:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dd3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44660/0 (socket says 172.21.15.178:44660) 2023-10-16T14:24:59.834 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dd3fff700 1 -- 172.21.15.178:0/3748481411 learned_addr learned my addr 172.21.15.178:0/3748481411 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:24:59.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dd3fff700 1 -- 172.21.15.178:0/3748481411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009550 con 0x7f9dd4106590 2023-10-16T14:24:59.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.159+0000 7f9dd3fff700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000c00 tx=0x7f9dc4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:24:59.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.160+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc40039f0 con 0x7f9dd4106590 2023-10-16T14:24:59.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.160+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dc4003b50 con 0x7f9dd4106590 2023-10-16T14:24:59.835 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.160+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd419aa20 con 0x7f9dd4106590 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.160+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc401a780 con 0x7f9dd4106590 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.160+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd419ae40 con 0x7f9dd4106590 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.161+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd40624e0 con 0x7f9dd4106590 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.161+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f9dc4024440 con 0x7f9dd4106590 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.162+0000 7f9dd17fa700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 0x7f9dbc03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:24:59.836 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.162+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9dc404c980 con 0x7f9dd4106590 2023-10-16T14:24:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.162+0000 7f9dd37fe700 1 -- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 msgr2=0x7f9dbc03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:24:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.162+0000 7f9dd37fe700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 0x7f9dbc03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:24:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.167+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9dc40189a0 con 0x7f9dd4106590 2023-10-16T14:24:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.363+0000 7f9dd37fe700 1 -- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 msgr2=0x7f9dbc03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:24:59.837 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.363+0000 7f9dd37fe700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 0x7f9dbc03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T14:24:59.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.426+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9dd410a9c0 con 0x7f9dd4106590 2023-10-16T14:24:59.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.426+0000 7f9dd17fa700 1 -- 172.21.15.178:0/3748481411 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9dc4018350 con 0x7f9dd4106590 2023-10-16T14:24:59.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 msgr2=0x7f9dbc03ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:24:59.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 0x7f9dbc03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:59.838 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 msgr2=0x7f9dd419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:24:59.839 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000c00 tx=0x7f9dc4011a30 comp rx=0 tx=0).stop 2023-10-16T14:24:59.839 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 shutdown_connections 2023-10-16T14:24:59.839 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f9dbc0388c0 0x7f9dbc03ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:59.839 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 --2- 172.21.15.178:0/3748481411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9dd4106590 0x7f9dd419a2e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:24:59.839 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 >> 172.21.15.178:0/3748481411 conn(0x7f9dd4101b70 msgr2=0x7f9dd4103230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:24:59.840 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 shutdown_connections 2023-10-16T14:24:59.840 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:24:59.429+0000 7f9dda56a700 1 -- 172.21.15.178:0/3748481411 wait complete. 2023-10-16T14:24:59.840 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for the mgr to restart... 2023-10-16T14:24:59.840 INFO:teuthology.orchestra.run.smithi178.stdout:Waiting for mgr epoch 9... 2023-10-16T14:25:00.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:24:59 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3748481411' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-10-16T14:25:00.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:00.195+0000 7f068b617200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T14:25:01.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:01.053+0000 7f068b617200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T14:25:01.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:01.159+0000 7f068b617200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T14:25:02.460 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:02.113+0000 7f068b617200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:25:02.461 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:02.232+0000 7f068b617200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T14:25:02.802 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:02.457+0000 7f068b617200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:25:03.109 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:02.823+0000 7f068b617200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T14:25:03.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:03.106+0000 7f068b617200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T14:25:03.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:03.297+0000 7f068b617200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T14:25:04.052 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:03.645+0000 7f068b617200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T14:25:04.999 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:04.575+0000 7f068b617200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T14:25:04.999 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:04.748+0000 7f068b617200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T14:25:05.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:04.996+0000 7f068b617200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T14:25:05.303 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:05.141+0000 7f068b617200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T14:25:05.802 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:05.418+0000 7f068b617200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T14:25:06.802 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:06.499+0000 7f068b617200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T14:25:06.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: Active manager daemon x restarted 2023-10-16T14:25:06.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: Activating manager daemon x 2023-10-16T14:25:06.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: osdmap e3: 0 total, 0 up, 0 in 2023-10-16T14:25:06.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: mgrmap e10: x(active, starting, since 0.00430901s) 2023-10-16T14:25:06.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:25:06.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-10-16T14:25:06.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-10-16T14:25:06.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-10-16T14:25:06.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-10-16T14:25:06.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: Manager daemon x is now available 2023-10-16T14:25:06.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:06.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:07.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-10-16T14:25:07.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-10-16T14:25:07.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:07 smithi178 ceph-mon[98495]: mgrmap e11: x(active, since 1.00851s) 2023-10-16T14:25:08.030 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout { 2023-10-16T14:25:08.031 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-10-16T14:25:08.031 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout "initialized": true 2023-10-16T14:25:08.032 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout } 2023-10-16T14:25:08.032 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.365+0000 7f8059c0c700 1 Processor -- start 2023-10-16T14:25:08.032 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.365+0000 7f8059c0c700 1 -- start start 2023-10-16T14:25:08.032 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.365+0000 7f8059c0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:08.033 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.365+0000 7f8059c0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805407a8c0 con 0x7f805407be90 2023-10-16T14:25:08.033 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:08.033 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44672/0 (socket says 172.21.15.178:44672) 2023-10-16T14:25:08.033 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80537fe700 1 -- 172.21.15.178:0/3669846392 learned_addr learned my addr 172.21.15.178:0/3669846392 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:08.034 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80537fe700 1 -- 172.21.15.178:0/3669846392 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805407aa00 con 0x7f805407be90 2023-10-16T14:25:08.034 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80537fe700 1 --2- 172.21.15.178:0/3669846392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80440098a0 tx=0x7f8044009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d083f10bca85dc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:08.034 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.366+0000 7f80527fc700 1 -- 172.21.15.178:0/3669846392 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8044004160 con 0x7f805407be90 2023-10-16T14:25:08.034 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f80527fc700 1 -- 172.21.15.178:0/3669846392 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80440042c0 con 0x7f805407be90 2023-10-16T14:25:08.034 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f80527fc700 1 -- 172.21.15.178:0/3669846392 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80440119b0 con 0x7f805407be90 2023-10-16T14:25:08.035 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 -- 172.21.15.178:0/3669846392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 msgr2=0x7f805407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:08.035 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 --2- 172.21.15.178:0/3669846392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f80440098a0 tx=0x7f8044009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.035 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 -- 172.21.15.178:0/3669846392 shutdown_connections 2023-10-16T14:25:08.035 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 --2- 172.21.15.178:0/3669846392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805407a2f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 -- 172.21.15.178:0/3669846392 >> 172.21.15.178:0/3669846392 conn(0x7f8054101b70 msgr2=0x7f8054103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 -- 172.21.15.178:0/3669846392 shutdown_connections 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.367+0000 7f8059c0c700 1 -- 172.21.15.178:0/3669846392 wait complete. 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.369+0000 7f8059c0c700 1 Processor -- start 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.369+0000 7f8059c0c700 1 -- start start 2023-10-16T14:25:08.036 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.369+0000 7f8059c0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:08.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.369+0000 7f8059c0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805419c990 con 0x7f805407be90 2023-10-16T14:25:08.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.370+0000 7f80537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:08.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.370+0000 7f80537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44688/0 (socket says 172.21.15.178:44688) 2023-10-16T14:25:08.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.370+0000 7f80537fe700 1 -- 172.21.15.178:0/309673495 learned_addr learned my addr 172.21.15.178:0/309673495 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:08.037 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.370+0000 7f80537fe700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044009550 con 0x7f805407be90 2023-10-16T14:25:08.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.370+0000 7f80537fe700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8044011980 tx=0x7f8044011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:08.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.371+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8044003a60 con 0x7f805407be90 2023-10-16T14:25:08.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.371+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8044003bc0 con 0x7f805407be90 2023-10-16T14:25:08.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.371+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f804401a670 con 0x7f805407be90 2023-10-16T14:25:08.038 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.371+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805419cb90 con 0x7f805407be90 2023-10-16T14:25:08.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.371+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805419cfb0 con 0x7f805407be90 2023-10-16T14:25:08.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 9) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f8044024a00 con 0x7f805407be90 2023-10-16T14:25:08.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8050ff9700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:08.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f803c03b490 con 0x7f803c0388c0 2023-10-16T14:25:08.039 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f804404ca60 con 0x7f805407be90 2023-10-16T14:25:08.040 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8052ffd700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:25:08.040 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.372+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:25:08.040 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.574+0000 7f8052ffd700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:25:08.041 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.574+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T14:25:08.041 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.975+0000 7f8052ffd700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:25:08.041 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:00.975+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-16T14:25:08.041 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:01.776+0000 7f8052ffd700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:25:08.042 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:01.776+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-16T14:25:08.042 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:03.377+0000 7f8052ffd700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.178:6800/3157107959 2023-10-16T14:25:08.042 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:03.377+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-16T14:25:08.043 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:06.504+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mgrmap(e 10) v1 ==== 45359+0+0 (secure 0 0 0) 0x7f8044024620 con 0x7f805407be90 2023-10-16T14:25:08.043 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:06.504+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 msgr2=0x7f803c03ad80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:25:08.043 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:06.504+0000 7f8050ff9700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.044 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.507+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 11) v1 ==== 45486+0+0 (secure 0 0 0) 0x7f804404e870 con 0x7f805407be90 2023-10-16T14:25:08.044 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.507+0000 7f8050ff9700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 0x7f803c03e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:08.044 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.507+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f803c03b490 con 0x7f803c03c360 2023-10-16T14:25:08.044 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.508+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 0x7f803c03e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:08.045 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.509+0000 7f8052ffd700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 0x7f803c03e750 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8048003a10 tx=0x7f8048009270 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:08.045 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.510+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f803c03b490 con 0x7f803c03c360 2023-10-16T14:25:08.045 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.517+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f805419d790 con 0x7f803c03c360 2023-10-16T14:25:08.046 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.517+0000 7f8050ff9700 1 -- 172.21.15.178:0/309673495 <== mgr.14148 v2:172.21.15.178:6800/706630784 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f805419d790 con 0x7f803c03c360 2023-10-16T14:25:08.046 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 msgr2=0x7f803c03e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:08.046 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 0x7f803c03e750 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8048003a10 tx=0x7f8048009270 comp rx=0 tx=0).stop 2023-10-16T14:25:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 msgr2=0x7f805419c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8044011980 tx=0x7f8044011a60 comp rx=0 tx=0).stop 2023-10-16T14:25:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 shutdown_connections 2023-10-16T14:25:08.047 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f803c03c360 0x7f803c03e750 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.048 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f805407be90 0x7f805419c450 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.048 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 --2- 172.21.15.178:0/309673495 >> [v2:172.21.15.178:6800/3157107959,v1:172.21.15.178:6801/3157107959] conn(0x7f803c0388c0 0x7f803c03ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:08.048 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 >> 172.21.15.178:0/309673495 conn(0x7f8054101b70 msgr2=0x7f8054102940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:08.049 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 shutdown_connections 2023-10-16T14:25:08.049 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:07.518+0000 7f8059c0c700 1 -- 172.21.15.178:0/309673495 wait complete. 2023-10-16T14:25:08.049 INFO:teuthology.orchestra.run.smithi178.stdout:mgr epoch 9 is available 2023-10-16T14:25:08.050 INFO:teuthology.orchestra.run.smithi178.stdout:Generating a dashboard self-signed certificate... 2023-10-16T14:25:08.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: [16/Oct/2023:14:25:07] ENGINE Bus STARTING 2023-10-16T14:25:08.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: [16/Oct/2023:14:25:07] ENGINE Serving on https://172.21.15.178:7150 2023-10-16T14:25:08.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: [16/Oct/2023:14:25:07] ENGINE Serving on http://172.21.15.178:8765 2023-10-16T14:25:08.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: [16/Oct/2023:14:25:07] ENGINE Bus STARTED 2023-10-16T14:25:08.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-10-16T14:25:08.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:08 smithi178 ceph-mon[98495]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-10-16T14:25:09.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-10-16T14:25:09.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.607+0000 7f3feb98c700 1 Processor -- start 2023-10-16T14:25:09.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.607+0000 7f3feb98c700 1 -- start start 2023-10-16T14:25:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.608+0000 7f3feb98c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.608+0000 7f3feb98c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe4079e60 con 0x7f3fe4104e50 2023-10-16T14:25:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.609+0000 7f3fe9728700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.609+0000 7f3fe9728700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44754/0 (socket says 172.21.15.178:44754) 2023-10-16T14:25:09.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.609+0000 7f3fe9728700 1 -- 172.21.15.178:0/4108041850 learned_addr learned my addr 172.21.15.178:0/4108041850 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.609+0000 7f3fe9728700 1 -- 172.21.15.178:0/4108041850 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe41077c0 con 0x7f3fe4104e50 2023-10-16T14:25:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.609+0000 7f3fe9728700 1 --2- 172.21.15.178:0/4108041850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00098a0 tx=0x7f3fe0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcaed8480352c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.610+0000 7f3fdbfff700 1 -- 172.21.15.178:0/4108041850 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fe0004160 con 0x7f3fe4104e50 2023-10-16T14:25:09.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.610+0000 7f3fdbfff700 1 -- 172.21.15.178:0/4108041850 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe00042c0 con 0x7f3fe4104e50 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.610+0000 7f3fdbfff700 1 -- 172.21.15.178:0/4108041850 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fe00119b0 con 0x7f3fe4104e50 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.610+0000 7f3feb98c700 1 -- 172.21.15.178:0/4108041850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 msgr2=0x7f3fe4107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.610+0000 7f3feb98c700 1 --2- 172.21.15.178:0/4108041850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00098a0 tx=0x7f3fe0009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.611+0000 7f3feb98c700 1 -- 172.21.15.178:0/4108041850 shutdown_connections 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.611+0000 7f3feb98c700 1 --2- 172.21.15.178:0/4108041850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe4107280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:09.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.611+0000 7f3feb98c700 1 -- 172.21.15.178:0/4108041850 >> 172.21.15.178:0/4108041850 conn(0x7f3fe4100a50 msgr2=0x7f3fe4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:09.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.611+0000 7f3feb98c700 1 -- 172.21.15.178:0/4108041850 shutdown_connections 2023-10-16T14:25:09.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.611+0000 7f3feb98c700 1 -- 172.21.15.178:0/4108041850 wait complete. 2023-10-16T14:25:09.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.612+0000 7f3feb98c700 1 Processor -- start 2023-10-16T14:25:09.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.612+0000 7f3feb98c700 1 -- start start 2023-10-16T14:25:09.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.613+0000 7f3feb98c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:09.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.613+0000 7f3feb98c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe41a0d90 con 0x7f3fe4104e50 2023-10-16T14:25:09.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.613+0000 7f3fe9728700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:09.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.613+0000 7f3fe9728700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44762/0 (socket says 172.21.15.178:44762) 2023-10-16T14:25:09.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.613+0000 7f3fe9728700 1 -- 172.21.15.178:0/3468374616 learned_addr learned my addr 172.21.15.178:0/3468374616 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:09.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3fe9728700 1 -- 172.21.15.178:0/3468374616 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe0009550 con 0x7f3fe4104e50 2023-10-16T14:25:09.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3fe9728700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0011980 tx=0x7f3fe0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:09.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fe0003a60 con 0x7f3fe4104e50 2023-10-16T14:25:09.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe0003bc0 con 0x7f3fe4104e50 2023-10-16T14:25:09.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe41a0f90 con 0x7f3fe4104e50 2023-10-16T14:25:09.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3fe001a670 con 0x7f3fe4104e50 2023-10-16T14:25:09.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.614+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe41a13b0 con 0x7f3fe4104e50 2023-10-16T14:25:09.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.615+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f3fe0024440 con 0x7f3fe4104e50 2023-10-16T14:25:09.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.616+0000 7f3fda7fc700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 0x7f3fd003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:09.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.616+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe419a130 con 0x7f3fe4104e50 2023-10-16T14:25:09.320 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.616+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3fe004cb10 con 0x7f3fe4104e50 2023-10-16T14:25:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.616+0000 7f3fe8f27700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 0x7f3fd003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.617+0000 7f3fe8f27700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 0x7f3fd003ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006fd0 tx=0x7f3fd4006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.621+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3fe00189a0 con 0x7f3fe4104e50 2023-10-16T14:25:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.839+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fe402d0e0 con 0x7f3fd00388c0 2023-10-16T14:25:09.321 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.911+0000 7f3fda7fc700 1 -- 172.21.15.178:0/3468374616 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3fe402d0e0 con 0x7f3fd00388c0 2023-10-16T14:25:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 msgr2=0x7f3fd003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 0x7f3fd003ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4006fd0 tx=0x7f3fd4006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 msgr2=0x7f3fe41a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0011980 tx=0x7f3fe0011a60 comp rx=0 tx=0).stop 2023-10-16T14:25:09.322 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 shutdown_connections 2023-10-16T14:25:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3fd00388c0 0x7f3fd003ad80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 --2- 172.21.15.178:0/3468374616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fe4104e50 0x7f3fe41a0850 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 >> 172.21.15.178:0/3468374616 conn(0x7f3fe4100a50 msgr2=0x7f3fe4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 shutdown_connections 2023-10-16T14:25:09.323 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:08.914+0000 7f3feb98c700 1 -- 172.21.15.178:0/3468374616 wait complete. 2023-10-16T14:25:09.324 INFO:teuthology.orchestra.run.smithi178.stdout:Creating initial admin user... 2023-10-16T14:25:09.616 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:09 smithi178 ceph-mon[98495]: mgrmap e12: x(active, since 2s) 2023-10-16T14:25:09.617 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:09 smithi178 ceph-mon[98495]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:09.617 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:09.617 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:09.618 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5VHugSOx/RX8QunV40xkDuHLkZHG0//ZX1XEVPh6k/KaOKqBUCqKS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1697466310, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-10-16T14:25:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.877+0000 7f661b3b3700 1 Processor -- start 2023-10-16T14:25:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.877+0000 7f661b3b3700 1 -- start start 2023-10-16T14:25:10.798 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.877+0000 7f661b3b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.877+0000 7f661b3b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661407a8e0 con 0x7f661407beb0 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.878+0000 7f661914f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.878+0000 7f661914f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41038/0 (socket says 172.21.15.178:41038) 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.878+0000 7f661914f700 1 -- 172.21.15.178:0/2569827880 learned_addr learned my addr 172.21.15.178:0/2569827880 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.878+0000 7f661914f700 1 -- 172.21.15.178:0/2569827880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661407aa20 con 0x7f661407beb0 2023-10-16T14:25:10.799 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.878+0000 7f661914f700 1 --2- 172.21.15.178:0/2569827880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66100098a0 tx=0x7f6610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b287ad7208fc3d45 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:10.800 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f660bfff700 1 -- 172.21.15.178:0/2569827880 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6610004160 con 0x7f661407beb0 2023-10-16T14:25:10.800 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f660bfff700 1 -- 172.21.15.178:0/2569827880 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66100042c0 con 0x7f661407beb0 2023-10-16T14:25:10.800 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f660bfff700 1 -- 172.21.15.178:0/2569827880 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66100119b0 con 0x7f661407beb0 2023-10-16T14:25:10.800 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 -- 172.21.15.178:0/2569827880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 msgr2=0x7f661407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2569827880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66100098a0 tx=0x7f6610009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 -- 172.21.15.178:0/2569827880 shutdown_connections 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2569827880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661407a310 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 -- 172.21.15.178:0/2569827880 >> 172.21.15.178:0/2569827880 conn(0x7f6614101b70 msgr2=0x7f6614103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.879+0000 7f661b3b3700 1 -- 172.21.15.178:0/2569827880 shutdown_connections 2023-10-16T14:25:10.801 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.880+0000 7f661b3b3700 1 -- 172.21.15.178:0/2569827880 wait complete. 2023-10-16T14:25:10.802 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.881+0000 7f661b3b3700 1 Processor -- start 2023-10-16T14:25:10.802 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.881+0000 7f661b3b3700 1 -- start start 2023-10-16T14:25:10.802 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661b3b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:10.802 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661b3b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661419c9e0 con 0x7f661407beb0 2023-10-16T14:25:10.802 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661914f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:10.803 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661914f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41044/0 (socket says 172.21.15.178:41044) 2023-10-16T14:25:10.803 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661914f700 1 -- 172.21.15.178:0/2070049536 learned_addr learned my addr 172.21.15.178:0/2070049536 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:10.803 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.882+0000 7f661914f700 1 -- 172.21.15.178:0/2070049536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610009550 con 0x7f661407beb0 2023-10-16T14:25:10.803 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f661914f700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6610000c00 tx=0x7f6610011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:10.803 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66100039f0 con 0x7f661407beb0 2023-10-16T14:25:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6610003b50 con 0x7f661407beb0 2023-10-16T14:25:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661419cbe0 con 0x7f661407beb0 2023-10-16T14:25:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f661001a780 con 0x7f661407beb0 2023-10-16T14:25:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.883+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661419d000 con 0x7f661407beb0 2023-10-16T14:25:10.804 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.884+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f6610024440 con 0x7f661407beb0 2023-10-16T14:25:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.884+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66140624e0 con 0x7f661407beb0 2023-10-16T14:25:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.885+0000 7f660a7fc700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 0x7f660003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.885+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f661004c9d0 con 0x7f661407beb0 2023-10-16T14:25:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.885+0000 7f661894e700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 0x7f660003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:10.805 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.886+0000 7f661894e700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 0x7f660003ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6604006fd0 tx=0x7f6604006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:09.891+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f66100189a0 con 0x7f661407beb0 2023-10-16T14:25:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.126+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f661419fa40 con 0x7f66000388c0 2023-10-16T14:25:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.351+0000 7f660a7fc700 1 -- 172.21.15.178:0/2070049536 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f661419fa40 con 0x7f66000388c0 2023-10-16T14:25:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 msgr2=0x7f660003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:10.806 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 0x7f660003ad80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6604006fd0 tx=0x7f6604006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 msgr2=0x7f661419c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6610000c00 tx=0x7f6610011a30 comp rx=0 tx=0).stop 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 shutdown_connections 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f66000388c0 0x7f660003ad80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 --2- 172.21.15.178:0/2070049536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661407beb0 0x7f661419c4a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:10.807 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 >> 172.21.15.178:0/2070049536 conn(0x7f6614101b70 msgr2=0x7f6614103260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:10.808 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 shutdown_connections 2023-10-16T14:25:10.808 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:10.354+0000 7f661b3b3700 1 -- 172.21.15.178:0/2070049536 wait complete. 2023-10-16T14:25:10.808 INFO:teuthology.orchestra.run.smithi178.stdout:Fetching dashboard port number... 2023-10-16T14:25:11.642 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:11 smithi178 ceph-mon[98495]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:11.643 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:11.643 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:12.308 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stdout 8443 2023-10-16T14:25:12.308 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.374+0000 7f5716c9b700 1 Processor -- start 2023-10-16T14:25:12.309 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.374+0000 7f5716c9b700 1 -- start start 2023-10-16T14:25:12.309 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.375+0000 7f5716c9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:12.309 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.375+0000 7f5716c9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571007a8c0 con 0x7f571007be90 2023-10-16T14:25:12.309 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.375+0000 7f5714a37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:12.310 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.375+0000 7f5714a37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41050/0 (socket says 172.21.15.178:41050) 2023-10-16T14:25:12.310 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.375+0000 7f5714a37700 1 -- 172.21.15.178:0/4060612264 learned_addr learned my addr 172.21.15.178:0/4060612264 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:12.310 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.376+0000 7f5714a37700 1 -- 172.21.15.178:0/4060612264 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571007aa00 con 0x7f571007be90 2023-10-16T14:25:12.310 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.376+0000 7f5714a37700 1 --2- 172.21.15.178:0/4060612264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57040098a0 tx=0x7f5704009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6743cd218e2fee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:12.310 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.376+0000 7f570f7fe700 1 -- 172.21.15.178:0/4060612264 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5704004160 con 0x7f571007be90 2023-10-16T14:25:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.376+0000 7f570f7fe700 1 -- 172.21.15.178:0/4060612264 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57040042c0 con 0x7f571007be90 2023-10-16T14:25:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.376+0000 7f570f7fe700 1 -- 172.21.15.178:0/4060612264 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57040119b0 con 0x7f571007be90 2023-10-16T14:25:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 -- 172.21.15.178:0/4060612264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 msgr2=0x7f571007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 --2- 172.21.15.178:0/4060612264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f57040098a0 tx=0x7f5704009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:12.311 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 -- 172.21.15.178:0/4060612264 shutdown_connections 2023-10-16T14:25:12.312 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 --2- 172.21.15.178:0/4060612264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571007a2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:12.312 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 -- 172.21.15.178:0/4060612264 >> 172.21.15.178:0/4060612264 conn(0x7f5710101b70 msgr2=0x7f5710103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:12.312 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 -- 172.21.15.178:0/4060612264 shutdown_connections 2023-10-16T14:25:12.312 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.377+0000 7f5716c9b700 1 -- 172.21.15.178:0/4060612264 wait complete. 2023-10-16T14:25:12.312 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.378+0000 7f5716c9b700 1 Processor -- start 2023-10-16T14:25:12.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.378+0000 7f5716c9b700 1 -- start start 2023-10-16T14:25:12.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.378+0000 7f5716c9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:12.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.378+0000 7f5716c9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571019c990 con 0x7f571007be90 2023-10-16T14:25:12.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.380+0000 7f5714a37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:12.313 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.380+0000 7f5714a37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41054/0 (socket says 172.21.15.178:41054) 2023-10-16T14:25:12.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.380+0000 7f5714a37700 1 -- 172.21.15.178:0/2156032246 learned_addr learned my addr 172.21.15.178:0/2156032246 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:12.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.381+0000 7f5714a37700 1 -- 172.21.15.178:0/2156032246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704009550 con 0x7f571007be90 2023-10-16T14:25:12.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.381+0000 7f5714a37700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5704011980 tx=0x7f5704011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:12.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.381+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5704003a60 con 0x7f571007be90 2023-10-16T14:25:12.314 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.381+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5704003bc0 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.381+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571019cb90 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.382+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571019cfb0 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.382+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f570401a670 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.383+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57100624e0 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.383+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7f570401a930 con 0x7f571007be90 2023-10-16T14:25:12.315 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.383+0000 7f570dffb700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 0x7f570003ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:12.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.383+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f570404ca40 con 0x7f571007be90 2023-10-16T14:25:12.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.384+0000 7f570ffff700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 0x7f570003ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:12.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.384+0000 7f570ffff700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 0x7f570003ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:12.316 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.391+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f570404ee60 con 0x7f571007be90 2023-10-16T14:25:12.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.605+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f571019f9f0 con 0x7f571007be90 2023-10-16T14:25:12.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.605+0000 7f570dffb700 1 -- 172.21.15.178:0/2156032246 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5704018db0 con 0x7f571007be90 2023-10-16T14:25:12.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 msgr2=0x7f570003ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:12.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 0x7f570003ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006fd0 tx=0x7f56fc006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:12.317 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 msgr2=0x7f571019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:12.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5704011980 tx=0x7f5704011a60 comp rx=0 tx=0).stop 2023-10-16T14:25:12.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 shutdown_connections 2023-10-16T14:25:12.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5700038870 0x7f570003ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:12.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 --2- 172.21.15.178:0/2156032246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f571007be90 0x7f571019c450 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:12.318 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.609+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 >> 172.21.15.178:0/2156032246 conn(0x7f5710101b70 msgr2=0x7f5710103200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:12.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.610+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 shutdown_connections 2023-10-16T14:25:12.319 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:11.610+0000 7f5716c9b700 1 -- 172.21.15.178:0/2156032246 wait complete. 2023-10-16T14:25:12.324 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-10-16T14:25:12.325 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout disabled 2023-10-16T14:25:12.341 INFO:teuthology.orchestra.run.smithi178.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-10-16T14:25:12.342 INFO:teuthology.orchestra.run.smithi178.stdout:systemctl: stdout inactive 2023-10-16T14:25:12.342 INFO:teuthology.orchestra.run.smithi178.stdout:firewalld.service is not enabled 2023-10-16T14:25:12.342 INFO:teuthology.orchestra.run.smithi178.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-10-16T14:25:12.344 INFO:teuthology.orchestra.run.smithi178.stdout:Ceph Dashboard is now available at: 2023-10-16T14:25:12.344 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:12.344 INFO:teuthology.orchestra.run.smithi178.stdout: URL: https://smithi178.front.sepia.ceph.com:8443/ 2023-10-16T14:25:12.345 INFO:teuthology.orchestra.run.smithi178.stdout: User: admin 2023-10-16T14:25:12.345 INFO:teuthology.orchestra.run.smithi178.stdout: Password: 7idzof0zpq 2023-10-16T14:25:12.345 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:12.346 INFO:teuthology.orchestra.run.smithi178.stdout:Saving cluster configuration to /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config directory 2023-10-16T14:25:12.346 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling autotune for osd_memory_target 2023-10-16T14:25:12.719 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:12 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2156032246' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-10-16T14:25:13.573 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.969+0000 7fd614f28700 1 Processor -- start 2023-10-16T14:25:13.573 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd614f28700 1 -- start start 2023-10-16T14:25:13.573 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd614f28700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:13.574 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd614f28700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd610079ec0 con 0x7fd610107000 2023-10-16T14:25:13.574 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd60e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:13.574 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd60e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41058/0 (socket says 172.21.15.178:41058) 2023-10-16T14:25:13.574 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.970+0000 7fd60e59c700 1 -- 172.21.15.178:0/3743662331 learned_addr learned my addr 172.21.15.178:0/3743662331 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.971+0000 7fd60e59c700 1 -- 172.21.15.178:0/3743662331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd610109930 con 0x7fd610107000 2023-10-16T14:25:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.971+0000 7fd60e59c700 1 --2- 172.21.15.178:0/3743662331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098a0 tx=0x7fd600009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ab1e73080c523b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.971+0000 7fd60d59a700 1 -- 172.21.15.178:0/3743662331 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd600004160 con 0x7fd610107000 2023-10-16T14:25:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.971+0000 7fd60d59a700 1 -- 172.21.15.178:0/3743662331 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd6000042c0 con 0x7fd610107000 2023-10-16T14:25:13.575 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.971+0000 7fd60d59a700 1 -- 172.21.15.178:0/3743662331 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6000119b0 con 0x7fd610107000 2023-10-16T14:25:13.576 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 -- 172.21.15.178:0/3743662331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 msgr2=0x7fd6101093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:13.576 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 --2- 172.21.15.178:0/3743662331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098a0 tx=0x7fd600009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:13.576 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 -- 172.21.15.178:0/3743662331 shutdown_connections 2023-10-16T14:25:13.576 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 --2- 172.21.15.178:0/3743662331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd6101093f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:13.576 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 -- 172.21.15.178:0/3743662331 >> 172.21.15.178:0/3743662331 conn(0x7fd610100a50 msgr2=0x7fd610102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 -- 172.21.15.178:0/3743662331 shutdown_connections 2023-10-16T14:25:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.972+0000 7fd614f28700 1 -- 172.21.15.178:0/3743662331 wait complete. 2023-10-16T14:25:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.973+0000 7fd614f28700 1 Processor -- start 2023-10-16T14:25:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.973+0000 7fd614f28700 1 -- start start 2023-10-16T14:25:13.577 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.973+0000 7fd614f28700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:13.578 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.973+0000 7fd614f28700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61019c960 con 0x7fd610107000 2023-10-16T14:25:13.578 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd60e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:13.578 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd60e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41064/0 (socket says 172.21.15.178:41064) 2023-10-16T14:25:13.578 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd60e59c700 1 -- 172.21.15.178:0/3779188008 learned_addr learned my addr 172.21.15.178:0/3779188008 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:13.578 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd60e59c700 1 -- 172.21.15.178:0/3779188008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd600009550 con 0x7fd610107000 2023-10-16T14:25:13.579 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd60e59c700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd600011980 tx=0x7fd600011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:13.579 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd600003a60 con 0x7fd610107000 2023-10-16T14:25:13.579 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd600003bc0 con 0x7fd610107000 2023-10-16T14:25:13.579 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61019cb60 con 0x7fd610107000 2023-10-16T14:25:13.579 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.974+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd60001a670 con 0x7fd610107000 2023-10-16T14:25:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.975+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61019cf80 con 0x7fd610107000 2023-10-16T14:25:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.975+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6100624e0 con 0x7fd610107000 2023-10-16T14:25:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.976+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 12) v1 ==== 45592+0+0 (secure 0 0 0) 0x7fd600024a00 con 0x7fd610107000 2023-10-16T14:25:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.977+0000 7fd5ff7fe700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 0x7fd5f803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.977+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd60004ce40 con 0x7fd610107000 2023-10-16T14:25:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.977+0000 7fd60dd9b700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 0x7fd5f803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.977+0000 7fd60dd9b700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 0x7fd5f803ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd604006fd0 tx=0x7fd604006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:12.981+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd600024440 con 0x7fd610107000 2023-10-16T14:25:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.179+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fd61010a770 con 0x7fd610107000 2023-10-16T14:25:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.179+0000 7fd5ff7fe700 1 -- 172.21.15.178:0/3779188008 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fd60002c350 con 0x7fd610107000 2023-10-16T14:25:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 msgr2=0x7fd5f803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 0x7fd5f803ad80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd604006fd0 tx=0x7fd604006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 msgr2=0x7fd61019c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd600011980 tx=0x7fd600011a60 comp rx=0 tx=0).stop 2023-10-16T14:25:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 shutdown_connections 2023-10-16T14:25:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd5f80388c0 0x7fd5f803ad80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 --2- 172.21.15.178:0/3779188008 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd610107000 0x7fd61019c420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 >> 172.21.15.178:0/3779188008 conn(0x7fd610100a50 msgr2=0x7fd6101020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 shutdown_connections 2023-10-16T14:25:13.584 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:13.182+0000 7fd614f28700 1 -- 172.21.15.178:0/3779188008 wait complete. 2023-10-16T14:25:14.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:14 smithi178 ceph-mon[98495]: mgrmap e13: x(active, since 6s) 2023-10-16T14:25:14.956 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.085+0000 7f7c92d9b700 1 Processor -- start 2023-10-16T14:25:14.956 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.085+0000 7f7c92d9b700 1 -- start start 2023-10-16T14:25:14.957 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.085+0000 7f7c92d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:14.957 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.085+0000 7f7c92d9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c07a900 con 0x7f7c8c07bed0 2023-10-16T14:25:14.957 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.086+0000 7f7c90b37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:14.958 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.086+0000 7f7c90b37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41072/0 (socket says 172.21.15.178:41072) 2023-10-16T14:25:14.958 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.086+0000 7f7c90b37700 1 -- 172.21.15.178:0/2112696559 learned_addr learned my addr 172.21.15.178:0/2112696559 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:14.958 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.086+0000 7f7c90b37700 1 -- 172.21.15.178:0/2112696559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c07aa40 con 0x7f7c8c07bed0 2023-10-16T14:25:14.958 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.086+0000 7f7c90b37700 1 --2- 172.21.15.178:0/2112696559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c800098a0 tx=0x7f7c80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2ffc0ad3dc4096 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:14.958 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c8b7fe700 1 -- 172.21.15.178:0/2112696559 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c80004160 con 0x7f7c8c07bed0 2023-10-16T14:25:14.959 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c8b7fe700 1 -- 172.21.15.178:0/2112696559 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c800042c0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.959 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c8b7fe700 1 -- 172.21.15.178:0/2112696559 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c800119b0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.959 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 -- 172.21.15.178:0/2112696559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 msgr2=0x7f7c8c07a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:14.959 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/2112696559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7c800098a0 tx=0x7f7c80009bb0 comp rx=0 tx=0).stop 2023-10-16T14:25:14.960 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 -- 172.21.15.178:0/2112696559 shutdown_connections 2023-10-16T14:25:14.960 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/2112696559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c07a330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:14.960 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 -- 172.21.15.178:0/2112696559 >> 172.21.15.178:0/2112696559 conn(0x7f7c8c101b70 msgr2=0x7f7c8c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:14.960 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 -- 172.21.15.178:0/2112696559 shutdown_connections 2023-10-16T14:25:14.960 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.087+0000 7f7c92d9b700 1 -- 172.21.15.178:0/2112696559 wait complete. 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.088+0000 7f7c92d9b700 1 Processor -- start 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c92d9b700 1 -- start start 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c92d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c92d9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c19c9d0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c90b37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:14.961 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c90b37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41078/0 (socket says 172.21.15.178:41078) 2023-10-16T14:25:14.962 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c90b37700 1 -- 172.21.15.178:0/1829386520 learned_addr learned my addr 172.21.15.178:0/1829386520 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:14.962 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.089+0000 7f7c90b37700 1 -- 172.21.15.178:0/1829386520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80009550 con 0x7f7c8c07bed0 2023-10-16T14:25:14.962 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c90b37700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7c80003e60 tx=0x7f7c80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:14.962 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c80003970 con 0x7f7c8c07bed0 2023-10-16T14:25:14.962 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c80003ad0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.963 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c19cbd0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.963 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c800195e0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.963 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.090+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c19cff0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.963 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.091+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c10c990 con 0x7f7c8c07bed0 2023-10-16T14:25:14.963 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.091+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f7c800198a0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.964 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.092+0000 7f7c89ffb700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 0x7f7c7c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:14.964 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.092+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7c8004cbd0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.964 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.092+0000 7f7c8bfff700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 0x7f7c7c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:14.964 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.093+0000 7f7c8bfff700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 0x7f7c7c03ad80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7c78006fd0 tx=0x7f7c78006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:14.964 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.097+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7c80003c40 con 0x7f7c8c07bed0 2023-10-16T14:25:14.965 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.401+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f7c8c10cbd0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.965 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.403+0000 7f7c89ffb700 1 -- 172.21.15.178:0/1829386520 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f7c8004c4c0 con 0x7f7c8c07bed0 2023-10-16T14:25:14.965 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 msgr2=0x7f7c7c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:14.965 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 0x7f7c7c03ad80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7c78006fd0 tx=0x7f7c78006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:14.965 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 msgr2=0x7f7c8c19c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:14.966 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7c80003e60 tx=0x7f7c80011aa0 comp rx=0 tx=0).stop 2023-10-16T14:25:14.966 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 shutdown_connections 2023-10-16T14:25:14.966 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0388c0 0x7f7c7c03ad80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:14.966 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 --2- 172.21.15.178:0/1829386520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c8c07bed0 0x7f7c8c19c490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:14.966 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.405+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 >> 172.21.15.178:0/1829386520 conn(0x7f7c8c101b70 msgr2=0x7f7c8c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.406+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 shutdown_connections 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr 2023-10-16T14:25:14.406+0000 7f7c92d9b700 1 -- 172.21.15.178:0/1829386520 wait complete. 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.967 INFO:teuthology.orchestra.run.smithi178.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout:Or, if you are only running a single cluster on this host: 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.968 INFO:teuthology.orchestra.run.smithi178.stdout:Please consider enabling telemetry to help improve Ceph: 2023-10-16T14:25:14.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.969 INFO:teuthology.orchestra.run.smithi178.stdout: ceph telemetry on 2023-10-16T14:25:14.969 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.969 INFO:teuthology.orchestra.run.smithi178.stdout:For more information see: 2023-10-16T14:25:14.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.970 INFO:teuthology.orchestra.run.smithi178.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-10-16T14:25:14.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:14.970 INFO:teuthology.orchestra.run.smithi178.stdout:Bootstrap complete. 2023-10-16T14:25:15.011 INFO:tasks.cephadm:Fetching config... 2023-10-16T14:25:15.012 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:25:15.012 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-10-16T14:25:15.057 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-10-16T14:25:15.058 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:25:15.058 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-10-16T14:25:15.143 INFO:tasks.cephadm:Fetching mon keyring... 2023-10-16T14:25:15.143 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:25:15.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/keyring of=/dev/stdout 2023-10-16T14:25:15.254 INFO:tasks.cephadm:Fetching pub ssh key... 2023-10-16T14:25:15.255 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:25:15.255 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-10-16T14:25:15.339 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-10-16T14:25:15.339 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-16T14:25:15.481 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:25:15.502 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:15 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1829386520' entity='client.admin' 2023-10-16T14:25:15.513 DEBUG:teuthology.orchestra.run.smithi186:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-16T14:25:15.564 INFO:teuthology.orchestra.run.smithi186.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:25:15.576 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-10-16T14:25:15.621 INFO:teuthology.orchestra.run.smithi187.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDm81tpYMbmJaR9Z+EV7B0mBMx5Rk1wDq6IMM/K7TEd0R42LOkiFXNwHZuLDDm46GRe7WdHkphl/d8NKuD6ZpVTJId3dCO0bc0cAdv7r6cuhQdTi4WYgpYAZ2BIPWT5CAzrwhptFocY635VJkObT50MG15bBOE7ckR8mUr/YblOGWoMRebB3CzY7FsNaQjkoOMT3XiBzt6+CQalm4hqmTQ3jTwkjf9BTi0v95I3gfAdqIJHI1BGIQLF85ofJoHyX/qsFLneMXXD/vaxfL6XPmM/v+QU3Me1GhyMK6rtvKE2aZEoXhv5dbY0JicZIWGp5RPbKZOj71WBRjz/hXegZGj/ucFModDZdSswXAfFj0aXrP6lVG5lZvapP3/dZqBWtXCEFJZbNMou+BQqHeFzJZrLw2M1LAWEorRGtg2nACdPN6fo4Qk63pcDgnZIzAYR2l/2BwspnISx+VlwRHG5BZEBEhZul3NKqfInMMzivbuy4g01+TW8uW66j7F+43GOLmM= ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:25:15.634 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-10-16T14:25:16.400 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:18.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 -- 172.21.15.178:0/3143354631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 msgr2=0x7f0528101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:18.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 --2- 172.21.15.178:0/3143354631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f0528101690 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0518009910 tx=0x7f0518009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:18.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 -- 172.21.15.178:0/3143354631 shutdown_connections 2023-10-16T14:25:18.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 --2- 172.21.15.178:0/3143354631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f0528101690 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:18.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 -- 172.21.15.178:0/3143354631 >> 172.21.15.178:0/3143354631 conn(0x7f05280fae80 msgr2=0x7f05280fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:18.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.266+0000 7f052d9c1700 1 -- 172.21.15.178:0/3143354631 shutdown_connections 2023-10-16T14:25:18.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.267+0000 7f052d9c1700 1 -- 172.21.15.178:0/3143354631 wait complete. 2023-10-16T14:25:18.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.267+0000 7f052d9c1700 1 Processor -- start 2023-10-16T14:25:18.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f052d9c1700 1 -- start start 2023-10-16T14:25:18.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f052d9c1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:18.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f052d9c1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052819c5e0 con 0x7f05280ff260 2023-10-16T14:25:18.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f0526ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:18.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f0526ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41090/0 (socket says 172.21.15.178:41090) 2023-10-16T14:25:18.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.268+0000 7f0526ffd700 1 -- 172.21.15.178:0/946541056 learned_addr learned my addr 172.21.15.178:0/946541056 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:18.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f0526ffd700 1 -- 172.21.15.178:0/946541056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180095f0 con 0x7f05280ff260 2023-10-16T14:25:18.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f0526ffd700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0518005f50 tx=0x7f05180057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:18.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f051801d070 con 0x7f05280ff260 2023-10-16T14:25:18.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0518022470 con 0x7f05280ff260 2023-10-16T14:25:18.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f051800f5f0 con 0x7f05280ff260 2023-10-16T14:25:18.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052819c7e0 con 0x7f05280ff260 2023-10-16T14:25:18.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.269+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052819cc80 con 0x7f05280ff260 2023-10-16T14:25:18.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.271+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f0518022850 con 0x7f05280ff260 2023-10-16T14:25:18.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.271+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0528196550 con 0x7f05280ff260 2023-10-16T14:25:18.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.271+0000 7f052c9bf700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 0x7f051003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:18.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.271+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f051804cbe0 con 0x7f05280ff260 2023-10-16T14:25:18.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.272+0000 7f05267fc700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 0x7f051003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:18.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.272+0000 7f05267fc700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 0x7f051003ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f051c006fd0 tx=0x7f051c006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:18.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.280+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f051802a360 con 0x7f05280ff260 2023-10-16T14:25:18.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.494+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f0528104620 con 0x7f05280ff260 2023-10-16T14:25:18.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.496+0000 7f052c9bf700 1 -- 172.21.15.178:0/946541056 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0528104620 con 0x7f05280ff260 2023-10-16T14:25:18.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 msgr2=0x7f051003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:18.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 0x7f051003ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f051c006fd0 tx=0x7f051c006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:18.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 msgr2=0x7f052819c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:18.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0518005f50 tx=0x7f05180057c0 comp rx=0 tx=0).stop 2023-10-16T14:25:18.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 shutdown_connections 2023-10-16T14:25:18.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05100388c0 0x7f051003ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:18.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 --2- 172.21.15.178:0/946541056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05280ff260 0x7f052819c0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:18.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 >> 172.21.15.178:0/946541056 conn(0x7f05280fae80 msgr2=0x7f05280fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:18.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.499+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 shutdown_connections 2023-10-16T14:25:18.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:18.500+0000 7f052d9c1700 1 -- 172.21.15.178:0/946541056 wait complete. 2023-10-16T14:25:19.003 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-10-16T14:25:19.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-10-16T14:25:19.691 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:19.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:19 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/946541056' entity='client.admin' 2023-10-16T14:25:21.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 -- 172.21.15.178:0/3178132032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 msgr2=0x7f1d10100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:21.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 --2- 172.21.15.178:0/3178132032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10100e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009910 tx=0x7f1d00009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:21.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 -- 172.21.15.178:0/3178132032 shutdown_connections 2023-10-16T14:25:21.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 --2- 172.21.15.178:0/3178132032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10100e20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:21.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 -- 172.21.15.178:0/3178132032 >> 172.21.15.178:0/3178132032 conn(0x7f1d100fbf80 msgr2=0x7f1d100fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:21.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 -- 172.21.15.178:0/3178132032 shutdown_connections 2023-10-16T14:25:21.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.521+0000 7f1d15617700 1 -- 172.21.15.178:0/3178132032 wait complete. 2023-10-16T14:25:21.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.522+0000 7f1d15617700 1 Processor -- start 2023-10-16T14:25:21.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.522+0000 7f1d15617700 1 -- start start 2023-10-16T14:25:21.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d15617700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:21.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d15617700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d10075190 con 0x7f1d10100a00 2023-10-16T14:25:21.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:21.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d0effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54578/0 (socket says 172.21.15.178:54578) 2023-10-16T14:25:21.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d0effd700 1 -- 172.21.15.178:0/1564395443 learned_addr learned my addr 172.21.15.178:0/1564395443 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:21.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.523+0000 7f1d0effd700 1 -- 172.21.15.178:0/1564395443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000095f0 con 0x7f1d10100a00 2023-10-16T14:25:21.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1d0effd700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004f40 tx=0x7f1d000057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:21.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d0001d070 con 0x7f1d10100a00 2023-10-16T14:25:21.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d0000bc10 con 0x7f1d10100a00 2023-10-16T14:25:21.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d100732a0 con 0x7f1d10100a00 2023-10-16T14:25:21.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d00003a70 con 0x7f1d10100a00 2023-10-16T14:25:21.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.524+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d10073740 con 0x7f1d10100a00 2023-10-16T14:25:21.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.525+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d100624e0 con 0x7f1d10100a00 2023-10-16T14:25:21.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.525+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f1d00003ce0 con 0x7f1d10100a00 2023-10-16T14:25:21.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.526+0000 7f1cf7fff700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 0x7f1cf803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:21.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.526+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1d0004c6b0 con 0x7f1d10100a00 2023-10-16T14:25:21.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.526+0000 7f1d0e7fc700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 0x7f1cf803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:21.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.526+0000 7f1d0e7fc700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 0x7f1cf803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d04006fd0 tx=0x7f1d04006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:21.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.531+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1d0002abf0 con 0x7f1d10100a00 2023-10-16T14:25:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:25:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:21.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:21.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.734+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d1018f230 con 0x7f1cf8038910 2023-10-16T14:25:21.741 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.738+0000 7f1cf7fff700 1 -- 172.21.15.178:0/1564395443 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d1018f230 con 0x7f1cf8038910 2023-10-16T14:25:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 msgr2=0x7f1cf803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 0x7f1cf803add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d04006fd0 tx=0x7f1d04006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:21.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 msgr2=0x7f1d10074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004f40 tx=0x7f1d000057c0 comp rx=0 tx=0).stop 2023-10-16T14:25:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 shutdown_connections 2023-10-16T14:25:21.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf8038910 0x7f1cf803add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 --2- 172.21.15.178:0/1564395443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d10100a00 0x7f1d10074c50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.740+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 >> 172.21.15.178:0/1564395443 conn(0x7f1d100fbf80 msgr2=0x7f1d100fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:21.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.741+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 shutdown_connections 2023-10-16T14:25:21.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:21.741+0000 7f1d15617700 1 -- 172.21.15.178:0/1564395443 wait complete. 2023-10-16T14:25:22.279 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi186 2023-10-16T14:25:22.279 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:25:22.279 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.conf 2023-10-16T14:25:22.301 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:25:22.302 DEBUG:teuthology.orchestra.run.smithi186:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:22.361 INFO:tasks.cephadm:Adding host smithi186 to orchestrator... 2023-10-16T14:25:22.362 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi186 2023-10-16T14:25:23.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:23.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:22 smithi178 ceph-mon[98495]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-16T14:25:23.124 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:23.995 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:23 smithi178 ceph-mon[98495]: Updating smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:23.995 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:23 smithi178 ceph-mon[98495]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:24.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.644+0000 7f8e21a29700 1 -- 172.21.15.178:0/1929688896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 msgr2=0x7f8e1c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:24.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.644+0000 7f8e21a29700 1 --2- 172.21.15.178:0/1929688896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c106d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c009910 tx=0x7f8e0c009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:24.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.645+0000 7f8e21a29700 1 -- 172.21.15.178:0/1929688896 shutdown_connections 2023-10-16T14:25:24.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.645+0000 7f8e21a29700 1 --2- 172.21.15.178:0/1929688896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c106d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:24.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.645+0000 7f8e21a29700 1 -- 172.21.15.178:0/1929688896 >> 172.21.15.178:0/1929688896 conn(0x7f8e1c0fc4a0 msgr2=0x7f8e1c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:24.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.645+0000 7f8e21a29700 1 -- 172.21.15.178:0/1929688896 shutdown_connections 2023-10-16T14:25:24.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.645+0000 7f8e21a29700 1 -- 172.21.15.178:0/1929688896 wait complete. 2023-10-16T14:25:24.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e21a29700 1 Processor -- start 2023-10-16T14:25:24.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e21a29700 1 -- start start 2023-10-16T14:25:24.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e21a29700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:24.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e21a29700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1963a0 con 0x7f8e1c106990 2023-10-16T14:25:24.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:24.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e1affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:54596/0 (socket says 172.21.15.178:54596) 2023-10-16T14:25:24.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.646+0000 7f8e1affd700 1 -- 172.21.15.178:0/997776807 learned_addr learned my addr 172.21.15.178:0/997776807 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:24.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e1affd700 1 -- 172.21.15.178:0/997776807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c0095f0 con 0x7f8e1c106990 2023-10-16T14:25:24.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e1affd700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006010 tx=0x7f8e0c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:24.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e0c01d070 con 0x7f8e1c106990 2023-10-16T14:25:24.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e0c00bbb0 con 0x7f8e1c106990 2023-10-16T14:25:24.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e0c003930 con 0x7f8e1c106990 2023-10-16T14:25:24.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c196600 con 0x7f8e1c106990 2023-10-16T14:25:24.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.647+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c196a80 con 0x7f8e1c106990 2023-10-16T14:25:24.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.648+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f8e0c00bd20 con 0x7f8e1c106990 2023-10-16T14:25:24.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.648+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c0624e0 con 0x7f8e1c106990 2023-10-16T14:25:24.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.649+0000 7f8e20a27700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 0x7f8e0403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:24.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.649+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e0c04c6d0 con 0x7f8e1c106990 2023-10-16T14:25:24.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.649+0000 7f8e1a7fc700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 0x7f8e0403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:24.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.650+0000 7f8e1a7fc700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 0x7f8e0403add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e10006fd0 tx=0x7f8e10006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:24.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.654+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8e0c029770 con 0x7f8e1c106990 2023-10-16T14:25:24.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:24.854+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e1c10ae10 con 0x7f8e04038910 2023-10-16T14:25:25.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:24 smithi178 ceph-mon[98495]: Updating smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-16T14:25:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:24 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:24 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:24 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:26.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:25 smithi178 ceph-mon[98495]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi186", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:27.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.275+0000 7f8e20a27700 1 -- 172.21.15.178:0/997776807 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8e1c10ae10 con 0x7f8e04038910 2023-10-16T14:25:27.280 INFO:teuthology.orchestra.run.smithi178.stdout:Added host 'smithi186' with addr '172.21.15.186' 2023-10-16T14:25:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 msgr2=0x7f8e0403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:27.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 0x7f8e0403add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8e10006fd0 tx=0x7f8e10006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 msgr2=0x7f8e1c195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c006010 tx=0x7f8e0c004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 shutdown_connections 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8e04038910 0x7f8e0403add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 --2- 172.21.15.178:0/997776807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8e1c106990 0x7f8e1c195e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:27.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 >> 172.21.15.178:0/997776807 conn(0x7f8e1c0fc4a0 msgr2=0x7f8e1c108df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 shutdown_connections 2023-10-16T14:25:27.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:27.278+0000 7f8e21a29700 1 -- 172.21.15.178:0/997776807 wait complete. 2023-10-16T14:25:27.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:26 smithi178 ceph-mon[98495]: Deploying cephadm binary to smithi186 2023-10-16T14:25:27.710 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-16T14:25:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:27 smithi178 ceph-mon[98495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:28.444 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:28 smithi178 ceph-mon[98495]: Added host smithi186 2023-10-16T14:25:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 -- 172.21.15.178:0/3099800154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 msgr2=0x7fc7d8106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:29.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/3099800154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8106d70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009910 tx=0x7fc7c8009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 -- 172.21.15.178:0/3099800154 shutdown_connections 2023-10-16T14:25:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/3099800154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8106d70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:29.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 -- 172.21.15.178:0/3099800154 >> 172.21.15.178:0/3099800154 conn(0x7fc7d80fc4a0 msgr2=0x7fc7d80fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 -- 172.21.15.178:0/3099800154 shutdown_connections 2023-10-16T14:25:29.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.925+0000 7fc7dde5d700 1 -- 172.21.15.178:0/3099800154 wait complete. 2023-10-16T14:25:29.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.926+0000 7fc7dde5d700 1 Processor -- start 2023-10-16T14:25:29.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.926+0000 7fc7dde5d700 1 -- start start 2023-10-16T14:25:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7dde5d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:29.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7dde5d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8198590 con 0x7fc7d8106990 2023-10-16T14:25:29.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7d77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:29.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7d77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36630/0 (socket says 172.21.15.178:36630) 2023-10-16T14:25:29.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7d77fe700 1 -- 172.21.15.178:0/1537506179 learned_addr learned my addr 172.21.15.178:0/1537506179 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:29.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.927+0000 7fc7d77fe700 1 -- 172.21.15.178:0/1537506179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80095f0 con 0x7fc7d8106990 2023-10-16T14:25:29.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7d77fe700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8006010 tx=0x7fc7c8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:29.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c801d070 con 0x7fc7d8106990 2023-10-16T14:25:29.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7c800bbb0 con 0x7fc7d8106990 2023-10-16T14:25:29.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7c8003930 con 0x7fc7d8106990 2023-10-16T14:25:29.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d81987f0 con 0x7fc7d8106990 2023-10-16T14:25:29.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.928+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d8198c70 con 0x7fc7d8106990 2023-10-16T14:25:29.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.929+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d80624e0 con 0x7fc7d8106990 2023-10-16T14:25:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.929+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fc7c800bd20 con 0x7fc7d8106990 2023-10-16T14:25:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.929+0000 7fc7d4ff9700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 0x7fc7c003add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.930+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc7c804c6f0 con 0x7fc7d8106990 2023-10-16T14:25:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.930+0000 7fc7d6ffd700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 0x7fc7c003add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:29.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.930+0000 7fc7d6ffd700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 0x7fc7c003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc006fd0 tx=0x7fc7cc006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:29.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:29.935+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc7c8029360 con 0x7fc7d8106990 2023-10-16T14:25:30.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:29 smithi178 ceph-mon[98495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:30.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.138+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7d80008d0 con 0x7fc7c0038910 2023-10-16T14:25:30.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.139+0000 7fc7d4ff9700 1 -- 172.21.15.178:0/1537506179 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fc7d80008d0 con 0x7fc7c0038910 2023-10-16T14:25:30.143 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:30.144 INFO:teuthology.orchestra.run.smithi178.stdout:[{"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}] 2023-10-16T14:25:30.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 msgr2=0x7fc7c003add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:30.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 0x7fc7c003add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc006fd0 tx=0x7fc7cc006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:30.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 msgr2=0x7fc7d8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:30.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8006010 tx=0x7fc7c8004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:30.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 shutdown_connections 2023-10-16T14:25:30.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc7c0038910 0x7fc7c003add0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:30.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 --2- 172.21.15.178:0/1537506179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7d8106990 0x7fc7d8198050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:30.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 >> 172.21.15.178:0/1537506179 conn(0x7fc7d80fc4a0 msgr2=0x7fc7d8108d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:30.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 shutdown_connections 2023-10-16T14:25:30.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:30.142+0000 7fc7dde5d700 1 -- 172.21.15.178:0/1537506179 wait complete. 2023-10-16T14:25:30.622 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi187 2023-10-16T14:25:30.622 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:25:30.623 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.conf 2023-10-16T14:25:30.643 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:25:30.643 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:30.703 INFO:tasks.cephadm:Adding host smithi187 to orchestrator... 2023-10-16T14:25:30.703 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch host add smithi187 2023-10-16T14:25:31.435 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:31.975 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:31 smithi178 ceph-mon[98495]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:25:31.975 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:31 smithi178 ceph-mon[98495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:32.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 -- 172.21.15.178:0/998991720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 msgr2=0x7f57a0106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 --2- 172.21.15.178:0/998991720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a0106d70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5790009910 tx=0x7f5790009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 -- 172.21.15.178:0/998991720 shutdown_connections 2023-10-16T14:25:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 --2- 172.21.15.178:0/998991720 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a0106d70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 -- 172.21.15.178:0/998991720 >> 172.21.15.178:0/998991720 conn(0x7f57a00fc4a0 msgr2=0x7f57a00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:32.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 -- 172.21.15.178:0/998991720 shutdown_connections 2023-10-16T14:25:32.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.894+0000 7f57a6263700 1 -- 172.21.15.178:0/998991720 wait complete. 2023-10-16T14:25:32.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.895+0000 7f57a6263700 1 Processor -- start 2023-10-16T14:25:32.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.895+0000 7f57a6263700 1 -- start start 2023-10-16T14:25:32.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f57a6263700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:32.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f57a6263700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a01985f0 con 0x7f57a0106990 2023-10-16T14:25:32.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f579ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:32.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f579ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36646/0 (socket says 172.21.15.178:36646) 2023-10-16T14:25:32.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f579ffff700 1 -- 172.21.15.178:0/3418080 learned_addr learned my addr 172.21.15.178:0/3418080 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f579ffff700 1 -- 172.21.15.178:0/3418080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900095f0 con 0x7f57a0106990 2023-10-16T14:25:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.896+0000 7f579ffff700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5790006010 tx=0x7f5790004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.897+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f579001d070 con 0x7f57a0106990 2023-10-16T14:25:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.897+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f579000bbb0 con 0x7f57a0106990 2023-10-16T14:25:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.897+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a0198850 con 0x7f57a0106990 2023-10-16T14:25:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.897+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5790003930 con 0x7f57a0106990 2023-10-16T14:25:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.897+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a0198cd0 con 0x7f57a0106990 2023-10-16T14:25:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.898+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a00624e0 con 0x7f57a0106990 2023-10-16T14:25:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.898+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f579000bd20 con 0x7f57a0106990 2023-10-16T14:25:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.899+0000 7f579dffb700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 0x7f578c03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.899+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f579004c6d0 con 0x7f57a0106990 2023-10-16T14:25:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.899+0000 7f5797fff700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 0x7f578c03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.900+0000 7f5797fff700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 0x7f578c03add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5788006fd0 tx=0x7f5788006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:32.903+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5790029770 con 0x7f57a0106990 2023-10-16T14:25:33.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:33.113+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a010ada0 con 0x7f578c038910 2023-10-16T14:25:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:25:34.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:34.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:34.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-16T14:25:34.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:34.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:34.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:35.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:34 smithi178 ceph-mon[98495]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:35.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:34 smithi178 ceph-mon[98495]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:35.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:34 smithi178 ceph-mon[98495]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-16T14:25:35.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:34 smithi178 ceph-mon[98495]: Deploying cephadm binary to smithi187 2023-10-16T14:25:35.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.623+0000 7f579dffb700 1 -- 172.21.15.178:0/3418080 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f57a010ada0 con 0x7f578c038910 2023-10-16T14:25:35.627 INFO:teuthology.orchestra.run.smithi178.stdout:Added host 'smithi187' with addr '172.21.15.187' 2023-10-16T14:25:35.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 msgr2=0x7f578c03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:35.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 0x7f578c03add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5788006fd0 tx=0x7f5788006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:35.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 msgr2=0x7f57a01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:35.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5790006010 tx=0x7f5790004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:35.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 shutdown_connections 2023-10-16T14:25:35.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f578c038910 0x7f578c03add0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:35.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 --2- 172.21.15.178:0/3418080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57a0106990 0x7f57a01980b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:35.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.625+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 >> 172.21.15.178:0/3418080 conn(0x7f57a00fc4a0 msgr2=0x7f57a0108d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:35.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.626+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 shutdown_connections 2023-10-16T14:25:35.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:35.626+0000 7f57a6263700 1 -- 172.21.15.178:0/3418080 wait complete. 2023-10-16T14:25:36.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:35 smithi178 ceph-mon[98495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:36.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:36.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:36.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch host ls --format=json 2023-10-16T14:25:36.849 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:37.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:36 smithi178 ceph-mon[98495]: Added host smithi187 2023-10-16T14:25:37.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:37.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:37 smithi178 ceph-mon[98495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:37.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:38.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 -- 172.21.15.178:0/712356452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 msgr2=0x7fe448106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:38.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 --2- 172.21.15.178:0/712356452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe448106d70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe444009910 tx=0x7fe444009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:38.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 -- 172.21.15.178:0/712356452 shutdown_connections 2023-10-16T14:25:38.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 --2- 172.21.15.178:0/712356452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe448106d70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:38.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 -- 172.21.15.178:0/712356452 >> 172.21.15.178:0/712356452 conn(0x7fe4480fc4a0 msgr2=0x7fe4480fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:38.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 -- 172.21.15.178:0/712356452 shutdown_connections 2023-10-16T14:25:38.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.332+0000 7fe44ff71700 1 -- 172.21.15.178:0/712356452 wait complete. 2023-10-16T14:25:38.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.333+0000 7fe44ff71700 1 Processor -- start 2023-10-16T14:25:38.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44ff71700 1 -- start start 2023-10-16T14:25:38.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44ff71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:38.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44ff71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481985f0 con 0x7fe448106990 2023-10-16T14:25:38.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44dd0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:38.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44dd0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36668/0 (socket says 172.21.15.178:36668) 2023-10-16T14:25:38.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.334+0000 7fe44dd0d700 1 -- 172.21.15.178:0/3598249298 learned_addr learned my addr 172.21.15.178:0/3598249298 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:38.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe44dd0d700 1 -- 172.21.15.178:0/3598249298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440095f0 con 0x7fe448106990 2023-10-16T14:25:38.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe44dd0d700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe444006010 tx=0x7fe444004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:38.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe44401d070 con 0x7fe448106990 2023-10-16T14:25:38.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448198850 con 0x7fe448106990 2023-10-16T14:25:38.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe44400bbb0 con 0x7fe448106990 2023-10-16T14:25:38.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe444003930 con 0x7fe448106990 2023-10-16T14:25:38.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.335+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe448198cd0 con 0x7fe448106990 2023-10-16T14:25:38.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.336+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fe444003b70 con 0x7fe448106990 2023-10-16T14:25:38.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.336+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4480624e0 con 0x7fe448106990 2023-10-16T14:25:38.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.337+0000 7fe43effd700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 0x7fe43403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:38.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.337+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe44404c6f0 con 0x7fe448106990 2023-10-16T14:25:38.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.337+0000 7fe44d50c700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 0x7fe43403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:38.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.338+0000 7fe44d50c700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 0x7fe43403ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe438006fd0 tx=0x7fe438006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:38.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.342+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe444029540 con 0x7fe448106990 2023-10-16T14:25:38.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.548+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe44810ae10 con 0x7fe4340388c0 2023-10-16T14:25:38.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.549+0000 7fe43effd700 1 -- 172.21.15.178:0/3598249298 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fe44810ae10 con 0x7fe4340388c0 2023-10-16T14:25:38.553 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:25:38.553 INFO:teuthology.orchestra.run.smithi178.stdout:[{"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}, {"addr": "172.21.15.186", "hostname": "smithi186", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}] 2023-10-16T14:25:38.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 msgr2=0x7fe43403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:38.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 0x7fe43403ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe438006fd0 tx=0x7fe438006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:38.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 msgr2=0x7fe4481980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:38.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe444006010 tx=0x7fe444004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:38.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 shutdown_connections 2023-10-16T14:25:38.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe4340388c0 0x7fe43403ad80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:38.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.551+0000 7fe44ff71700 1 --2- 172.21.15.178:0/3598249298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe448106990 0x7fe4481980b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:38.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.552+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 >> 172.21.15.178:0/3598249298 conn(0x7fe4480fc4a0 msgr2=0x7fe448108d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:38.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.552+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 shutdown_connections 2023-10-16T14:25:38.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:38.552+0000 7fe44ff71700 1 -- 172.21.15.178:0/3598249298 wait complete. 2023-10-16T14:25:39.028 INFO:tasks.cephadm:Setting crush tunables to default 2023-10-16T14:25:39.028 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd crush tunables default 2023-10-16T14:25:39.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:39.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:39 smithi178 ceph-mon[98495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:39.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:39 smithi178 ceph-mon[98495]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:25:39.762 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:25:41.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.218+0000 7f3113164700 1 -- 172.21.15.178:0/3078013879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 msgr2=0x7f310c0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.218+0000 7f3113164700 1 --2- 172.21.15.178:0/3078013879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c0736c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3100009910 tx=0x7f3100009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.219+0000 7f3113164700 1 -- 172.21.15.178:0/3078013879 shutdown_connections 2023-10-16T14:25:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.219+0000 7f3113164700 1 --2- 172.21.15.178:0/3078013879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c0736c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:41.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.219+0000 7f3113164700 1 -- 172.21.15.178:0/3078013879 >> 172.21.15.178:0/3078013879 conn(0x7f310c0fc4a0 msgr2=0x7f310c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.219+0000 7f3113164700 1 -- 172.21.15.178:0/3078013879 shutdown_connections 2023-10-16T14:25:41.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.219+0000 7f3113164700 1 -- 172.21.15.178:0/3078013879 wait complete. 2023-10-16T14:25:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3113164700 1 Processor -- start 2023-10-16T14:25:41.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3113164700 1 -- start start 2023-10-16T14:25:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3113164700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3113164700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c198600 con 0x7f310c0732e0 2023-10-16T14:25:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3110f00700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3110f00700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:50382/0 (socket says 172.21.15.178:50382) 2023-10-16T14:25:41.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.220+0000 7f3110f00700 1 -- 172.21.15.178:0/3067678656 learned_addr learned my addr 172.21.15.178:0/3067678656 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:25:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3110f00700 1 -- 172.21.15.178:0/3067678656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000095f0 con 0x7f310c0732e0 2023-10-16T14:25:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3110f00700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3100006010 tx=0x7f3100004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f310001d070 con 0x7f310c0732e0 2023-10-16T14:25:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31000044a0 con 0x7f310c0732e0 2023-10-16T14:25:41.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c198800 con 0x7f310c0732e0 2023-10-16T14:25:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3100003a70 con 0x7f310c0732e0 2023-10-16T14:25:41.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.221+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c198c20 con 0x7f310c0732e0 2023-10-16T14:25:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.222+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f3100003c60 con 0x7f310c0732e0 2023-10-16T14:25:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.222+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c0624e0 con 0x7f310c0732e0 2023-10-16T14:25:41.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.223+0000 7f3109ffb700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 0x7f30fc03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:41.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.223+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f310004c790 con 0x7f310c0732e0 2023-10-16T14:25:41.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.223+0000 7f310bfff700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 0x7f30fc03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:41.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.224+0000 7f310bfff700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 0x7f30fc03add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:41.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.228+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3100029540 con 0x7f310c0732e0 2023-10-16T14:25:41.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.435+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f310c10d000 con 0x7f310c0732e0 2023-10-16T14:25:41.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.991+0000 7f3109ffb700 1 -- 172.21.15.178:0/3067678656 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3100026020 con 0x7f310c0732e0 2023-10-16T14:25:41.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.993+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 msgr2=0x7f30fc03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:41.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.993+0000 7f3113164700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 0x7f30fc03add0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f30f8006fd0 tx=0x7f30f8006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:41.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.993+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 msgr2=0x7f310c1980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:41.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.993+0000 7f3113164700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3100006010 tx=0x7f3100004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:41.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 shutdown_connections 2023-10-16T14:25:41.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30fc038910 0x7f30fc03add0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:41.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 --2- 172.21.15.178:0/3067678656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f310c0732e0 0x7f310c1980c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:41.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 >> 172.21.15.178:0/3067678656 conn(0x7f310c0fc4a0 msgr2=0x7f310c1074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:41.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 shutdown_connections 2023-10-16T14:25:41.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:25:41.994+0000 7f3113164700 1 -- 172.21.15.178:0/3067678656 wait complete. 2023-10-16T14:25:42.000 INFO:teuthology.orchestra.run.smithi178.stderr:adjusted tunables profile to default 2023-10-16T14:25:42.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:42.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:42.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:42.019 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:42.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:42.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:25:42.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:42.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:42.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:41 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3067678656' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-16T14:25:42.442 INFO:tasks.cephadm:Adding mon.a on smithi178 2023-10-16T14:25:42.442 INFO:tasks.cephadm:Adding mon.b on smithi186 2023-10-16T14:25:42.443 INFO:tasks.cephadm:Adding mon.c on smithi187 2023-10-16T14:25:42.443 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch apply mon '3;smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c' 2023-10-16T14:25:43.029 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:43.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: Updating smithi187:/etc/ceph/ceph.conf 2023-10-16T14:25:43.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:43.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:43.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-16T14:25:43.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:43.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:43.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:43.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3067678656' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-16T14:25:43.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:42 smithi178 ceph-mon[98495]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:25:44.257 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 -- 172.21.15.187:0/216162050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 msgr2=0x7f0528105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:44.257 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 --2- 172.21.15.187:0/216162050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528105c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0524009910 tx=0x7f0524009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:44.257 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 -- 172.21.15.187:0/216162050 shutdown_connections 2023-10-16T14:25:44.258 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 --2- 172.21.15.187:0/216162050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528105c40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:44.258 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 -- 172.21.15.187:0/216162050 >> 172.21.15.187:0/216162050 conn(0x7f05280fb390 msgr2=0x7f05280fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:44.258 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 -- 172.21.15.187:0/216162050 shutdown_connections 2023-10-16T14:25:44.258 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.253+0000 7f052fbb1700 1 -- 172.21.15.187:0/216162050 wait complete. 2023-10-16T14:25:44.260 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052fbb1700 1 Processor -- start 2023-10-16T14:25:44.260 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052fbb1700 1 -- start start 2023-10-16T14:25:44.261 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052fbb1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:44.261 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052fbb1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05281963c0 con 0x7f0528105860 2023-10-16T14:25:44.261 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052d94d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:44.261 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052d94d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.187:53412/0 (socket says 172.21.15.187:53412) 2023-10-16T14:25:44.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.254+0000 7f052d94d700 1 -- 172.21.15.187:0/1462401070 learned_addr learned my addr 172.21.15.187:0/1462401070 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:25:44.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f052d94d700 1 -- 172.21.15.187:0/1462401070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05240095f0 con 0x7f0528105860 2023-10-16T14:25:44.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f052d94d700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0524006010 tx=0x7f0524004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:44.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f052401d070 con 0x7f0528105860 2023-10-16T14:25:44.262 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0528196620 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f052400bbb0 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0524003930 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.255+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528196aa0 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.256+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05280624e0 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.256+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f0524003b70 con 0x7f0528105860 2023-10-16T14:25:44.263 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.256+0000 7f051effd700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 0x7f051403ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:44.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.256+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f052404c630 con 0x7f0528105860 2023-10-16T14:25:44.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.257+0000 7f052d14c700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 0x7f051403ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:44.264 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.257+0000 7f052d14c700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 0x7f051403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0518006fd0 tx=0x7f0518006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:44.265 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.262+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0524029540 con 0x7f0528105860 2023-10-16T14:25:44.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:43 smithi178 ceph-mon[98495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:44.464 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.461+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f0528109da0 con 0x7f05140388c0 2023-10-16T14:25:44.472 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.469+0000 7f051effd700 1 -- 172.21.15.187:0/1462401070 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0528109da0 con 0x7f05140388c0 2023-10-16T14:25:44.472 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mon update... 2023-10-16T14:25:44.474 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.470+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 msgr2=0x7f051403ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:44.474 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.470+0000 7f052fbb1700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 0x7f051403ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0518006fd0 tx=0x7f0518006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:44.474 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.470+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 msgr2=0x7f0528195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:44.475 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.470+0000 7f052fbb1700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0524006010 tx=0x7f0524004ff0 comp rx=0 tx=0).stop 2023-10-16T14:25:44.475 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 shutdown_connections 2023-10-16T14:25:44.475 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f05140388c0 0x7f051403ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:44.475 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 --2- 172.21.15.187:0/1462401070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0528105860 0x7f0528195e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:44.475 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 >> 172.21.15.187:0/1462401070 conn(0x7f05280fb390 msgr2=0x7f0528103370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:44.476 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 shutdown_connections 2023-10-16T14:25:44.476 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:44.471+0000 7f052fbb1700 1 -- 172.21.15.187:0/1462401070 wait complete. 2023-10-16T14:25:44.911 DEBUG:teuthology.orchestra.run.smithi186:mon.b> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.b.service 2023-10-16T14:25:44.914 DEBUG:teuthology.orchestra.run.smithi187:mon.c> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.c.service 2023-10-16T14:25:44.916 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-16T14:25:44.917 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-16T14:25:44.948 INFO:journalctl@ceph.mon.c.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:25:44.950 INFO:journalctl@ceph.mon.b.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:25:45.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:45.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: Saving service mon spec with placement smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c;count:3 2023-10-16T14:25:45.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:45.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:45.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:45.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:45.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:45.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:25:45.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:45.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: Deploying daemon mon.c on smithi187 2023-10-16T14:25:45.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:45 smithi178 ceph-mon[98495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:45.921 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:47.567 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.563+0000 7ff71a4d1700 1 -- 172.21.15.187:0/965330522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 msgr2=0x7ff7140ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:47.567 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.563+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/965330522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff7140ffb10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff704009910 tx=0x7ff704009c20 comp rx=0 tx=0).stop 2023-10-16T14:25:47.568 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 -- 172.21.15.187:0/965330522 shutdown_connections 2023-10-16T14:25:47.568 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/965330522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff7140ffb10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:47.568 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 -- 172.21.15.187:0/965330522 >> 172.21.15.187:0/965330522 conn(0x7ff7140fb390 msgr2=0x7ff7140fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:47.568 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 -- 172.21.15.187:0/965330522 shutdown_connections 2023-10-16T14:25:47.569 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 -- 172.21.15.187:0/965330522 wait complete. 2023-10-16T14:25:47.570 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 Processor -- start 2023-10-16T14:25:47.570 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.564+0000 7ff71a4d1700 1 -- start start 2023-10-16T14:25:47.570 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff71a4d1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:47.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff71a4d1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff714198540 con 0x7ff7140ff730 2023-10-16T14:25:47.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff713fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:47.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff713fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.187:53418/0 (socket says 172.21.15.187:53418) 2023-10-16T14:25:47.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff713fff700 1 -- 172.21.15.187:0/3364844204 learned_addr learned my addr 172.21.15.187:0/3364844204 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:25:47.571 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.565+0000 7ff713fff700 1 -- 172.21.15.187:0/3364844204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7040095f0 con 0x7ff7140ff730 2023-10-16T14:25:47.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff713fff700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff704004f40 tx=0x7ff704005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:47.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff70401d070 con 0x7ff7140ff730 2023-10-16T14:25:47.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff704004370 con 0x7ff7140ff730 2023-10-16T14:25:47.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7040039b0 con 0x7ff7140ff730 2023-10-16T14:25:47.572 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff714198740 con 0x7ff7140ff730 2023-10-16T14:25:47.573 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.566+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff714198b60 con 0x7ff7140ff730 2023-10-16T14:25:47.573 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.567+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7ff704003c20 con 0x7ff7140ff730 2023-10-16T14:25:47.574 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.567+0000 7ff7117fa700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 0x7ff6fc03add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:47.574 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.567+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff70404c750 con 0x7ff7140ff730 2023-10-16T14:25:47.574 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.568+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7140624e0 con 0x7ff7140ff730 2023-10-16T14:25:47.574 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.568+0000 7ff7137fe700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 0x7ff6fc03add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:47.575 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.568+0000 7ff7137fe700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 0x7ff6fc03add0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff708006fd0 tx=0x7ff708006e40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:47.577 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.573+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff704026070 con 0x7ff7140ff730 2023-10-16T14:25:47.865 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.862+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff714199360 con 0x7ff7140ff730 2023-10-16T14:25:47.866 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.863+0000 7ff7117fa700 1 -- 172.21.15.187:0/3364844204 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7ff70402b440 con 0x7ff7140ff730 2023-10-16T14:25:47.866 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:25:47.867 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":1,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","modified":"2023-10-16T14:24:04.380004Z","created":"2023-10-16T14:24:04.380004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-10-16T14:25:47.868 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 msgr2=0x7ff6fc03add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:47.868 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 0x7ff6fc03add0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff708006fd0 tx=0x7ff708006e40 comp rx=0 tx=0).stop 2023-10-16T14:25:47.869 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 msgr2=0x7ff714198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:47.869 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff704004f40 tx=0x7ff704005020 comp rx=0 tx=0).stop 2023-10-16T14:25:47.869 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 shutdown_connections 2023-10-16T14:25:47.869 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6fc038910 0x7ff6fc03add0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:47.869 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 --2- 172.21.15.187:0/3364844204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7140ff730 0x7ff714198000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:47.870 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 >> 172.21.15.187:0/3364844204 conn(0x7ff7140fb390 msgr2=0x7ff7140fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:47.870 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 shutdown_connections 2023-10-16T14:25:47.870 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:47.865+0000 7ff71a4d1700 1 -- 172.21.15.187:0/3364844204 wait complete. 2023-10-16T14:25:47.871 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 1 2023-10-16T14:25:48.000 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:47 smithi187 ceph-mon[135297]: expand_channel_meta expanded map: {default=debug} 2023-10-16T14:25:48.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:47 smithi178 ceph-mon[98495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:48.255 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:47 smithi187 systemd[1]: Started Ceph mon.c for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:25:49.302 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-16T14:25:49.302 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-16T14:25:49.868 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:25:53.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:25:53.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: mon.a calling monitor election 2023-10-16T14:25:53.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: mon.c calling monitor election 2023-10-16T14:25:53.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:25:53.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: monmap e2: 2 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:25:53.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: fsmap 2023-10-16T14:25:53.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:25:53.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: mgrmap e13: x(active, since 46s) 2023-10-16T14:25:53.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: overall HEALTH_OK 2023-10-16T14:25:53.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:53.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:53.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:25:53.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: mon.a calling monitor election 2023-10-16T14:25:53.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: mon.c calling monitor election 2023-10-16T14:25:53.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:53.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:53.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:25:53.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: monmap e2: 2 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: fsmap 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: mgrmap e13: x(active, since 46s) 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: overall HEALTH_OK 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:53.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:54.197 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 -- 172.21.15.187:0/3760705745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab00035c0 msgr2=0x7fdab0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:54.197 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 --2- 172.21.15.187:0/3760705745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab00035c0 0x7fdab0005a50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdac4077150 tx=0x7fdab400feb0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.198 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 -- 172.21.15.187:0/3760705745 shutdown_connections 2023-10-16T14:25:54.198 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 --2- 172.21.15.187:0/3760705745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdab00035c0 0x7fdab0005a50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.198 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 -- 172.21.15.187:0/3760705745 >> 172.21.15.187:0/3760705745 conn(0x7fdac40fb050 msgr2=0x7fdac40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:54.198 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 -- 172.21.15.187:0/3760705745 shutdown_connections 2023-10-16T14:25:54.198 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.193+0000 7fdacc50c700 1 -- 172.21.15.187:0/3760705745 wait complete. 2023-10-16T14:25:54.199 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 Processor -- start 2023-10-16T14:25:54.199 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 -- start start 2023-10-16T14:25:54.201 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:54.201 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 0x7fdac419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:54.201 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4196aa0 con 0x7fdac4196510 2023-10-16T14:25:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdacc50c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4196c10 con 0x7fdab00035c0 2023-10-16T14:25:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33492/0 (socket says 172.21.15.187:33492) 2023-10-16T14:25:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 -- 172.21.15.187:0/1856760549 learned_addr learned my addr 172.21.15.187:0/1856760549 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:25:54.202 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 -- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 msgr2=0x7fdac4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-16T14:25:54.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 -- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 msgr2=0x7fdac4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-16T14:25:54.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T14:25:54.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdac9aa7700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 0x7fdac419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:54.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.194+0000 7fdaca2a8700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:25:54.203 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdac9aa7700 1 -- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 msgr2=0x7fdac4195fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:25:54.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdac9aa7700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdac9aa7700 1 -- 172.21.15.187:0/1856760549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab400f900 con 0x7fdac4196510 2023-10-16T14:25:54.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdac9aa7700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 0x7fdac419a980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdac000b4d0 tx=0x7fdac000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:54.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdac0011630 con 0x7fdac4196510 2023-10-16T14:25:54.204 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdac0011c50 con 0x7fdac4196510 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac419af20 con 0x7fdac4196510 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fdac0010d90 con 0x7fdac4196510 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.195+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac419b470 con 0x7fdac4196510 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.196+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fdac001a400 con 0x7fdac4196510 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.197+0000 7fdabb7fe700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 0x7fdaa803abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:25:54.205 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.197+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac419b7a0 con 0x7fdac4196510 2023-10-16T14:25:54.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.197+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdac004af00 con 0x7fdac4196510 2023-10-16T14:25:54.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.197+0000 7fdaca2a8700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 0x7fdaa803abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:25:54.206 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.198+0000 7fdaca2a8700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 0x7fdaa803abb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011510 tx=0x7fdab4005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:25:54.207 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.203+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdac00193b0 con 0x7fdac4196510 2023-10-16T14:25:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:25:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-mon[98495]: Deploying daemon mon.b on smithi186 2023-10-16T14:25:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:54 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:54 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:25:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:54 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:54 smithi187 ceph-mon[135297]: Deploying daemon mon.b on smithi186 2023-10-16T14:25:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:54 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:54.476 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.472+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdac40624e0 con 0x7fdac4196510 2023-10-16T14:25:54.477 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.473+0000 7fdabb7fe700 1 -- 172.21.15.187:0/1856760549 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7fdac00172c0 con 0x7fdac4196510 2023-10-16T14:25:54.477 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:25:54.477 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":2,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","modified":"2023-10-16T14:25:47.941152Z","created":"2023-10-16T14:24:04.380004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-16T14:25:54.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 msgr2=0x7fdaa803abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:54.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 0x7fdaa803abb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdab4011510 tx=0x7fdab4005960 comp rx=0 tx=0).stop 2023-10-16T14:25:54.479 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 msgr2=0x7fdac419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:25:54.480 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 0x7fdac419a980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdac000b4d0 tx=0x7fdac000b890 comp rx=0 tx=0).stop 2023-10-16T14:25:54.480 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 shutdown_connections 2023-10-16T14:25:54.480 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdaa80386f0 0x7fdaa803abb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.480 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdab00035c0 0x7fdac4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.480 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 --2- 172.21.15.187:0/1856760549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac4196510 0x7fdac419a980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:25:54.481 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 >> 172.21.15.187:0/1856760549 conn(0x7fdac40fb050 msgr2=0x7fdac40ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:25:54.481 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.475+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 shutdown_connections 2023-10-16T14:25:54.481 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:25:54.476+0000 7fdacc50c700 1 -- 172.21.15.187:0/1856760549 wait complete. 2023-10-16T14:25:54.481 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 2 2023-10-16T14:25:55.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:55 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/1856760549' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:25:55.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:25:55 smithi187 ceph-mon[135297]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:55.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:55 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/1856760549' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:25:55.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:25:55 smithi178 ceph-mon[98495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:55.306 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:25:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:25:54.941+0000 7f0644c9e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T14:25:55.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:55 smithi186 systemd[1]: Starting Ceph mon.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:25:55.929 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-10-16T14:25:55.930 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph mon dump -f json 2023-10-16T14:25:56.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:55 smithi186 podman[134852]: 2023-10-16T14:25:56.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: set uid:gid to 167:167 (ceph:ceph) 2023-10-16T14:25:56.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev), process ceph-mon, pid 7 2023-10-16T14:25:56.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pidfile_write: ignore empty --pid-file 2023-10-16T14:25:56.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 bash[134852]: c242daa6e1cd563b83ac0344dbdf6d6969e3c869c4a2e553ee289c2f076d4906 2023-10-16T14:25:56.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: load: jerasure load: lrc 2023-10-16T14:25:56.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: RocksDB version: 7.9.2 2023-10-16T14:25:56.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Git sha 0 2023-10-16T14:25:56.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Compile date 2023-10-13 13:48:46 2023-10-16T14:25:56.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: DB SUMMARY 2023-10-16T14:25:56.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: DB Session ID: TD8FXMCJVXGYU0GJ47T8 2023-10-16T14:25:56.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: CURRENT file: CURRENT 2023-10-16T14:25:56.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: IDENTITY file: IDENTITY 2023-10-16T14:25:56.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-10-16T14:25:56.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-10-16T14:25:56.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-10-16T14:25:56.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.error_if_exists: 0 2023-10-16T14:25:56.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.create_if_missing: 0 2023-10-16T14:25:56.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.paranoid_checks: 1 2023-10-16T14:25:56.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.flush_verify_memtable_count: 1 2023-10-16T14:25:56.331 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-10-16T14:25:56.331 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-10-16T14:25:56.331 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.env: 0x560a245c2da0 2023-10-16T14:25:56.331 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.fs: PosixFileSystem 2023-10-16T14:25:56.331 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.info_log: 0x560a255f15e0 2023-10-16T14:25:56.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_file_opening_threads: 16 2023-10-16T14:25:56.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.statistics: (nil) 2023-10-16T14:25:56.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.use_fsync: 0 2023-10-16T14:25:56.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_log_file_size: 0 2023-10-16T14:25:56.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-10-16T14:25:56.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.log_file_time_to_roll: 0 2023-10-16T14:25:56.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.keep_log_file_num: 1000 2023-10-16T14:25:56.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.recycle_log_file_num: 0 2023-10-16T14:25:56.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_fallocate: 1 2023-10-16T14:25:56.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_mmap_reads: 0 2023-10-16T14:25:56.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_mmap_writes: 0 2023-10-16T14:25:56.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.use_direct_reads: 0 2023-10-16T14:25:56.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-10-16T14:25:56.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.create_missing_column_families: 0 2023-10-16T14:25:56.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.db_log_dir: 2023-10-16T14:25:56.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.wal_dir: 2023-10-16T14:25:56.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.table_cache_numshardbits: 6 2023-10-16T14:25:56.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.WAL_ttl_seconds: 0 2023-10-16T14:25:56.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.WAL_size_limit_MB: 0 2023-10-16T14:25:56.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.is_fd_close_on_exec: 1 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.advise_random_on_open: 1 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.db_write_buffer_size: 0 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.write_buffer_manager: 0x560a255a9360 2023-10-16T14:25:56.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-10-16T14:25:56.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-10-16T14:25:56.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.use_adaptive_mutex: 0 2023-10-16T14:25:56.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.rate_limiter: (nil) 2023-10-16T14:25:56.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-10-16T14:25:56.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.wal_recovery_mode: 2 2023-10-16T14:25:56.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enable_thread_tracking: 0 2023-10-16T14:25:56.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enable_pipelined_write: 0 2023-10-16T14:25:56.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.unordered_write: 0 2023-10-16T14:25:56.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-10-16T14:25:56.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-10-16T14:25:56.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-10-16T14:25:56.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-10-16T14:25:56.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.row_cache: None 2023-10-16T14:25:56.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.wal_filter: None 2023-10-16T14:25:56.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-10-16T14:25:56.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_ingest_behind: 0 2023-10-16T14:25:56.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.two_write_queues: 0 2023-10-16T14:25:56.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.manual_wal_flush: 0 2023-10-16T14:25:56.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.wal_compression: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.atomic_flush: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.persist_stats_to_disk: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.write_dbid_to_manifest: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.log_readahead_size: 0 2023-10-16T14:25:56.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-10-16T14:25:56.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.best_efforts_recovery: 0 2023-10-16T14:25:56.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-10-16T14:25:56.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-10-16T14:25:56.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.allow_data_in_errors: 0 2023-10-16T14:25:56.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.db_host_id: __hostname__ 2023-10-16T14:25:56.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enforce_single_del_contracts: true 2023-10-16T14:25:56.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_background_jobs: 2 2023-10-16T14:25:56.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_background_compactions: -1 2023-10-16T14:25:56.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_subcompactions: 1 2023-10-16T14:25:56.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-10-16T14:25:56.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-10-16T14:25:56.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.delayed_write_rate : 16777216 2023-10-16T14:25:56.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_total_wal_size: 0 2023-10-16T14:25:56.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-10-16T14:25:56.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.stats_dump_period_sec: 600 2023-10-16T14:25:56.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.stats_persist_period_sec: 600 2023-10-16T14:25:56.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-10-16T14:25:56.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_open_files: -1 2023-10-16T14:25:56.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bytes_per_sync: 0 2023-10-16T14:25:56.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.wal_bytes_per_sync: 0 2023-10-16T14:25:56.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.strict_bytes_per_sync: 0 2023-10-16T14:25:56.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_readahead_size: 0 2023-10-16T14:25:56.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_background_flushes: -1 2023-10-16T14:25:56.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Compression algorithms supported: 2023-10-16T14:25:56.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-10-16T14:25:56.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kZSTD supported: 0 2023-10-16T14:25:56.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kXpressCompression supported: 0 2023-10-16T14:25:56.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kLZ4HCCompression supported: 1 2023-10-16T14:25:56.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kZlibCompression supported: 1 2023-10-16T14:25:56.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kSnappyCompression supported: 1 2023-10-16T14:25:56.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kLZ4Compression supported: 1 2023-10-16T14:25:56.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: kBZip2Compression supported: 0 2023-10-16T14:25:56.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Fast CRC32 supported: Supported on x86 2023-10-16T14:25:56.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: DMutex implementation: pthread_mutex_t 2023-10-16T14:25:56.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-10-16T14:25:56.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-10-16T14:25:56.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-10-16T14:25:56.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.merge_operator: 2023-10-16T14:25:56.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_filter: None 2023-10-16T14:25:56.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_filter_factory: None 2023-10-16T14:25:56.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.sst_partitioner_factory: None 2023-10-16T14:25:56.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.memtable_factory: SkipListFactory 2023-10-16T14:25:56.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.table_factory: BlockBasedTable 2023-10-16T14:25:56.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560a255f16e0) 2023-10-16T14:25:56.351 INFO:journalctl@ceph.mon.b.smithi186.stdout: cache_index_and_filter_blocks: 1 2023-10-16T14:25:56.352 INFO:journalctl@ceph.mon.b.smithi186.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-10-16T14:25:56.352 INFO:journalctl@ceph.mon.b.smithi186.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-10-16T14:25:56.352 INFO:journalctl@ceph.mon.b.smithi186.stdout: pin_top_level_index_and_filter: 1 2023-10-16T14:25:56.352 INFO:journalctl@ceph.mon.b.smithi186.stdout: index_type: 0 2023-10-16T14:25:56.352 INFO:journalctl@ceph.mon.b.smithi186.stdout: data_block_index_type: 0 2023-10-16T14:25:56.353 INFO:journalctl@ceph.mon.b.smithi186.stdout: index_shortening: 1 2023-10-16T14:25:56.353 INFO:journalctl@ceph.mon.b.smithi186.stdout: data_block_hash_table_util_ratio: 0.750000 2023-10-16T14:25:56.353 INFO:journalctl@ceph.mon.b.smithi186.stdout: checksum: 4 2023-10-16T14:25:56.353 INFO:journalctl@ceph.mon.b.smithi186.stdout: no_block_cache: 0 2023-10-16T14:25:56.353 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_cache: 0x560a256471f0 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_cache_name: BinnedLRUCache 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_cache_options: 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: capacity : 536870912 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: num_shard_bits : 4 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: strict_capacity_limit : 0 2023-10-16T14:25:56.354 INFO:journalctl@ceph.mon.b.smithi186.stdout: high_pri_pool_ratio: 0.000 2023-10-16T14:25:56.355 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_cache_compressed: (nil) 2023-10-16T14:25:56.355 INFO:journalctl@ceph.mon.b.smithi186.stdout: persistent_cache: (nil) 2023-10-16T14:25:56.355 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_size: 4096 2023-10-16T14:25:56.355 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_size_deviation: 10 2023-10-16T14:25:56.355 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_restart_interval: 16 2023-10-16T14:25:56.356 INFO:journalctl@ceph.mon.b.smithi186.stdout: index_block_restart_interval: 1 2023-10-16T14:25:56.356 INFO:journalctl@ceph.mon.b.smithi186.stdout: metadata_block_size: 4096 2023-10-16T14:25:56.356 INFO:journalctl@ceph.mon.b.smithi186.stdout: partition_filters: 0 2023-10-16T14:25:56.356 INFO:journalctl@ceph.mon.b.smithi186.stdout: use_delta_encoding: 1 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: filter_policy: bloomfilter 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: whole_key_filtering: 1 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: verify_compression: 0 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: read_amp_bytes_per_bit: 0 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: format_version: 5 2023-10-16T14:25:56.357 INFO:journalctl@ceph.mon.b.smithi186.stdout: enable_index_compression: 1 2023-10-16T14:25:56.358 INFO:journalctl@ceph.mon.b.smithi186.stdout: block_align: 0 2023-10-16T14:25:56.358 INFO:journalctl@ceph.mon.b.smithi186.stdout: max_auto_readahead_size: 262144 2023-10-16T14:25:56.358 INFO:journalctl@ceph.mon.b.smithi186.stdout: prepopulate_block_cache: 0 2023-10-16T14:25:56.358 INFO:journalctl@ceph.mon.b.smithi186.stdout: initial_auto_readahead_size: 8192 2023-10-16T14:25:56.359 INFO:journalctl@ceph.mon.b.smithi186.stdout: num_file_reads_for_auto_readahead: 2 2023-10-16T14:25:56.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.write_buffer_size: 33554432 2023-10-16T14:25:56.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_write_buffer_number: 2 2023-10-16T14:25:56.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression: NoCompression 2023-10-16T14:25:56.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression: Disabled 2023-10-16T14:25:56.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.prefix_extractor: nullptr 2023-10-16T14:25:56.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-10-16T14:25:56.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.num_levels: 7 2023-10-16T14:25:56.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-10-16T14:25:56.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-10-16T14:25:56.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-10-16T14:25:56.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-10-16T14:25:56.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-10-16T14:25:56.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-10-16T14:25:56.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-10-16T14:25:56.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-10-16T14:25:56.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-10-16T14:25:56.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-10-16T14:25:56.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-10-16T14:25:56.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-10-16T14:25:56.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.window_bits: -14 2023-10-16T14:25:56.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.level: 32767 2023-10-16T14:25:56.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.strategy: 0 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.enabled: false 2023-10-16T14:25:56.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-10-16T14:25:56.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-10-16T14:25:56.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-10-16T14:25:56.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-10-16T14:25:56.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.target_file_size_base: 67108864 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.target_file_size_multiplier: 1 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-10-16T14:25:56.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-10-16T14:25:56.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-10-16T14:25:56.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-10-16T14:25:56.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-10-16T14:25:56.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-10-16T14:25:56.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-10-16T14:25:56.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-10-16T14:25:56.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-10-16T14:25:56.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-10-16T14:25:56.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.arena_block_size: 1048576 2023-10-16T14:25:56.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.disable_auto_compactions: 0 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-10-16T14:25:56.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-10-16T14:25:56.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-10-16T14:25:56.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-10-16T14:25:56.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-10-16T14:25:56.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-10-16T14:25:56.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-10-16T14:25:56.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-10-16T14:25:56.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.table_properties_collectors: 2023-10-16T14:25:56.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.inplace_update_support: 0 2023-10-16T14:25:56.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.inplace_update_num_locks: 10000 2023-10-16T14:25:56.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-10-16T14:25:56.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-10-16T14:25:56.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.memtable_huge_page_size: 0 2023-10-16T14:25:56.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.bloom_locality: 0 2023-10-16T14:25:56.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.max_successive_merges: 0 2023-10-16T14:25:56.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.optimize_filters_for_hits: 0 2023-10-16T14:25:56.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.paranoid_file_checks: 0 2023-10-16T14:25:56.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.force_consistency_checks: 1 2023-10-16T14:25:56.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.report_bg_io_stats: 0 2023-10-16T14:25:56.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.ttl: 2592000 2023-10-16T14:25:56.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.periodic_compaction_seconds: 0 2023-10-16T14:25:56.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-10-16T14:25:56.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-10-16T14:25:56.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enable_blob_files: false 2023-10-16T14:25:56.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.min_blob_size: 0 2023-10-16T14:25:56.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_file_size: 268435456 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_compression_type: NoCompression 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.enable_blob_garbage_collection: false 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-10-16T14:25:56.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.blob_file_starting_level: 0 2023-10-16T14:25:56.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-10-16T14:25:56.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-10-16T14:25:56.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-10-16T14:25:56.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b8a55255-9bdf-419e-8061-dfa148d852fc 2023-10-16T14:25:56.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466356054703, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-10-16T14:25:56.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-10-16T14:25:56.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466356055125, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1697466356, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b8a55255-9bdf-419e-8061-dfa148d852fc", "db_session_id": "TD8FXMCJVXGYU0GJ47T8", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-10-16T14:25:56.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1697466356055190, "job": 1, "event": "recovery_finished"} 2023-10-16T14:25:56.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-10-16T14:25:56.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-10-16T14:25:56.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560a256e4000 2023-10-16T14:25:56.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: DB pointer 0x560a256d0000 2023-10-16T14:25:56.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-10-16T14:25:56.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: ** DB Stats ** 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-16T14:25:56.380 INFO:journalctl@ceph.mon.b.smithi186.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-10-16T14:25:56.381 INFO:journalctl@ceph.mon.b.smithi186.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-10-16T14:25:56.381 INFO:journalctl@ceph.mon.b.smithi186.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-10-16T14:25:56.381 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.381 INFO:journalctl@ceph.mon.b.smithi186.stdout: ** Compaction Stats [default] ** 2023-10-16T14:25:56.381 INFO:journalctl@ceph.mon.b.smithi186.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-16T14:25:56.382 INFO:journalctl@ceph.mon.b.smithi186.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-10-16T14:25:56.382 INFO:journalctl@ceph.mon.b.smithi186.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-16T14:25:56.382 INFO:journalctl@ceph.mon.b.smithi186.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-16T14:25:56.382 INFO:journalctl@ceph.mon.b.smithi186.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-16T14:25:56.383 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.383 INFO:journalctl@ceph.mon.b.smithi186.stdout: ** Compaction Stats [default] ** 2023-10-16T14:25:56.383 INFO:journalctl@ceph.mon.b.smithi186.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-10-16T14:25:56.383 INFO:journalctl@ceph.mon.b.smithi186.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-10-16T14:25:56.383 INFO:journalctl@ceph.mon.b.smithi186.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-10-16T14:25:56.384 INFO:journalctl@ceph.mon.b.smithi186.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-10-16T14:25:56.385 INFO:journalctl@ceph.mon.b.smithi186.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-10-16T14:25:56.385 INFO:journalctl@ceph.mon.b.smithi186.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-10-16T14:25:56.385 INFO:journalctl@ceph.mon.b.smithi186.stdout: AddFile(Keys): cumulative 0, interval 0 2023-10-16T14:25:56.385 INFO:journalctl@ceph.mon.b.smithi186.stdout: Cumulative compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-16T14:25:56.385 INFO:journalctl@ceph.mon.b.smithi186.stdout: Interval compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-10-16T14:25:56.386 INFO:journalctl@ceph.mon.b.smithi186.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-10-16T14:25:56.386 INFO:journalctl@ceph.mon.b.smithi186.stdout: Block cache BinnedLRUCache@0x560a256471f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-10-16T14:25:56.386 INFO:journalctl@ceph.mon.b.smithi186.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-10-16T14:25:56.386 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.386 INFO:journalctl@ceph.mon.b.smithi186.stdout: ** File Read Latency Histogram By Level [default] ** 2023-10-16T14:25:56.387 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-10-16T14:25:56.387 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: using public_addr v2:172.21.15.186:0/0 -> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] 2023-10-16T14:25:56.387 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: starting mon.b rank -1 at public addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] at bind addrs [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:25:56.387 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(???) e0 preinit fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:25:56.387 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).mds e1 new map 2023-10-16T14:25:56.388 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).mds e1 print_map 2023-10-16T14:25:56.388 INFO:journalctl@ceph.mon.b.smithi186.stdout: e1 2023-10-16T14:25:56.388 INFO:journalctl@ceph.mon.b.smithi186.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-10-16T14:25:56.388 INFO:journalctl@ceph.mon.b.smithi186.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-16T14:25:56.388 INFO:journalctl@ceph.mon.b.smithi186.stdout: legacy client fscid: -1 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout: 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout: No filesystems configured 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-10-16T14:25:56.389 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-10-16T14:25:56.390 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-10-16T14:25:56.390 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-10-16T14:25:56.390 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-10-16T14:25:56.390 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:25:56.390 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:25:56.391 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-10-16T14:25:56.391 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.391 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.391 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.391 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi186", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-16T14:25:56.392 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:56.393 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.393 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.393 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.393 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:56.393 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi186:/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:56.394 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-16T14:25:56.394 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Deploying cephadm binary to smithi187 2023-10-16T14:25:56.394 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.394 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.394 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Added host smithi187 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.395 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.396 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:25:56.396 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.396 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.396 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.396 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3067678656' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-10-16T14:25:56.397 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi187:/etc/ceph/ceph.conf 2023-10-16T14:25:56.398 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:25:56.398 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:25:56.398 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.client.admin.keyring 2023-10-16T14:25:56.398 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.398 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.399 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.399 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3067678656' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-10-16T14:25:56.399 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:25:56.399 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.399 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:25:56.400 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Saving service mon spec with placement smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c;count:3 2023-10-16T14:25:56.400 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.400 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:25:56.400 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:56.400 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:25:56.401 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.401 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:25:56.401 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:56.401 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Deploying daemon mon.c on smithi187 2023-10-16T14:25:56.401 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.402 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.402 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:25:56.402 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.a calling monitor election 2023-10-16T14:25:56.402 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.402 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.403 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.403 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.403 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.c calling monitor election 2023-10-16T14:25:56.403 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.403 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.404 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.404 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.404 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.404 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:25:56.404 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: monmap e2: 2 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:25:56.405 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: fsmap 2023-10-16T14:25:56.405 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:25:56.405 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mgrmap e13: x(active, since 46s) 2023-10-16T14:25:56.405 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: overall HEALTH_OK 2023-10-16T14:25:56.406 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.406 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.406 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:25:56.406 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:25:56.406 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:25:56.407 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: Deploying daemon mon.b on smithi186 2023-10-16T14:25:56.407 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:25:56.407 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/1856760549' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:25:56.407 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:25:56.407 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expand map: {default=false} 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta from 'false' to 'false' 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expanded map: {default=false} 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expand map: {default=info} 2023-10-16T14:25:56.408 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta from 'info' to 'info' 2023-10-16T14:25:56.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expanded map: {default=info} 2023-10-16T14:25:56.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expand map: {default=daemon} 2023-10-16T14:25:56.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta from 'daemon' to 'daemon' 2023-10-16T14:25:56.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expanded map: {default=daemon} 2023-10-16T14:25:56.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expand map: {default=debug} 2023-10-16T14:25:56.410 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta from 'debug' to 'debug' 2023-10-16T14:25:56.410 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 ceph-mon[134892]: expand_channel_meta expanded map: {default=debug} 2023-10-16T14:25:56.410 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:25:56 smithi186 systemd[1]: Started Ceph mon.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:25:56.488 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:26:01.290 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 -- 172.21.15.187:0/1057409041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f00035c0 msgr2=0x7f68f0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:01.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 --2- 172.21.15.187:0/1057409041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f00035c0 0x7f68f0005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f69081009c0 tx=0x7f6900009e80 comp rx=0 tx=0).stop 2023-10-16T14:26:01.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 -- 172.21.15.187:0/1057409041 shutdown_connections 2023-10-16T14:26:01.291 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 --2- 172.21.15.187:0/1057409041 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68f00035c0 0x7f68f0005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.292 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 -- 172.21.15.187:0/1057409041 >> 172.21.15.187:0/1057409041 conn(0x7f69080fc580 msgr2=0x7f69080fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:01.292 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 -- 172.21.15.187:0/1057409041 shutdown_connections 2023-10-16T14:26:01.292 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.287+0000 7f690f5da700 1 -- 172.21.15.187:0/1057409041 wait complete. 2023-10-16T14:26:01.296 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 Processor -- start 2023-10-16T14:26:01.296 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 -- start start 2023-10-16T14:26:01.296 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:01.297 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:01.297 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 0x7f690819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:01.297 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908196c30 con 0x7f690819aef0 2023-10-16T14:26:01.298 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69081a2c60 con 0x7f68f00035c0 2023-10-16T14:26:01.298 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690f5da700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69081a2da0 con 0x7f6908196540 2023-10-16T14:26:01.298 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690cb75700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:01.298 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690cb75700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33518/0 (socket says 172.21.15.187:33518) 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.289+0000 7f690cb75700 1 -- 172.21.15.187:0/1728912361 learned_addr learned my addr 172.21.15.187:0/1728912361 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690cb75700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 msgr2=0x7f690819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690cb75700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 msgr2=0x7f690819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690cb75700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690d376700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:01.299 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690cb75700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:26:01.300 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 0x7f690819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:01.300 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690d376700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 msgr2=0x7f6908196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-10-16T14:26:01.300 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690d376700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 msgr2=0x7f6908196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-10-16T14:26:01.300 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690d376700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T14:26:01.300 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690d376700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T14:26:01.301 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 msgr2=0x7f690819a9b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:26:01.301 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.301 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 msgr2=0x7f6908196000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:26:01.301 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.301 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 -- 172.21.15.187:0/1728912361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000098d0 con 0x7f690819aef0 2023-10-16T14:26:01.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690db77700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 0x7f690819b3a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006fd0 tx=0x7f68f800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:01.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8011830 con 0x7f690819aef0 2023-10-16T14:26:01.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f68f8011e50 con 0x7f690819aef0 2023-10-16T14:26:01.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69081a3080 con 0x7f690819aef0 2023-10-16T14:26:01.302 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8010e80 con 0x7f690819aef0 2023-10-16T14:26:01.303 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.290+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69081a3580 con 0x7f690819aef0 2023-10-16T14:26:01.303 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.292+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f68f801a460 con 0x7f690819aef0 2023-10-16T14:26:01.303 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.292+0000 7f69067fc700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 0x7f68f403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:01.303 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.292+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f68f804af20 con 0x7f690819aef0 2023-10-16T14:26:01.303 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.292+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69080624e0 con 0x7f690819aef0 2023-10-16T14:26:01.304 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.292+0000 7f690d376700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 0x7f68f403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:01.304 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.293+0000 7f690d376700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 0x7f68f403b200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6900009160 tx=0x7f6900023040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:01.304 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.298+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f68f80178b0 con 0x7f690819aef0 2023-10-16T14:26:01.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:26:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: mon.a calling monitor election 2023-10-16T14:26:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: mon.c calling monitor election 2023-10-16T14:26:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:26:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:26:01.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:01.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: fsmap 2023-10-16T14:26:01.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:01.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: mgrmap e13: x(active, since 54s) 2023-10-16T14:26:01.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-16T14:26:01.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-16T14:26:01.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-16T14:26:01.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: mon.b (rank 2) addr [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] is down (out of quorum) 2023-10-16T14:26:01.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.432 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:26:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: mon.a calling monitor election 2023-10-16T14:26:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: mon.c calling monitor election 2023-10-16T14:26:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:26:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:01.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: fsmap 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: mgrmap e13: x(active, since 54s) 2023-10-16T14:26:01.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: mon.b (rank 2) addr [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] is down (out of quorum) 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.558 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:01.587 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.585+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f69081087d0 con 0x7f690819aef0 2023-10-16T14:26:01.588 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.585+0000 7f69067fc700 1 -- 172.21.15.187:0/1728912361 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1278 (secure 0 0 0) 0x7f68f80172c0 con 0x7f690819aef0 2023-10-16T14:26:01.588 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-10-16T14:26:01.589 INFO:teuthology.orchestra.run.smithi187.stdout:{"epoch":3,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","modified":"2023-10-16T14:25:56.080999Z","created":"2023-10-16T14:24:04.380004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:3300","nonce":0},{"type":"v1","addr":"172.21.15.186:6789","nonce":0}]},"addr":"172.21.15.186:6789/0","public_addr":"172.21.15.186:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-10-16T14:26:01.590 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.587+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 msgr2=0x7f68f403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:01.590 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.587+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 0x7f68f403b200 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6900009160 tx=0x7f6900023040 comp rx=0 tx=0).stop 2023-10-16T14:26:01.591 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.587+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 msgr2=0x7f690819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:01.591 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 0x7f690819b3a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006fd0 tx=0x7f68f800b7e0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.591 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 shutdown_connections 2023-10-16T14:26:01.591 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68f4038d40 0x7f68f403b200 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.591 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f690819aef0 0x7f690819b3a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.592 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68f00035c0 0x7f6908196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.592 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 --2- 172.21.15.187:0/1728912361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6908196540 0x7f690819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:01.592 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 >> 172.21.15.187:0/1728912361 conn(0x7f69080fc580 msgr2=0x7f6908102630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:01.592 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 shutdown_connections 2023-10-16T14:26:01.593 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:01.588+0000 7f690f5da700 1 -- 172.21.15.187:0/1728912361 wait complete. 2023-10-16T14:26:01.593 INFO:teuthology.orchestra.run.smithi187.stderr:dumped monmap epoch 3 2023-10-16T14:26:02.058 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-10-16T14:26:02.058 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph config generate-minimal-conf 2023-10-16T14:26:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/1728912361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:26:02.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:02.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:02.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:02.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/1728912361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:26:02.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:02.799 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:26:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:26:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.a calling monitor election 2023-10-16T14:26:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.c calling monitor election 2023-10-16T14:26:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:26:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:03.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:03.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-10-16T14:26:03.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:03.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: fsmap 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mgrmap e13: x(active, since 54s) 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-10-16T14:26:03.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.b (rank 2) addr [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] is down (out of quorum) 2023-10-16T14:26:03.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:03.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:03.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:03.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:03.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:03.582 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:03.582 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/1728912361' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-10-16T14:26:03.582 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mon.b calling monitor election 2023-10-16T14:26:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-16T14:26:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-16T14:26:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi187:/etc/ceph/ceph.conf 2023-10-16T14:26:04.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mon.b calling monitor election 2023-10-16T14:26:04.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mon.c calling monitor election 2023-10-16T14:26:04.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mon.a calling monitor election 2023-10-16T14:26:04.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-16T14:26:04.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:04.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: fsmap 2023-10-16T14:26:04.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:04.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: mgrmap e13: x(active, since 56s) 2023-10-16T14:26:04.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-16T14:26:04.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:26:04.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: overall HEALTH_OK 2023-10-16T14:26:04.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:03 smithi178 ceph-mon[98495]: Updating smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.b calling monitor election 2023-10-16T14:26:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-16T14:26:04.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-16T14:26:04.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi187:/etc/ceph/ceph.conf 2023-10-16T14:26:04.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.b calling monitor election 2023-10-16T14:26:04.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.c calling monitor election 2023-10-16T14:26:04.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.a calling monitor election 2023-10-16T14:26:04.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-16T14:26:04.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:04.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: fsmap 2023-10-16T14:26:04.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:04.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: mgrmap e13: x(active, since 56s) 2023-10-16T14:26:04.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-16T14:26:04.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:26:04.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: overall HEALTH_OK 2023-10-16T14:26:04.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:03 smithi186 ceph-mon[134892]: Updating smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 -- 172.21.15.178:0/1167005162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff264106990 msgr2=0x7ff264106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 --2- 172.21.15.178:0/1167005162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff264106990 0x7ff264106d70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff260009910 tx=0x7ff260009c20 comp rx=0 tx=0).stop 2023-10-16T14:26:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 -- 172.21.15.178:0/1167005162 shutdown_connections 2023-10-16T14:26:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 --2- 172.21.15.178:0/1167005162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff264106990 0x7ff264106d70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 -- 172.21.15.178:0/1167005162 >> 172.21.15.178:0/1167005162 conn(0x7ff2640fc4a0 msgr2=0x7ff2640fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 -- 172.21.15.178:0/1167005162 shutdown_connections 2023-10-16T14:26:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.405+0000 7ff26c54a700 1 -- 172.21.15.178:0/1167005162 wait complete. 2023-10-16T14:26:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.406+0000 7ff26c54a700 1 Processor -- start 2023-10-16T14:26:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.406+0000 7ff26c54a700 1 -- start start 2023-10-16T14:26:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff264106990 0x7ff2640731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2641a2180 0x7ff2641a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2641a4bc0 con 0x7ff2640736f0 2023-10-16T14:26:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2641a4d30 con 0x7ff264106990 2023-10-16T14:26:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26c54a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2641a4ef0 con 0x7ff2641a2180 2023-10-16T14:26:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff269ae5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff269ae5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34292/0 (socket says 172.21.15.178:34292) 2023-10-16T14:26:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff269ae5700 1 -- 172.21.15.178:0/3464766787 learned_addr learned my addr 172.21.15.178:0/3464766787 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:26:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26a2e6700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff264106990 0x7ff2640731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:04.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.407+0000 7ff26aae7700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2641a2180 0x7ff2641a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:04.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 -- 172.21.15.178:0/3464766787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2641a2180 msgr2=0x7ff2641a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:04.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2641a2180 0x7ff2641a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 -- 172.21.15.178:0/3464766787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff264106990 msgr2=0x7ff2640731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff264106990 0x7ff2640731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 -- 172.21.15.178:0/3464766787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600095f0 con 0x7ff2640736f0 2023-10-16T14:26:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff269ae5700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff25400b780 tx=0x7ff25400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254004230 con 0x7ff2640736f0 2023-10-16T14:26:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2641a51d0 con 0x7ff2640736f0 2023-10-16T14:26:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff254004390 con 0x7ff2640736f0 2023-10-16T14:26:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254011b90 con 0x7ff2640736f0 2023-10-16T14:26:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.408+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2641a5720 con 0x7ff2640736f0 2023-10-16T14:26:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.410+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7ff254011cf0 con 0x7ff2640736f0 2023-10-16T14:26:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.410+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2640624e0 con 0x7ff2640736f0 2023-10-16T14:26:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.410+0000 7ff25b7fe700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 0x7ff25003b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.410+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff25404a1e0 con 0x7ff2640736f0 2023-10-16T14:26:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.410+0000 7ff26a2e6700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 0x7ff25003b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.411+0000 7ff26a2e6700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 0x7ff25003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff260009fd0 tx=0x7ff2600058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.415+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff254003bf0 con 0x7ff2640736f0 2023-10-16T14:26:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mon.b calling monitor election 2023-10-16T14:26:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi178:/etc/ceph/ceph.conf 2023-10-16T14:26:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi186:/etc/ceph/ceph.conf 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi187:/etc/ceph/ceph.conf 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mon.b calling monitor election 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mon.c calling monitor election 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mon.a calling monitor election 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-10-16T14:26:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: monmap e3: 3 mons at {a=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],b=[v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0],c=[v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0]} removed_ranks: {} 2023-10-16T14:26:04.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: fsmap 2023-10-16T14:26:04.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: osdmap e4: 0 total, 0 up, 0 in 2023-10-16T14:26:04.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: mgrmap e13: x(active, since 56s) 2023-10-16T14:26:04.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-10-16T14:26:04.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:26:04.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: overall HEALTH_OK 2023-10-16T14:26:04.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:04.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:03 smithi187 ceph-mon[135297]: Updating smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/config/ceph.conf 2023-10-16T14:26:04.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.630+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff26410ad90 con 0x7ff2640736f0 2023-10-16T14:26:04.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.631+0000 7ff25b7fe700 1 -- 172.21.15.178:0/3464766787 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7ff254016030 con 0x7ff2640736f0 2023-10-16T14:26:04.634 INFO:teuthology.orchestra.run.smithi178.stdout:# minimal ceph.conf for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:26:04.635 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2023-10-16T14:26:04.635 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:26:04.635 INFO:teuthology.orchestra.run.smithi178.stdout: mon_host = [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2023-10-16T14:26:04.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 msgr2=0x7ff25003b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:04.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 0x7ff25003b250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff260009fd0 tx=0x7ff2600058e0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 msgr2=0x7ff264073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff25400b780 tx=0x7ff25400ba90 comp rx=0 tx=0).stop 2023-10-16T14:26:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 shutdown_connections 2023-10-16T14:26:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff250038d90 0x7ff25003b250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2641a2180 0x7ff2641a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff264106990 0x7ff2640731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 --2- 172.21.15.178:0/3464766787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2640736f0 0x7ff264073b70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:04.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.633+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 >> 172.21.15.178:0/3464766787 conn(0x7ff2640fc4a0 msgr2=0x7ff264102490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:04.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.634+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 shutdown_connections 2023-10-16T14:26:04.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:04.634+0000 7ff26c54a700 1 -- 172.21.15.178:0/3464766787 wait complete. 2023-10-16T14:26:05.086 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-10-16T14:26:05.086 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:26:05.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2023-10-16T14:26:05.159 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:26:05.159 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:26:05.287 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:26:05.287 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.conf 2023-10-16T14:26:05.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:05.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: Reconfiguring mon.a (unknown last config time)... 2023-10-16T14:26:05.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:05.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:05.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: Reconfiguring daemon mon.a on smithi178 2023-10-16T14:26:05.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:05.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3464766787' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.319 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.319 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.319 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.319 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:05.320 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.320 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.320 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.320 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.320 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: Reconfiguring mon.a (unknown last config time)... 2023-10-16T14:26:05.321 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:05.321 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:05.321 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.321 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: Reconfiguring daemon mon.a on smithi178 2023-10-16T14:26:05.321 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:05.322 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3464766787' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.324 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:26:05.324 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:26:05.394 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:26:05.394 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.conf 2023-10-16T14:26:05.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:26:05.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:05.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: Reconfiguring mon.a (unknown last config time)... 2023-10-16T14:26:05.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:05.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:05.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: Reconfiguring daemon mon.a on smithi178 2023-10-16T14:26:05.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:05.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3464766787' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:05.431 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:26:05.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-10-16T14:26:05.501 INFO:tasks.cephadm:Adding mgr.x on smithi178 2023-10-16T14:26:05.502 INFO:tasks.cephadm:Adding mgr.y on smithi186 2023-10-16T14:26:05.502 INFO:tasks.cephadm:Adding mgr.z on smithi187 2023-10-16T14:26:05.503 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch apply mgr '3;smithi178=x;smithi186=y;smithi187=z' 2023-10-16T14:26:06.099 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:26:06.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:06.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:06.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: Reconfiguring mon.b (monmap changed)... 2023-10-16T14:26:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:06 smithi187 ceph-mon[135297]: Reconfiguring daemon mon.b on smithi186 2023-10-16T14:26:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: Reconfiguring mon.b (monmap changed)... 2023-10-16T14:26:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:06 smithi186 ceph-mon[134892]: Reconfiguring daemon mon.b on smithi186 2023-10-16T14:26:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: Reconfiguring mon.b (monmap changed)... 2023-10-16T14:26:07.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:07.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:07.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:07.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:06 smithi178 ceph-mon[98495]: Reconfiguring daemon mon.b on smithi186 2023-10-16T14:26:07.405 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 -- 172.21.15.187:0/894969837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24101630 msgr2=0x7fdf24103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:07.405 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/894969837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24101630 0x7fdf24103a20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009910 tx=0x7fdf20009c20 comp rx=0 tx=0).stop 2023-10-16T14:26:07.406 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 -- 172.21.15.187:0/894969837 shutdown_connections 2023-10-16T14:26:07.406 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/894969837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24101630 0x7fdf24103a20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.406 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 -- 172.21.15.187:0/894969837 >> 172.21.15.187:0/894969837 conn(0x7fdf240fafc0 msgr2=0x7fdf240fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:07.406 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 -- 172.21.15.187:0/894969837 shutdown_connections 2023-10-16T14:26:07.406 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.401+0000 7fdf2ba58700 1 -- 172.21.15.187:0/894969837 wait complete. 2023-10-16T14:26:07.407 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 Processor -- start 2023-10-16T14:26:07.407 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 -- start start 2023-10-16T14:26:07.407 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf24101630 0x7fdf24195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:07.407 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:07.407 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24196700 0x7fdf2419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:07.408 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419e030 con 0x7fdf24196700 2023-10-16T14:26:07.408 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419e1a0 con 0x7fdf24101630 2023-10-16T14:26:07.408 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf2ba58700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419e360 con 0x7fdf24196280 2023-10-16T14:26:07.408 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:07.408 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33528/0 (socket says 172.21.15.187:33528) 2023-10-16T14:26:07.409 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 -- 172.21.15.187:0/2616391392 learned_addr learned my addr 172.21.15.187:0/2616391392 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:26:07.409 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf297f4700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf24101630 0x7fdf24195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:07.409 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 -- 172.21.15.187:0/2616391392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf24101630 msgr2=0x7fdf24195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:07.409 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf24101630 0x7fdf24195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.409 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 -- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24196700 msgr2=0x7fdf2419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:07.410 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf29ff5700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24196700 0x7fdf2419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:07.410 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24196700 0x7fdf2419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.410 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 -- 172.21.15.187:0/2616391392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf200095f0 con 0x7fdf24196280 2023-10-16T14:26:07.410 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.402+0000 7fdf28ff3700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400b780 tx=0x7fdf1400bb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:07.410 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.403+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004230 con 0x7fdf24196280 2023-10-16T14:26:07.411 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.403+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf14004390 con 0x7fdf24196280 2023-10-16T14:26:07.411 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.403+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2419e640 con 0x7fdf24196280 2023-10-16T14:26:07.411 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.403+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14011b30 con 0x7fdf24196280 2023-10-16T14:26:07.411 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.403+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2419ebc0 con 0x7fdf24196280 2023-10-16T14:26:07.412 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.404+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2418f5c0 con 0x7fdf24196280 2023-10-16T14:26:07.412 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.404+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fdf140036a0 con 0x7fdf24196280 2023-10-16T14:26:07.413 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.404+0000 7fdf1a7fc700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 0x7fdf1003b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:07.413 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.404+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdf1404aec0 con 0x7fdf24196280 2023-10-16T14:26:07.413 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.404+0000 7fdf297f4700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 0x7fdf1003b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:07.413 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.405+0000 7fdf297f4700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 0x7fdf1003b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005130 tx=0x7fdf2000b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:07.413 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.409+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdf14004500 con 0x7fdf24196280 2023-10-16T14:26:07.623 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.621+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi178=x;smithi186=y;smithi187=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf2402d2c0 con 0x7fdf10038d40 2023-10-16T14:26:07.630 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.628+0000 7fdf1a7fc700 1 -- 172.21.15.187:0/2616391392 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdf2402d2c0 con 0x7fdf10038d40 2023-10-16T14:26:07.631 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mgr update... 2023-10-16T14:26:07.633 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 msgr2=0x7fdf1003b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:07.633 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 0x7fdf1003b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdf20005130 tx=0x7fdf2000b540 comp rx=0 tx=0).stop 2023-10-16T14:26:07.633 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 msgr2=0x7fdf2419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:07.633 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400b780 tx=0x7fdf1400bb40 comp rx=0 tx=0).stop 2023-10-16T14:26:07.634 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 shutdown_connections 2023-10-16T14:26:07.634 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdf10038d40 0x7fdf1003b200 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.634 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf24196700 0x7fdf2419d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.634 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdf24101630 0x7fdf24195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.634 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 --2- 172.21.15.187:0/2616391392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf24196280 0x7fdf2419b2b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:07.635 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 >> 172.21.15.187:0/2616391392 conn(0x7fdf240fafc0 msgr2=0x7fdf240fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:07.635 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 shutdown_connections 2023-10-16T14:26:07.635 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:26:07.630+0000 7fdf2ba58700 1 -- 172.21.15.187:0/2616391392 wait complete. 2023-10-16T14:26:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.243 DEBUG:teuthology.orchestra.run.smithi186:mgr.y> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.y.service 2023-10-16T14:26:08.246 DEBUG:teuthology.orchestra.run.smithi187:mgr.z> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.z.service 2023-10-16T14:26:08.249 INFO:tasks.cephadm:Deploying OSDs... 2023-10-16T14:26:08.249 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:26:08.250 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T14:26:08.274 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:08.275 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.275 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.275 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:08.275 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:08.276 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:08.276 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.278 INFO:journalctl@ceph.mgr.z.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:26:08.281 INFO:journalctl@ceph.mgr.y.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:26:08.298 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T14:26:08.298 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2023-10-16T14:26:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-10-16T14:26:08.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-10-16T14:26:08.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:08.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T14:26:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:08.351 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-16T14:26:08.351 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:08.351 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:08.351 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-16 14:25:19.126103055 +0000 2023-10-16T14:26:08.351 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-16 14:19:27.587160502 +0000 2023-10-16T14:26:08.352 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-16 14:19:27.587160502 +0000 2023-10-16T14:26:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2023-10-16 14:19:27.565160871 +0000 2023-10-16T14:26:08.352 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T14:26:08.459 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-16T14:26:08.459 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-16T14:26:08.459 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000215385 s, 2.4 MB/s 2023-10-16T14:26:08.461 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T14:26:08.508 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2023-10-16T14:26:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T14:26:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:08.594 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-16 14:25:19.197101852 +0000 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-16 14:19:28.423146483 +0000 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-16 14:19:28.423146483 +0000 2023-10-16T14:26:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2023-10-16 14:19:28.400146868 +0000 2023-10-16T14:26:08.596 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T14:26:08.699 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-16T14:26:08.700 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-16T14:26:08.700 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000171767 s, 3.0 MB/s 2023-10-16T14:26:08.702 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T14:26:08.748 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2023-10-16T14:26:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T14:26:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-16 14:25:19.265100699 +0000 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-16 14:19:29.223133067 +0000 2023-10-16T14:26:08.833 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-16 14:19:29.223133067 +0000 2023-10-16T14:26:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2023-10-16 14:19:29.199133469 +0000 2023-10-16T14:26:08.834 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T14:26:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-16T14:26:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-16T14:26:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.0001707 s, 3.0 MB/s 2023-10-16T14:26:08.941 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T14:26:08.986 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2023-10-16T14:26:09.071 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T14:26:09.071 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:09.071 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-16T14:26:09.072 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:09.072 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:09.072 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-16 14:25:19.342099393 +0000 2023-10-16T14:26:09.072 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-16 14:19:30.027119584 +0000 2023-10-16T14:26:09.072 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-16 14:19:30.027119584 +0000 2023-10-16T14:26:09.073 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2023-10-16 14:19:30.005119953 +0000 2023-10-16T14:26:09.073 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T14:26:09.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: Reconfiguring mon.c (monmap changed)... 2023-10-16T14:26:09.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: Reconfiguring daemon mon.c on smithi187 2023-10-16T14:26:09.155 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi178=x;smithi186=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:09.156 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: Saving service mgr spec with placement smithi178=x;smithi186=y;smithi187=z;count:3 2023-10-16T14:26:09.156 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.156 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.156 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:09.156 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.157 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:09.157 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.157 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:09.157 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:09.157 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:09.158 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.170 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: Reconfiguring mon.c (monmap changed)... 2023-10-16T14:26:09.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: Reconfiguring daemon mon.c on smithi187 2023-10-16T14:26:09.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi178=x;smithi186=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:09.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: Saving service mgr spec with placement smithi178=x;smithi186=y;smithi187=z;count:3 2023-10-16T14:26:09.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.171 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:09.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:09.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.172 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:09.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:09.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:09.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.176 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-10-16T14:26:09.177 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-10-16T14:26:09.177 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00018703 s, 2.7 MB/s 2023-10-16T14:26:09.179 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T14:26:09.225 DEBUG:teuthology.orchestra.run.smithi186:> set -ex 2023-10-16T14:26:09.225 DEBUG:teuthology.orchestra.run.smithi186:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T14:26:09.246 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T14:26:09.246 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 2023-10-16T14:26:09.306 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T14:26:09.306 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:09.306 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 392397 Links: 1 2023-10-16T14:26:09.307 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:09.307 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:09.307 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 14:25:55.565715916 +0000 2023-10-16T14:26:09.307 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 14:16:30.081574107 +0000 2023-10-16T14:26:09.307 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 14:16:30.081574107 +0000 2023-10-16T14:26:09.308 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-16T14:26:09.308 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T14:26:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: Reconfiguring mon.c (monmap changed)... 2023-10-16T14:26:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: Reconfiguring daemon mon.c on smithi187 2023-10-16T14:26:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi178=x;smithi186=y;smithi187=z", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:09.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: Saving service mgr spec with placement smithi178=x;smithi186=y;smithi187=z;count:3 2023-10-16T14:26:09.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:09.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:09.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:09.356 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T14:26:09.357 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T14:26:09.357 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.00029098 s, 1.8 MB/s 2023-10-16T14:26:09.358 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T14:26:09.417 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 2023-10-16T14:26:09.475 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T14:26:09.475 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:09.476 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 386686 Links: 1 2023-10-16T14:26:09.476 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:09.476 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:09.476 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 14:25:55.565715916 +0000 2023-10-16T14:26:09.476 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 14:16:30.576565283 +0000 2023-10-16T14:26:09.477 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 14:16:30.576565283 +0000 2023-10-16T14:26:09.477 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-16T14:26:09.477 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T14:26:09.543 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T14:26:09.543 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T14:26:09.544 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000255005 s, 2.0 MB/s 2023-10-16T14:26:09.545 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T14:26:09.604 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 2023-10-16T14:26:09.662 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T14:26:09.662 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:09.663 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 394459 Links: 1 2023-10-16T14:26:09.663 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:09.663 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:09.663 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 14:25:55.565715916 +0000 2023-10-16T14:26:09.663 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 14:16:31.086556192 +0000 2023-10-16T14:26:09.664 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 14:16:31.086556192 +0000 2023-10-16T14:26:09.664 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-16T14:26:09.664 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T14:26:09.730 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T14:26:09.730 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T14:26:09.730 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000218457 s, 2.3 MB/s 2023-10-16T14:26:09.731 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T14:26:09.789 DEBUG:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 2023-10-16T14:26:09.847 INFO:teuthology.orchestra.run.smithi186.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T14:26:09.847 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:09.847 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 6h/6d Inode: 395445 Links: 1 2023-10-16T14:26:09.847 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:09.848 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:09.848 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2023-10-16 14:25:55.565715916 +0000 2023-10-16T14:26:09.848 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2023-10-16 14:16:31.589547225 +0000 2023-10-16T14:26:09.848 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2023-10-16 14:16:31.589547225 +0000 2023-10-16T14:26:09.848 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2023-10-16T14:26:09.849 DEBUG:teuthology.orchestra.run.smithi186:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T14:26:09.913 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2023-10-16T14:26:09.913 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2023-10-16T14:26:09.913 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes copied, 0.000242305 s, 2.1 MB/s 2023-10-16T14:26:09.914 DEBUG:teuthology.orchestra.run.smithi186:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T14:26:09.973 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-10-16T14:26:09.973 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T14:26:09.991 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T14:26:09.992 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2023-10-16T14:26:10.049 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T14:26:10.050 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:10.050 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 392721 Links: 1 2023-10-16T14:26:10.050 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:10.050 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:10.051 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-10-16 14:26:09.916106333 +0000 2023-10-16T14:26:10.051 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-10-16 14:16:31.597160451 +0000 2023-10-16T14:26:10.051 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-10-16 14:16:31.597160451 +0000 2023-10-16T14:26:10.051 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-10-16T14:26:10.051 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T14:26:10.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:09 smithi186 ceph-mon[134892]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:09 smithi186 ceph-mon[134892]: Deploying daemon mgr.z on smithi187 2023-10-16T14:26:10.120 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-10-16T14:26:10.120 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-10-16T14:26:10.121 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000236716 s, 2.2 MB/s 2023-10-16T14:26:10.121 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:09 smithi187 ceph-mon[135297]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:10.121 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:09 smithi187 ceph-mon[135297]: Deploying daemon mgr.z on smithi187 2023-10-16T14:26:10.122 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 systemd[1]: Starting Ceph mgr.z for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:26:10.123 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T14:26:10.182 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2023-10-16T14:26:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T14:26:10.241 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:10.242 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 381784 Links: 1 2023-10-16T14:26:10.242 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:10.242 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:10.242 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-10-16 14:26:10.046104201 +0000 2023-10-16T14:26:10.243 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-10-16 14:16:32.091152162 +0000 2023-10-16T14:26:10.243 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-10-16 14:16:32.091152162 +0000 2023-10-16T14:26:10.243 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-10-16T14:26:10.243 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T14:26:10.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:09 smithi178 ceph-mon[98495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:10.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:09 smithi178 ceph-mon[98495]: Deploying daemon mgr.z on smithi187 2023-10-16T14:26:10.307 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-10-16T14:26:10.308 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-10-16T14:26:10.308 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000244386 s, 2.1 MB/s 2023-10-16T14:26:10.309 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T14:26:10.367 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2023-10-16T14:26:10.424 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T14:26:10.425 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:10.425 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 387915 Links: 1 2023-10-16T14:26:10.425 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:10.425 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:10.425 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-10-16 14:26:10.047104185 +0000 2023-10-16T14:26:10.426 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-10-16 14:16:32.598143656 +0000 2023-10-16T14:26:10.426 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-10-16 14:16:32.598143656 +0000 2023-10-16T14:26:10.426 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-10-16T14:26:10.426 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T14:26:10.492 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-10-16T14:26:10.493 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-10-16T14:26:10.493 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000215893 s, 2.4 MB/s 2023-10-16T14:26:10.494 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T14:26:10.552 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2023-10-16T14:26:10.608 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T14:26:10.609 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T14:26:10.609 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 395383 Links: 1 2023-10-16T14:26:10.609 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:26:10.609 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-10-16T14:26:10.610 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-10-16 14:26:10.047104185 +0000 2023-10-16T14:26:10.610 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-10-16 14:16:33.103135182 +0000 2023-10-16T14:26:10.610 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-10-16 14:16:33.103135182 +0000 2023-10-16T14:26:10.610 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-10-16T14:26:10.611 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T14:26:10.674 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 podman[137257]: 2023-10-16T14:26:10.676 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-10-16T14:26:10.677 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-10-16T14:26:10.677 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000196877 s, 2.6 MB/s 2023-10-16T14:26:10.678 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T14:26:10.736 INFO:tasks.cephadm:Deploying osd.0 on smithi178 with /dev/vg_nvme/lv_4... 2023-10-16T14:26:10.736 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-16T14:26:10.925 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 ceph-mgr[137423]: -- 172.21.15.187:0/1164966421 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f7b37d11e0 con 0x55f7b37a6800 2023-10-16T14:26:10.925 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 bash[137257]: fe8e286c133b09f9a1a0b7afff5a942b79620d1dabe1de4ff0a54c727382c3df 2023-10-16T14:26:10.925 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 systemd[1]: Started Ceph mgr.z for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:26:10.925 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:10.798+0000 7f32953e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T14:26:10.926 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:10.895+0000 7f32953e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T14:26:11.247 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:10.985+0000 7f32953e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T14:26:11.248 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.077+0000 7f32953e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T14:26:11.248 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.245+0000 7f32953e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T14:26:11.478 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:26:11.922 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:11.922 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:11.922 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:11.923 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:11.923 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:11.923 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:11.923 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:11.923 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:11.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-mon[135297]: Deploying daemon mgr.y on smithi186 2023-10-16T14:26:11.924 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.608+0000 7f32953e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T14:26:11.924 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.737+0000 7f32953e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T14:26:11.924 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.825+0000 7f32953e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T14:26:11.925 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:11.919+0000 7f32953e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T14:26:12.117 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:12.118 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.118 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.118 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.118 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:12.119 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:12.119 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:12.119 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:12.119 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:11 smithi178 ceph-mon[98495]: Deploying daemon mgr.y on smithi186 2023-10-16T14:26:12.142 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:12.142 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.143 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.143 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:12.143 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:12.143 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-10-16T14:26:12.143 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:12.144 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:12.144 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:11 smithi186 ceph-mon[134892]: Deploying daemon mgr.y on smithi186 2023-10-16T14:26:12.576 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:12 smithi186 systemd[1]: Starting Ceph mgr.y for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:26:12.675 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:12.339+0000 7f32953e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T14:26:12.910 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:12 smithi186 podman[136331]: 2023-10-16T14:26:13.210 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:12 smithi186 bash[136331]: 4be6c61c8e58ec4106afed8fb1720074aeae884a5a829dd2b27172d7e69d975b 2023-10-16T14:26:13.210 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:12 smithi186 ceph-mgr[136371]: -- 172.21.15.186:0/3852648634 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55865315f1e0 con 0x558653134800 2023-10-16T14:26:13.211 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:12 smithi186 systemd[1]: Started Ceph mgr.y for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:26:13.211 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.032+0000 7fc816a19200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T14:26:13.211 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.113+0000 7fc816a19200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T14:26:13.211 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.206+0000 7fc816a19200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T14:26:13.425 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:13.006+0000 7f32953e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T14:26:13.425 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:13.089+0000 7f32953e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T14:26:13.484 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.301+0000 7fc816a19200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T14:26:13.612 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-16T14:26:13.613 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-16T14:26:13.613 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-16T14:26:13.613 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-16T14:26:13.613 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0568748 s, 184 MB/s 2023-10-16T14:26:13.614 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-16T14:26:13.826 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.481+0000 7fc816a19200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T14:26:14.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:14.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.002 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.003 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.003 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.003 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:14.066 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2023-10-16T14:26:14.110 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:13.870+0000 7fc816a19200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T14:26:14.110 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:14.008+0000 7fc816a19200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T14:26:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.112 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:14.175 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:13.896+0000 7f32953e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:26:14.175 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:14.002+0000 7f32953e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T14:26:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:14.485 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:14.181+0000 7f32953e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:26:14.576 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:14.106+0000 7fc816a19200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T14:26:14.576 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:14.203+0000 7fc816a19200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T14:26:14.799 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:26:14.877 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:14.482+0000 7f32953e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T14:26:14.877 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:14.723+0000 7f32953e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T14:26:14.878 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:14.874+0000 7f32953e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T14:26:15.076 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:14.656+0000 7fc816a19200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T14:26:15.425 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:15.141+0000 7f32953e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T14:26:15.808 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:15.338+0000 7fc816a19200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-16T14:26:15.809 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:15.419+0000 7fc816a19200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T14:26:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:16.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:16.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:16.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.170 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:15.844+0000 7f32953e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T14:26:16.170 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:15.975+0000 7f32953e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T14:26:16.170 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:16.167+0000 7f32953e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T14:26:16.171 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:16.171 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.171 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.171 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.171 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:16.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:16.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-10-16T14:26:16.172 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-10-16T14:26:16.173 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:16.425 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:16.280+0000 7f32953e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T14:26:16.454 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:16.189+0000 7fc816a19200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:26:16.455 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:16.280+0000 7fc816a19200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T14:26:16.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 -- 172.21.15.178:0/501776571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 msgr2=0x7f55d41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:16.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 --2- 172.21.15.178:0/501776571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41025d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55c80099c0 tx=0x7f55c8009cd0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 -- 172.21.15.178:0/501776571 shutdown_connections 2023-10-16T14:26:16.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 --2- 172.21.15.178:0/501776571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4102b10 0x7f55d41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 --2- 172.21.15.178:0/501776571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4100f70 0x7f55d4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 --2- 172.21.15.178:0/501776571 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41025d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 -- 172.21.15.178:0/501776571 >> 172.21.15.178:0/501776571 conn(0x7f55d40fc4d0 msgr2=0x7f55d40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:16.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 -- 172.21.15.178:0/501776571 shutdown_connections 2023-10-16T14:26:16.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.719+0000 7f55da6a0700 1 -- 172.21.15.178:0/501776571 wait complete. 2023-10-16T14:26:16.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.720+0000 7f55da6a0700 1 Processor -- start 2023-10-16T14:26:16.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.720+0000 7f55da6a0700 1 -- start start 2023-10-16T14:26:16.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4100f70 0x7f55d4198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:16.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4102b10 0x7f55d41a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a0a00 con 0x7f55d4100f70 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a0b40 con 0x7f55d4102170 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55da6a0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d41a0cd0 con 0x7f55d4102b10 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:16.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4100f70 0x7f55d4198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:16.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52906/0 (socket says 172.21.15.178:52906) 2023-10-16T14:26:16.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 -- 172.21.15.178:0/143327430 learned_addr learned my addr 172.21.15.178:0/143327430 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:26:16.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d8c3d700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4102b10 0x7f55d41a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:16.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 -- 172.21.15.178:0/143327430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4102b10 msgr2=0x7f55d41a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:16.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4102b10 0x7f55d41a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 -- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4100f70 msgr2=0x7f55d4198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4100f70 0x7f55d4198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.721+0000 7f55d37fe700 1 -- 172.21.15.178:0/143327430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c80095f0 con 0x7f55d4102170 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55d37fe700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009fd0 tx=0x7f55c800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c801d070 con 0x7f55d4102170 2023-10-16T14:26:16.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c8005050 con 0x7f55d4102170 2023-10-16T14:26:16.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d41a0f50 con 0x7f55d4102170 2023-10-16T14:26:16.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8004a70 con 0x7f55d4102170 2023-10-16T14:26:16.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.722+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d41a1360 con 0x7f55d4102170 2023-10-16T14:26:16.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.723+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d410c7c0 con 0x7f55d4102170 2023-10-16T14:26:16.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.723+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f55c8004bf0 con 0x7f55d4102170 2023-10-16T14:26:16.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.724+0000 7f55d17fa700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 0x7f55bc03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:16.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.724+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f55c804c390 con 0x7f55d4102170 2023-10-16T14:26:16.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.724+0000 7f55d3fff700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 0x7f55bc03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:16.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.725+0000 7f55d3fff700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 0x7f55bc03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f55d4101fd0 tx=0x7f55c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:16.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.729+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f55c8029540 con 0x7f55d4102170 2023-10-16T14:26:16.734 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:16.451+0000 7fc816a19200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T14:26:16.811 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:16.494+0000 7f32953e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T14:26:16.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:16.968+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f55d4108ad0 con 0x7f55bc038af0 2023-10-16T14:26:17.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: Reconfiguring mgr.x (unknown last config time)... 2023-10-16T14:26:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: Reconfiguring daemon mgr.x on smithi178 2023-10-16T14:26:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:17.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:17.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:17.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.076 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:16.731+0000 7fc816a19200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-16T14:26:17.076 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:16.964+0000 7fc816a19200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T14:26:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: Reconfiguring mgr.x (unknown last config time)... 2023-10-16T14:26:17.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: Reconfiguring daemon mgr.x on smithi178 2023-10-16T14:26:17.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:17.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:17.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:17.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: Reconfiguring mgr.x (unknown last config time)... 2023-10-16T14:26:17.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: Reconfiguring daemon mgr.x on smithi178 2023-10-16T14:26:17.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:17.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:17.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:17.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:17.378 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:17.113+0000 7fc816a19200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T14:26:17.675 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z[137335]: 2023-10-16T14:26:17.362+0000 7f32953e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T14:26:17.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:17.814+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mgrmap(e 14) v1 ==== 90792+0+0 (secure 0 0 0) 0x7f55c802bce0 con 0x7f55d4102170 2023-10-16T14:26:17.820 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:17.375+0000 7fc816a19200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T14:26:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:18.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:18.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: Standby manager daemon z started 2023-10-16T14:26:18.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-16T14:26:18.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:18.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-16T14:26:18.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:17 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:18.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: Standby manager daemon z started 2023-10-16T14:26:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-16T14:26:18.084 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:18.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-16T14:26:18.085 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:17 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: Standby manager daemon z started 2023-10-16T14:26:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-10-16T14:26:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-10-16T14:26:18.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:17 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.187:0/4152530017' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:18.406 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:18.081+0000 7fc816a19200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T14:26:18.406 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:18.211+0000 7fc816a19200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T14:26:18.739 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:18.402+0000 7fc816a19200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T14:26:18.740 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:18.516+0000 7fc816a19200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T14:26:18.990 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:18 smithi178 ceph-mon[98495]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:18.991 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:18 smithi178 ceph-mon[98495]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-16T14:26:18.991 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-16T14:26:19.076 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:18.736+0000 7fc816a19200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T14:26:19.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-mon[134892]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:19.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-mon[134892]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-16T14:26:19.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-16T14:26:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:18 smithi187 ceph-mon[135297]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:18 smithi187 ceph-mon[135297]: mgrmap e14: x(active, since 71s), standbys: z 2023-10-16T14:26:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-10-16T14:26:20.076 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y[136347]: 2023-10-16T14:26:19.598+0000 7fc816a19200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T14:26:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:20.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: Standby manager daemon y started 2023-10-16T14:26:20.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-16T14:26:20.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:20.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-16T14:26:20.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:19 smithi186 ceph-mon[134892]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:20.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:20.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: Standby manager daemon y started 2023-10-16T14:26:20.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-16T14:26:20.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:20.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-16T14:26:20.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:19 smithi187 ceph-mon[135297]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:20.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:20.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: Standby manager daemon y started 2023-10-16T14:26:20.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-10-16T14:26:20.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-10-16T14:26:20.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-10-16T14:26:20.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:19 smithi178 ceph-mon[98495]: from='mgr.? 172.21.15.186:0/4148233274' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-10-16T14:26:20.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:20.603+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mon.2 v2:172.21.15.186:3300/0 8 ==== mgrmap(e 15) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f55c80163a0 con 0x7f55d4102170 2023-10-16T14:26:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/358889733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]': finished 2023-10-16T14:26:21.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: osdmap e5: 1 total, 0 up, 1 in 2023-10-16T14:26:21.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:21.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-16T14:26:21.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-16T14:26:21.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2574511482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:21.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/358889733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]': finished 2023-10-16T14:26:21.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: osdmap e5: 1 total, 0 up, 1 in 2023-10-16T14:26:21.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:21.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-16T14:26:21.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-16T14:26:21.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2574511482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:21.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/358889733' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]: dispatch 2023-10-16T14:26:21.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a72b3ff-3836-4ade-969c-531e6616f47e"}]': finished 2023-10-16T14:26:21.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: osdmap e5: 1 total, 0 up, 1 in 2023-10-16T14:26:21.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:21.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-10-16T14:26:21.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-10-16T14:26:21.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2574511482' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:22.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:21 smithi187 ceph-mon[135297]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:22.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:21 smithi178 ceph-mon[98495]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:22.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:21 smithi186 ceph-mon[134892]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:23 smithi187 ceph-mon[135297]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:24.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:23 smithi178 ceph-mon[98495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:23 smithi186 ceph-mon[134892]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:25 smithi178 ceph-mon[98495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:26.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:25 smithi187 ceph-mon[135297]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:26.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:25 smithi186 ceph-mon[134892]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:28.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:27 smithi187 ceph-mon[135297]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:28.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-16T14:26:28.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:28.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:27 smithi178 ceph-mon[98495]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:28.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-16T14:26:28.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:27 smithi186 ceph-mon[134892]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-10-16T14:26:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:28.952 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:28 smithi178 ceph-mon[98495]: Deploying daemon osd.0 on smithi178 2023-10-16T14:26:29.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:28 smithi187 ceph-mon[135297]: Deploying daemon osd.0 on smithi178 2023-10-16T14:26:29.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:28 smithi186 ceph-mon[134892]: Deploying daemon osd.0 on smithi178 2023-10-16T14:26:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:29 smithi187 ceph-mon[135297]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:30.296 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:29 smithi178 ceph-mon[98495]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:29 smithi186 ceph-mon[134892]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:32.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:31 smithi178 ceph-mon[98495]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:32.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:31 smithi187 ceph-mon[135297]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:32.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:31 smithi186 ceph-mon[134892]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:33 smithi187 ceph-mon[135297]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:33 smithi178 ceph-mon[98495]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:34.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:34.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:33 smithi186 ceph-mon[134892]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:34.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:34.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:36.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:35 smithi187 ceph-mon[135297]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:35 smithi178 ceph-mon[98495]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:35 smithi186 ceph-mon[134892]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:37.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.027+0000 7f55d17fa700 1 -- 172.21.15.178:0/143327430 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f55d4108ad0 con 0x7f55bc038af0 2023-10-16T14:26:37.031 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 0 on host 'smithi178' 2023-10-16T14:26:37.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 msgr2=0x7f55bc03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:37.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 0x7f55bc03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f55d4101fd0 tx=0x7f55c4009260 comp rx=0 tx=0).stop 2023-10-16T14:26:37.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 msgr2=0x7f55d41993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:37.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009fd0 tx=0x7f55c800f690 comp rx=0 tx=0).stop 2023-10-16T14:26:37.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 shutdown_connections 2023-10-16T14:26:37.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f55bc038af0 0x7f55bc03afb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:37.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55d4102b10 0x7f55d41a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:37.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55d4100f70 0x7f55d4198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:37.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.029+0000 7f55da6a0700 1 --2- 172.21.15.178:0/143327430 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55d4102170 0x7f55d41993b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:37.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.030+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 >> 172.21.15.178:0/143327430 conn(0x7f55d40fc4d0 msgr2=0x7f55d40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:37.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.030+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 shutdown_connections 2023-10-16T14:26:37.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:26:37.030+0000 7f55da6a0700 1 -- 172.21.15.178:0/143327430 wait complete. 2023-10-16T14:26:37.441 DEBUG:teuthology.orchestra.run.smithi178:osd.0> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.0.service 2023-10-16T14:26:37.446 INFO:tasks.cephadm:Deploying osd.1 on smithi186 with /dev/vg_nvme/lv_4... 2023-10-16T14:26:37.446 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-16T14:26:37.477 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.478 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.478 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:37.478 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:37.478 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.479 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:37.479 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.479 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.479 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:37.503 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.504 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.504 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:37.504 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:37.504 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.504 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:37.505 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.505 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.505 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:37.518 INFO:journalctl@ceph.osd.0.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:26:37.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:37.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:37.803 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:26:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:26:37.582+0000 7fd7eed6a640 -1 osd.0 0 log_to_monitors true 2023-10-16T14:26:38.015 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:26:38.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:38 smithi178 ceph-mon[98495]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:38.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:38 smithi178 ceph-mon[98495]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:38.508 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:38 smithi186 ceph-mon[134892]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:38.508 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:38 smithi186 ceph-mon[134892]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:38.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:38 smithi187 ceph-mon[135297]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:38.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:38 smithi187 ceph-mon[135297]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-10-16T14:26:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-16T14:26:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: osdmap e6: 1 total, 0 up, 1 in 2023-10-16T14:26:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:39 smithi186 ceph-mon[134892]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:39.508 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-16T14:26:39.509 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-16T14:26:39.509 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-16T14:26:39.509 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-16T14:26:39.509 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110982 s, 945 MB/s 2023-10-16T14:26:39.510 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-16T14:26:39.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-16T14:26:39.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: osdmap e6: 1 total, 0 up, 1 in 2023-10-16T14:26:39.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:39.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-mon[98495]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:39.555 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:26:39.250+0000 7fd7e4be2700 -1 osd.0 0 waiting for initial osdmap 2023-10-16T14:26:39.555 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:26:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:26:39.255+0000 7fd7dc1ce700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:26:39.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-10-16T14:26:39.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: osdmap e6: 1 total, 0 up, 1 in 2023-10-16T14:26:39.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:39.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: from='osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:26:39.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:39 smithi187 ceph-mon[135297]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:39.956 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_4 2023-10-16T14:26:40.529 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:40 smithi178 ceph-mon[98495]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:26:40.529 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:40 smithi178 ceph-mon[98495]: osdmap e7: 1 total, 0 up, 1 in 2023-10-16T14:26:40.530 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:40 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:40.530 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:40 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:40.555 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:26:40.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:40 smithi186 ceph-mon[134892]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:26:40.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:40 smithi186 ceph-mon[134892]: osdmap e7: 1 total, 0 up, 1 in 2023-10-16T14:26:40.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:40 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:40.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:40 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:40.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:40 smithi187 ceph-mon[135297]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:26:40.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:40 smithi187 ceph-mon[135297]: osdmap e7: 1 total, 0 up, 1 in 2023-10-16T14:26:40.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:40 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:40.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:40 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.315 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:26:41.315 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:26:41.316 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.316 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] boot 2023-10-16T14:26:41.316 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: osdmap e8: 1 total, 1 up, 1 in 2023-10-16T14:26:41.316 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.316 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:41.317 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi178 2023-10-16T14:26:41.317 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.317 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.317 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:26:41.317 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi178 to 17271M 2023-10-16T14:26:41.318 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.318 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:41.318 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:41.318 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:41 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:26:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:26:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] boot 2023-10-16T14:26:41.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: osdmap e8: 1 total, 1 up, 1 in 2023-10-16T14:26:41.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi178 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi178 to 17271M 2023-10-16T14:26:41.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:41.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:41.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:41 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:26:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:26:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: osd.0 [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] boot 2023-10-16T14:26:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: osdmap e8: 1 total, 1 up, 1 in 2023-10-16T14:26:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-10-16T14:26:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-10-16T14:26:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi178 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi178 to 17271M 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:41.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:41.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:41 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:41.756 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 -- 172.21.15.186:0/2142314078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101060 msgr2=0x7fcbb01014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:41.757 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 --2- 172.21.15.186:0/2142314078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101060 0x7fcbb01014e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009990 tx=0x7fcbac009ca0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.757 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 -- 172.21.15.186:0/2142314078 shutdown_connections 2023-10-16T14:26:41.757 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 --2- 172.21.15.186:0/2142314078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb0108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.758 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 --2- 172.21.15.186:0/2142314078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.758 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 --2- 172.21.15.186:0/2142314078 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb0101060 0x7fcbb01014e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.759 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 -- 172.21.15.186:0/2142314078 >> 172.21.15.186:0/2142314078 conn(0x7fcbb00fb480 msgr2=0x7fcbb00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:41.759 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 -- 172.21.15.186:0/2142314078 shutdown_connections 2023-10-16T14:26:41.759 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.753+0000 7fcbb7938700 1 -- 172.21.15.186:0/2142314078 wait complete. 2023-10-16T14:26:41.761 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 Processor -- start 2023-10-16T14:26:41.762 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 -- start start 2023-10-16T14:26:41.762 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:41.762 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 0x7fcbb0199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:41.762 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb01a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:41.763 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01a0b80 con 0x7fcbb0101060 2023-10-16T14:26:41.763 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01a0cc0 con 0x7fcbb00ffe70 2023-10-16T14:26:41.763 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb7938700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb01a0e50 con 0x7fcbb0101a20 2023-10-16T14:26:41.763 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:41.763 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47064/0 (socket says 172.21.15.186:47064) 2023-10-16T14:26:41.764 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 -- 172.21.15.186:0/1848786482 learned_addr learned my addr 172.21.15.186:0/1848786482 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T14:26:41.764 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb4ed3700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 0x7fcbb0199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:41.764 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb5ed5700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb01a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:41.764 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 -- 172.21.15.186:0/1848786482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 msgr2=0x7fcbb01a05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:41.764 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb01a05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.765 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 -- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 msgr2=0x7fcbb0199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:41.765 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 0x7fcbb0199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:41.765 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.754+0000 7fcbb56d4700 1 -- 172.21.15.186:0/1848786482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0095f0 con 0x7fcbb00ffe70 2023-10-16T14:26:41.765 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcbb4ed3700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 0x7fcbb0199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:26:41.766 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcbb5ed5700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:26:41.766 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcbb56d4700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcba000b500 tx=0x7fcba000b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:41.766 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011830 con 0x7fcbb00ffe70 2023-10-16T14:26:41.766 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcba0011e50 con 0x7fcbb00ffe70 2023-10-16T14:26:41.766 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb01a1130 con 0x7fcbb00ffe70 2023-10-16T14:26:41.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000f2e0 con 0x7fcbb00ffe70 2023-10-16T14:26:41.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.755+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb01a16b0 con 0x7fcbb00ffe70 2023-10-16T14:26:41.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.756+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb010b810 con 0x7fcbb00ffe70 2023-10-16T14:26:41.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.757+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 15) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fcba0010580 con 0x7fcbb00ffe70 2023-10-16T14:26:41.767 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.757+0000 7fcba67fc700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 0x7fcb9c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:26:41.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.758+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fcba00cae00 con 0x7fcbb00ffe70 2023-10-16T14:26:41.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.758+0000 7fcbb4ed3700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 0x7fcb9c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:26:41.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.758+0000 7fcbb4ed3700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 0x7fcb9c0a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006010 tx=0x7fcbac00b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:26:41.768 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.762+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcba00cf070 con 0x7fcbb00ffe70 2023-10-16T14:26:41.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:41.959+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbb01a1930 con 0x7fcb9c0a13f0 2023-10-16T14:26:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:42 smithi178 ceph-mon[98495]: osdmap e9: 1 total, 1 up, 1 in 2023-10-16T14:26:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:42 smithi186 ceph-mon[134892]: osdmap e9: 1 total, 1 up, 1 in 2023-10-16T14:26:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:42.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:42 smithi187 ceph-mon[135297]: osdmap e9: 1 total, 1 up, 1 in 2023-10-16T14:26:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:26:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:26:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:43 smithi178 ceph-mon[98495]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:43.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:43 smithi178 ceph-mon[98495]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:43.571 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:43 smithi186 ceph-mon[134892]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:43.572 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:43 smithi186 ceph-mon[134892]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:43.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:43 smithi187 ceph-mon[135297]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:26:43.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:43 smithi187 ceph-mon[135297]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:44.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:44 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/1654214862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:44 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:44 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]': finished 2023-10-16T14:26:44.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:44 smithi187 ceph-mon[135297]: osdmap e10: 2 total, 1 up, 2 in 2023-10-16T14:26:44.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:44 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:26:44.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:44 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/1654214862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:44 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:44 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]': finished 2023-10-16T14:26:44.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:44 smithi178 ceph-mon[98495]: osdmap e10: 2 total, 1 up, 2 in 2023-10-16T14:26:44.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:44 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:26:44.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:44 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/1654214862' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:44 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]: dispatch 2023-10-16T14:26:44.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:44 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0811af9a-0421-41e8-be52-d60c284536d6"}]': finished 2023-10-16T14:26:44.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:44 smithi186 ceph-mon[134892]: osdmap e10: 2 total, 1 up, 2 in 2023-10-16T14:26:44.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:44 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:26:45.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:45 smithi187 ceph-mon[135297]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:45.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/3011389713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:45 smithi178 ceph-mon[98495]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/3011389713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:45 smithi186 ceph-mon[134892]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/3011389713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:26:47.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:47 smithi187 ceph-mon[135297]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:48.033 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:47 smithi186 ceph-mon[134892]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:48.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:47 smithi178 ceph-mon[98495]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:49.770 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:49 smithi186 ceph-mon[134892]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:49.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:49 smithi187 ceph-mon[135297]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:50.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:49 smithi178 ceph-mon[98495]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:51.664 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:51 smithi186 ceph-mon[134892]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:51.665 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:51 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-16T14:26:51.665 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:51 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:51.665 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:51 smithi186 ceph-mon[134892]: Deploying daemon osd.1 on smithi186 2023-10-16T14:26:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:51 smithi187 ceph-mon[135297]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:51 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-16T14:26:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:51 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:51.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:51 smithi187 ceph-mon[135297]: Deploying daemon osd.1 on smithi186 2023-10-16T14:26:52.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:51 smithi178 ceph-mon[98495]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:52.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:51 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-10-16T14:26:52.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:51 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:52.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:51 smithi178 ceph-mon[98495]: Deploying daemon osd.1 on smithi186 2023-10-16T14:26:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:53 smithi187 ceph-mon[135297]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:54.038 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:53 smithi186 ceph-mon[134892]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:54.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:53 smithi178 ceph-mon[98495]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:56.057 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:55 smithi186 ceph-mon[134892]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:56.058 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:55 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.058 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:55 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.058 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:55 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:55 smithi178 ceph-mon[98495]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:55 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:55 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:55 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:55 smithi187 ceph-mon[135297]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:55 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:55 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:56.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:55 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:57.477 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.474+0000 7fcba67fc700 1 -- 172.21.15.186:0/1848786482 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcbb01a1930 con 0x7fcb9c0a13f0 2023-10-16T14:26:57.477 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 1 on host 'smithi186' 2023-10-16T14:26:57.479 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 msgr2=0x7fcb9c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:57.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 0x7fcb9c0a38b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcbac006010 tx=0x7fcbac00b540 comp rx=0 tx=0).stop 2023-10-16T14:26:57.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 msgr2=0x7fcbb0198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:26:57.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcba000b500 tx=0x7fcba000b8c0 comp rx=0 tx=0).stop 2023-10-16T14:26:57.480 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 shutdown_connections 2023-10-16T14:26:57.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb9c0a13f0 0x7fcb9c0a38b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:57.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbb0101a20 0x7fcbb01a05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:57.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcbb00ffe70 0x7fcbb0198ff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:57.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 --2- 172.21.15.186:0/1848786482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb0101060 0x7fcbb0199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:26:57.481 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 >> 172.21.15.186:0/1848786482 conn(0x7fcbb00fb480 msgr2=0x7fcbb00693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:26:57.482 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 shutdown_connections 2023-10-16T14:26:57.482 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:26:57.476+0000 7fcbb7938700 1 -- 172.21.15.186:0/1848786482 wait complete. 2023-10-16T14:26:57.833 DEBUG:teuthology.orchestra.run.smithi186:osd.1> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.1.service 2023-10-16T14:26:57.836 INFO:tasks.cephadm:Deploying osd.2 on smithi187 with /dev/vg_nvme/lv_4... 2023-10-16T14:26:57.837 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-10-16T14:26:57.865 INFO:journalctl@ceph.osd.1.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:26:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:58.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:58.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:26:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:26:58.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:26:58.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:26:58.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:26:58.403 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:26:59.326 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:26:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:26:58.957+0000 7f98fb064640 -1 osd.1 0 log_to_monitors true 2023-10-16T14:26:59.857 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-10-16T14:26:59.857 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-10-16T14:26:59.857 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-10-16T14:26:59.857 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-10-16T14:26:59.858 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109376 s, 959 MB/s 2023-10-16T14:26:59.858 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-10-16T14:27:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:59 smithi187 ceph-mon[135297]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:59 smithi187 ceph-mon[135297]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:26:59 smithi187 ceph-mon[135297]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi187:vg_nvme/lv_4 2023-10-16T14:27:00.269 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:59 smithi186 ceph-mon[134892]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:00.269 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:59 smithi186 ceph-mon[134892]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.269 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:26:59 smithi186 ceph-mon[134892]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:59 smithi178 ceph-mon[98495]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:00.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:59 smithi178 ceph-mon[98495]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:26:59 smithi178 ceph-mon[98495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-10-16T14:27:00.802 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:27:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-16T14:27:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: osdmap e11: 2 total, 1 up, 2 in 2023-10-16T14:27:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:00 smithi178 ceph-mon[98495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:27:01.326 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:27:00.946+0000 7f98f06db700 -1 osd.1 0 waiting for initial osdmap 2023-10-16T14:27:01.326 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:27:00.950+0000 7f98e9ccb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:27:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-16T14:27:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: osdmap e11: 2 total, 1 up, 2 in 2023-10-16T14:27:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:01.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:01.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:00 smithi186 ceph-mon[134892]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:27:01.337 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-10-16T14:27:01.337 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.337 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: osdmap e11: 2 total, 1 up, 2 in 2023-10-16T14:27:01.337 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:01.338 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:27:01.338 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.338 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.338 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:01.338 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.339 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:01.339 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:01.339 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:01.339 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:00 smithi187 ceph-mon[135297]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:27:02.012 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 -- 172.21.15.187:0/3617483674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c101990 msgr2=0x7f7f5c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:02.012 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 --2- 172.21.15.187:0/3617483674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c101990 0x7f7f5c1082b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400b0a0 tx=0x7f7f5400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.012 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 -- 172.21.15.187:0/3617483674 shutdown_connections 2023-10-16T14:27:02.012 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 --2- 172.21.15.187:0/3617483674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c101990 0x7f7f5c1082b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.013 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 --2- 172.21.15.187:0/3617483674 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.013 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 --2- 172.21.15.187:0/3617483674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.013 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 -- 172.21.15.187:0/3617483674 >> 172.21.15.187:0/3617483674 conn(0x7f7f5c0fb3f0 msgr2=0x7f7f5c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:27:02.013 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 -- 172.21.15.187:0/3617483674 shutdown_connections 2023-10-16T14:27:02.013 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.006+0000 7f7f62133700 1 -- 172.21.15.187:0/3617483674 wait complete. 2023-10-16T14:27:02.014 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 Processor -- start 2023-10-16T14:27:02.014 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 -- start start 2023-10-16T14:27:02.014 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:02.014 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:02.014 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 0x7f7f5c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:02.015 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19d540 con 0x7f7f5c101990 2023-10-16T14:27:02.015 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19d680 con 0x7f7f5c100fd0 2023-10-16T14:27:02.015 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f62133700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c19d810 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.015 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:02.015 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33558/0 (socket says 172.21.15.187:33558) 2023-10-16T14:27:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f5b7fe700 1 -- 172.21.15.187:0/2111405863 learned_addr learned my addr 172.21.15.187:0/2111405863 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:27:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f5affd700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.007+0000 7f7f5bfff700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 0x7f7f5c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 -- 172.21.15.187:0/2111405863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 msgr2=0x7f7f5c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:02.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 -- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 msgr2=0x7f7f5c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 0x7f7f5c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 -- 172.21.15.187:0/2111405863 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f54009d00 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5affd700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5bfff700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 0x7f7f5c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:27:02.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f5b7fe700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d9b0 tx=0x7f7f5000dd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:02.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000bb60 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f50007e80 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c19daf0 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50004dd0 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.008+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c19dff0 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.009+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c1073e0 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.010+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 15) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f7f50004020 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.010+0000 7f7f58ff9700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 0x7f7f440a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:02.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.010+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f7f500cb420 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.011+0000 7f7f5affd700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 0x7f7f440a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:02.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.011+0000 7f7f5affd700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 0x7f7f440a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005e10 tx=0x7f7f4c005d60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:02.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.016+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7f500962d0 con 0x7f7f5c0ffe70 2023-10-16T14:27:02.214 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:02.211+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f5c02d1e0 con 0x7f7f440a13f0 2023-10-16T14:27:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi186 2023-10-16T14:27:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-16T14:27:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: osdmap e12: 2 total, 1 up, 2 in 2023-10-16T14:27:02.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:02.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi186 2023-10-16T14:27:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-16T14:27:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: osdmap e12: 2 total, 1 up, 2 in 2023-10-16T14:27:02.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:02.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi186 2023-10-16T14:27:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi186 to 17168M 2023-10-16T14:27:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: osdmap e12: 2 total, 1 up, 2 in 2023-10-16T14:27:02.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:02.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.261 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:27:03.262 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:27:03.262 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.262 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] boot 2023-10-16T14:27:03.263 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: osdmap e13: 2 total, 2 up, 2 in 2023-10-16T14:27:03.263 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.263 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:03.263 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:03.263 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:03.264 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:03.264 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:02 smithi187 ceph-mon[135297]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:27:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:27:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] boot 2023-10-16T14:27:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: osdmap e13: 2 total, 2 up, 2 in 2023-10-16T14:27:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:03.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:02 smithi178 ceph-mon[98495]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:03.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:27:03.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:27:03.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: osd.1 [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] boot 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: osdmap e13: 2 total, 2 up, 2 in 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:02 smithi186 ceph-mon[134892]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-10-16T14:27:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:03 smithi187 ceph-mon[135297]: osdmap e14: 2 total, 2 up, 2 in 2023-10-16T14:27:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:03 smithi178 ceph-mon[98495]: osdmap e14: 2 total, 2 up, 2 in 2023-10-16T14:27:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:03 smithi186 ceph-mon[134892]: osdmap e14: 2 total, 2 up, 2 in 2023-10-16T14:27:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/264477856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]': finished 2023-10-16T14:27:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: osdmap e15: 3 total, 2 up, 3 in 2023-10-16T14:27:05.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:05.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/3707138308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/264477856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]': finished 2023-10-16T14:27:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: osdmap e15: 3 total, 2 up, 3 in 2023-10-16T14:27:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/3707138308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/264477856' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]: dispatch 2023-10-16T14:27:06.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f3786153-58c5-4fec-a44b-d3e5b81e9748"}]': finished 2023-10-16T14:27:06.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: osdmap e15: 3 total, 2 up, 3 in 2023-10-16T14:27:06.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:06.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/3707138308' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:07 smithi187 ceph-mon[135297]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:08.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:07 smithi178 ceph-mon[98495]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:07 smithi186 ceph-mon[134892]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:09.802 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:09 smithi187 ceph-mon[135297]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:09 smithi178 ceph-mon[98495]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:09 smithi186 ceph-mon[134892]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:11.892 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:11 smithi187 ceph-mon[135297]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:11.892 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-16T14:27:11.893 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:11 smithi178 ceph-mon[98495]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-16T14:27:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:11 smithi186 ceph-mon[134892]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-10-16T14:27:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:12 smithi187 ceph-mon[135297]: Deploying daemon osd.2 on smithi187 2023-10-16T14:27:13.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:12 smithi178 ceph-mon[98495]: Deploying daemon osd.2 on smithi187 2023-10-16T14:27:13.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:12 smithi186 ceph-mon[134892]: Deploying daemon osd.2 on smithi187 2023-10-16T14:27:13.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:13 smithi187 ceph-mon[135297]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:13 smithi178 ceph-mon[98495]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:13 smithi186 ceph-mon[134892]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:15.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:15 smithi187 ceph-mon[135297]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:15 smithi178 ceph-mon[98495]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:15 smithi186 ceph-mon[134892]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:16.817 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:16.818 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:16.818 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:17.971 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:17 smithi187 ceph-mon[135297]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:18.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:17 smithi178 ceph-mon[98495]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:17 smithi186 ceph-mon[134892]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:18.378 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.374+0000 7f7f58ff9700 1 -- 172.21.15.187:0/2111405863 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7f5c02d1e0 con 0x7f7f440a13f0 2023-10-16T14:27:18.378 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 2 on host 'smithi187' 2023-10-16T14:27:18.380 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 msgr2=0x7f7f440a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:18.380 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 0x7f7f440a38b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005e10 tx=0x7f7f4c005d60 comp rx=0 tx=0).stop 2023-10-16T14:27:18.381 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 msgr2=0x7f7f5c1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:18.381 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d9b0 tx=0x7f7f5000dd70 comp rx=0 tx=0).stop 2023-10-16T14:27:18.381 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 shutdown_connections 2023-10-16T14:27:18.381 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f440a13f0 0x7f7f440a38b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:18.382 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f5c101990 0x7f7f5c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:18.382 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0ffe70 0x7f7f5c1959b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:18.382 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 --2- 172.21.15.187:0/2111405863 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7f5c100fd0 0x7f7f5c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:18.382 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.376+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 >> 172.21.15.187:0/2111405863 conn(0x7f7f5c0fb3f0 msgr2=0x7f7f5c104520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:27:18.383 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.377+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 shutdown_connections 2023-10-16T14:27:18.383 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:27:18.377+0000 7f7f62133700 1 -- 172.21.15.187:0/2111405863 wait complete. 2023-10-16T14:27:18.913 DEBUG:teuthology.orchestra.run.smithi187:osd.2> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.2.service 2023-10-16T14:27:18.915 INFO:tasks.cephadm:Deploying osd.3 on smithi178 with /dev/vg_nvme/lv_3... 2023-10-16T14:27:18.916 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-16T14:27:18.947 INFO:journalctl@ceph.osd.2.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:27:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:19.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:19.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:19.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:18 smithi187 ceph-mon[135297]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:19.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:19.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:19.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:19.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:18 smithi178 ceph-mon[98495]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:19.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:19.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:19.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:19.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:19.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:18 smithi186 ceph-mon[134892]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:19.657 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:27:19.924 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:27:19.628+0000 7f9a25696640 -1 osd.2 0 log_to_monitors true 2023-10-16T14:27:20.796 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:20 smithi178 ceph-mon[98495]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:20.796 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:20 smithi178 ceph-mon[98495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:20.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:20 smithi187 ceph-mon[135297]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:20.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:20 smithi187 ceph-mon[135297]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:20 smithi186 ceph-mon[134892]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:20 smithi186 ceph-mon[134892]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-10-16T14:27:21.484 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-16T14:27:21.484 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-16T14:27:21.484 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-16T14:27:21.484 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-16T14:27:21.485 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304591 s, 344 MB/s 2023-10-16T14:27:21.485 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-16T14:27:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-16T14:27:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: osdmap e16: 3 total, 2 up, 3 in 2023-10-16T14:27:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:21.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:21.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.925 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:27:21.629+0000 7f9a1ad0d700 -1 osd.2 0 waiting for initial osdmap 2023-10-16T14:27:21.925 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:27:21.632+0000 7f9a13afc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:27:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-16T14:27:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: osdmap e16: 3 total, 2 up, 3 in 2023-10-16T14:27:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:21.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:21.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:21.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:21.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:21.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:22.037 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2023-10-16T14:27:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-10-16T14:27:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: osdmap e16: 3 total, 2 up, 3 in 2023-10-16T14:27:22.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:22.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:22.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:27:22.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:22.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:22.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:22.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:22.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:22.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:22.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:22.771 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:27:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi187 2023-10-16T14:27:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi187 to 17168M 2023-10-16T14:27:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:27:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: osdmap e17: 3 total, 2 up, 3 in 2023-10-16T14:27:22.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:22.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:22 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi187 2023-10-16T14:27:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi187 to 17168M 2023-10-16T14:27:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:27:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: osdmap e17: 3 total, 2 up, 3 in 2023-10-16T14:27:23.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:23.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:23.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi187 2023-10-16T14:27:23.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi187 to 17168M 2023-10-16T14:27:23.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:27:23.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: osdmap e17: 3 total, 2 up, 3 in 2023-10-16T14:27:23.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:23.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:22 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:27:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:27:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] boot 2023-10-16T14:27:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: osdmap e18: 3 total, 3 up, 3 in 2023-10-16T14:27:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:27:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:27:24.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:24.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] boot 2023-10-16T14:27:24.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: osdmap e18: 3 total, 3 up, 3 in 2023-10-16T14:27:24.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:27:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:27:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-10-16T14:27:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: osd.2 [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] boot 2023-10-16T14:27:24.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: osdmap e18: 3 total, 3 up, 3 in 2023-10-16T14:27:24.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-10-16T14:27:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3129140212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0102170 msgr2=0x7f6ab01025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3129140212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0102170 0x7f6ab01025d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40099c0 tx=0x7f6aa4009cd0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3129140212 shutdown_connections 2023-10-16T14:27:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3129140212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 0x7f6ab01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3129140212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0100f70 0x7f6ab0101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3129140212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0102170 0x7f6ab01025d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3129140212 >> 172.21.15.178:0/3129140212 conn(0x7f6ab00fc4d0 msgr2=0x7f6ab00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:27:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3129140212 shutdown_connections 2023-10-16T14:27:24.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.304+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3129140212 wait complete. 2023-10-16T14:27:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 Processor -- start 2023-10-16T14:27:24.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 -- start start 2023-10-16T14:27:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0102170 0x7f6ab0197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 0x7f6ab019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab019e780 con 0x7f6ab0100f70 2023-10-16T14:27:24.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab019e8c0 con 0x7f6ab0102b10 2023-10-16T14:27:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6ab5d49700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab019ea80 con 0x7f6ab0102170 2023-10-16T14:27:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6aaf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:58386/0 (socket says 172.21.15.178:58386) 2023-10-16T14:27:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.305+0000 7f6aaf7fe700 1 -- 172.21.15.178:0/3752974102 learned_addr learned my addr 172.21.15.178:0/3752974102 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:27:24.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaeffd700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0102170 0x7f6ab0197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaffff700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 0x7f6ab019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 -- 172.21.15.178:0/3752974102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0102170 msgr2=0x7f6ab0197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:24.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0102170 0x7f6ab0197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 -- 172.21.15.178:0/3752974102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 msgr2=0x7f6ab019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 0x7f6ab019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 -- 172.21.15.178:0/3752974102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40095f0 con 0x7f6ab0100f70 2023-10-16T14:27:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aaf7fe700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e970 tx=0x7f6aa000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:24.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0009790 con 0x7f6ab0100f70 2023-10-16T14:27:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6aa000c7e0 con 0x7f6ab0100f70 2023-10-16T14:27:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0003c70 con 0x7f6ab0100f70 2023-10-16T14:27:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.306+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab019ed60 con 0x7f6ab0100f70 2023-10-16T14:27:24.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.307+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab019f1b0 con 0x7f6ab0100f70 2023-10-16T14:27:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.308+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab010c830 con 0x7f6ab0100f70 2023-10-16T14:27:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.309+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 15) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f6aa000cc50 con 0x7f6ab0100f70 2023-10-16T14:27:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.311+0000 7f6aacff9700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 0x7f6a980a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.311+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f6aa0014070 con 0x7f6ab0100f70 2023-10-16T14:27:24.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.311+0000 7f6aaeffd700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 0x7f6a980a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.312+0000 7f6aaeffd700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 0x7f6a980a3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400b580 tx=0x7f6aa4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:24.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.315+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6aa0096440 con 0x7f6ab0100f70 2023-10-16T14:27:24.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:24.526+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ab00612f0 con 0x7f6a980a1470 2023-10-16T14:27:25.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:25 smithi187 ceph-mon[135297]: osdmap e19: 3 total, 3 up, 3 in 2023-10-16T14:27:25.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:25.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:25.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:25.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:25 smithi178 ceph-mon[98495]: osdmap e19: 3 total, 3 up, 3 in 2023-10-16T14:27:25.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:25.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:25.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:25.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:25 smithi186 ceph-mon[134892]: osdmap e19: 3 total, 3 up, 3 in 2023-10-16T14:27:25.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:25.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:25.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:26.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:26.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:26.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:26.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:26.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: osdmap e20: 3 total, 3 up, 3 in 2023-10-16T14:27:26.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:26.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:26.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:26.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:26.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:26.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: osdmap e20: 3 total, 3 up, 3 in 2023-10-16T14:27:26.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:26.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: from='client.14271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:26.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: pgmap v78: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:26.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:26.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:26.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: osdmap e20: 3 total, 3 up, 3 in 2023-10-16T14:27:26.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:27:27.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:27.178+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6aa000c950 con 0x7f6ab0100f70 2023-10-16T14:27:27.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123557]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:27.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123557]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:27.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123557]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:27.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123557]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:27.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:27.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:27 smithi178 ceph-mon[98495]: osdmap e21: 3 total, 3 up, 3 in 2023-10-16T14:27:27.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:27 smithi178 ceph-mon[98495]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:27 smithi186 ceph-mon[134892]: osdmap e21: 3 total, 3 up, 3 in 2023-10-16T14:27:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:27 smithi186 ceph-mon[134892]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:27.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:27:27.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:27 smithi187 ceph-mon[135297]: osdmap e21: 3 total, 3 up, 3 in 2023-10-16T14:27:27.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:27 smithi187 ceph-mon[135297]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:28.053 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123593]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:28.053 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:28.053 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123593]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:28.053 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:27:27 smithi178 sudo[123593]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:28.102 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:27 smithi186 sudo[144641]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:28.102 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:27 smithi186 sudo[144641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:28.102 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:27 smithi186 sudo[144641]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:28.102 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:27 smithi186 sudo[144641]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:28.425 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146369]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:28.425 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146369]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:28.425 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146369]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:28.425 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146369]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:28.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: osdmap e22: 3 total, 3 up, 3 in 2023-10-16T14:27:28.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-16T14:27:28.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]: dispatch 2023-10-16T14:27:28.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]': finished 2023-10-16T14:27:28.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: osdmap e23: 4 total, 3 up, 4 in 2023-10-16T14:27:28.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:28.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: osdmap e22: 3 total, 3 up, 3 in 2023-10-16T14:27:28.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-16T14:27:28.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]: dispatch 2023-10-16T14:27:28.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]': finished 2023-10-16T14:27:28.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: osdmap e23: 4 total, 3 up, 4 in 2023-10-16T14:27:28.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:28.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:28 smithi186 sudo[144644]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:28.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:28 smithi186 sudo[144644]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:28.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:28 smithi186 sudo[144644]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:28.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:27:28 smithi186 sudo[144644]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:28.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: osdmap e22: 3 total, 3 up, 3 in 2023-10-16T14:27:28.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-10-16T14:27:28.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]: dispatch 2023-10-16T14:27:28.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1164140051' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2b6e4a7c-b16b-4da0-a854-7d054ac16422"}]': finished 2023-10-16T14:27:28.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: osdmap e23: 4 total, 3 up, 4 in 2023-10-16T14:27:28.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:28.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:28.924 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146372]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:28.925 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146372]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:28.925 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146372]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:28.925 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:27:28 smithi187 sudo[146372]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:29.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[123864]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[123864]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[123864]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[123864]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:29.981 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3171794086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:29.981 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:29.981 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:29.981 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 sudo[146375]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:29.982 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 sudo[146375]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:29.982 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 sudo[146375]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:29.982 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:29 smithi187 sudo[146375]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[124056]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[124056]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[124056]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 sudo[124056]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3171794086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:30.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:30.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:29 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:29 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3171794086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:29 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 sudo[146378]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 sudo[146378]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 sudo[146378]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 sudo[146378]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:30.714 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144647]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-10-16T14:27:30.714 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144647]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:30.715 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144647]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:30.715 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144647]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:30 smithi178 ceph-mon[98495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144650]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-10-16T14:27:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144650]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-10-16T14:27:31.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144650]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-10-16T14:27:31.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 sudo[144650]: pam_unix(sudo:session): session closed for user root 2023-10-16T14:27:31.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:31.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:30 smithi186 ceph-mon[134892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-10-16T14:27:31.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-10-16T14:27:31.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-10-16T14:27:31.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-10-16T14:27:31.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:30 smithi187 ceph-mon[135297]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-10-16T14:27:31.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:31 smithi178 ceph-mon[98495]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:31 smithi186 ceph-mon[134892]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:32.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:31 smithi187 ceph-mon[135297]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:33 smithi178 ceph-mon[98495]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:33 smithi186 ceph-mon[134892]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:33 smithi187 ceph-mon[135297]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:36.050 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:35 smithi178 ceph-mon[98495]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-16T14:27:36.051 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-16T14:27:36.051 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:35 smithi186 ceph-mon[134892]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-16T14:27:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-16T14:27:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:35 smithi187 ceph-mon[135297]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2023-10-16T14:27:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-10-16T14:27:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:37.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:36 smithi178 ceph-mon[98495]: Deploying daemon osd.3 on smithi178 2023-10-16T14:27:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:36 smithi186 ceph-mon[134892]: Deploying daemon osd.3 on smithi178 2023-10-16T14:27:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:36 smithi187 ceph-mon[135297]: Deploying daemon osd.3 on smithi178 2023-10-16T14:27:37.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:37 smithi178 ceph-mon[98495]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-16T14:27:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:37 smithi186 ceph-mon[134892]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-16T14:27:38.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:37 smithi187 ceph-mon[135297]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 10 op/s 2023-10-16T14:27:40.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:39 smithi178 ceph-mon[98495]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-16T14:27:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:39 smithi186 ceph-mon[134892]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-16T14:27:40.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:39 smithi187 ceph-mon[135297]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2023-10-16T14:27:41.240 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:41 smithi178 ceph-mon[98495]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:41.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:41 smithi187 ceph-mon[135297]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:41 smithi186 ceph-mon[134892]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:42.701 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.701 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.702 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:42.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:42.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:43 smithi178 ceph-mon[98495]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:43.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:43 smithi187 ceph-mon[135297]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:43 smithi186 ceph-mon[134892]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:44.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.689+0000 7f6aacff9700 1 -- 172.21.15.178:0/3752974102 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6ab00612f0 con 0x7f6a980a1470 2023-10-16T14:27:44.694 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 3 on host 'smithi178' 2023-10-16T14:27:44.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.692+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 msgr2=0x7f6a980a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:44.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.692+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 0x7f6a980a3930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400b580 tx=0x7f6aa4005fb0 comp rx=0 tx=0).stop 2023-10-16T14:27:44.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 msgr2=0x7f6ab0196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:44.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000e970 tx=0x7f6aa000ec80 comp rx=0 tx=0).stop 2023-10-16T14:27:44.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 shutdown_connections 2023-10-16T14:27:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a980a1470 0x7f6a980a3930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6ab0102b10 0x7f6ab019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab0100f70 0x7f6ab0196c30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 --2- 172.21.15.178:0/3752974102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ab0102170 0x7f6ab0197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:44.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 >> 172.21.15.178:0/3752974102 conn(0x7f6ab00fc4d0 msgr2=0x7f6ab00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:27:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 shutdown_connections 2023-10-16T14:27:44.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:27:44.693+0000 7f6ab5d49700 1 -- 172.21.15.178:0/3752974102 wait complete. 2023-10-16T14:27:45.102 DEBUG:teuthology.orchestra.run.smithi178:osd.3> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.3.service 2023-10-16T14:27:45.107 INFO:tasks.cephadm:Deploying osd.4 on smithi186 with /dev/vg_nvme/lv_3... 2023-10-16T14:27:45.107 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-16T14:27:45.179 INFO:journalctl@ceph.osd.3.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:27:45.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:45.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:45.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:45.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:45.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:45.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:45.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:45.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:45.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-10-16T14:27:45.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:45.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:27:45.676 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:27:45.797 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:27:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:27:45.386+0000 7f27b6aa7640 -1 osd.3 0 log_to_monitors true 2023-10-16T14:27:46.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:46 smithi178 ceph-mon[98495]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:46.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:46 smithi178 ceph-mon[98495]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:46.409 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:46 smithi186 ceph-mon[134892]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:46.410 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:46 smithi186 ceph-mon[134892]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:46.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:46 smithi187 ceph-mon[135297]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:46 smithi187 ceph-mon[135297]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-10-16T14:27:47.147 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-16T14:27:47.147 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-16T14:27:47.148 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-16T14:27:47.148 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-16T14:27:47.148 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0155185 s, 676 MB/s 2023-10-16T14:27:47.148 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-16T14:27:47.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-16T14:27:47.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: osdmap e24: 4 total, 3 up, 4 in 2023-10-16T14:27:47.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:47.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:47 smithi187 ceph-mon[135297]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:47.506 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-16T14:27:47.507 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: osdmap e24: 4 total, 3 up, 4 in 2023-10-16T14:27:47.507 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.507 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:47.507 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.508 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:47 smithi186 ceph-mon[134892]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:47.520 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_3 2023-10-16T14:27:47.553 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:27:47.163+0000 7f27ac91f700 -1 osd.3 0 waiting for initial osdmap 2023-10-16T14:27:47.553 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:27:47.169+0000 7f27a570e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:27:47.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-10-16T14:27:47.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: osdmap e24: 4 total, 3 up, 4 in 2023-10-16T14:27:47.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: from='osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:47.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:27:47.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:47 smithi178 ceph-mon[98495]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:48.080 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:27:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:48 smithi178 ceph-mon[98495]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:27:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:48 smithi178 ceph-mon[98495]: osdmap e25: 4 total, 3 up, 4 in 2023-10-16T14:27:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:48 smithi187 ceph-mon[135297]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:27:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:48 smithi187 ceph-mon[135297]: osdmap e25: 4 total, 3 up, 4 in 2023-10-16T14:27:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:48.505 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:48 smithi186 ceph-mon[134892]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:27:48.505 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:48 smithi186 ceph-mon[134892]: osdmap e25: 4 total, 3 up, 4 in 2023-10-16T14:27:48.506 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:48.506 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:49.307 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 -- 172.21.15.186:0/3418442929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 msgr2=0x7ff81c1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 --2- 172.21.15.186:0/3418442929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c1082b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b0a0 tx=0x7ff81400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 -- 172.21.15.186:0/3418442929 shutdown_connections 2023-10-16T14:27:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 --2- 172.21.15.186:0/3418442929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c1082b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 --2- 172.21.15.186:0/3418442929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 0x7ff81c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.308 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 --2- 172.21.15.186:0/3418442929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 -- 172.21.15.186:0/3418442929 >> 172.21.15.186:0/3418442929 conn(0x7ff81c0fb3f0 msgr2=0x7ff81c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:27:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 -- 172.21.15.186:0/3418442929 shutdown_connections 2023-10-16T14:27:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.302+0000 7ff82234b700 1 -- 172.21.15.186:0/3418442929 wait complete. 2023-10-16T14:27:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 Processor -- start 2023-10-16T14:27:49.309 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 -- start start 2023-10-16T14:27:49.310 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 0x7ff81c195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:49.310 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:49.310 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:49.310 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c19d770 con 0x7ff81c100fd0 2023-10-16T14:27:49.310 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c19d8b0 con 0x7ff81c101990 2023-10-16T14:27:49.311 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff82234b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c19da70 con 0x7ff81c0ffe70 2023-10-16T14:27:49.311 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff8208e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:49.311 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff8208e8700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47142/0 (socket says 172.21.15.186:47142) 2023-10-16T14:27:49.311 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff8208e8700 1 -- 172.21.15.186:0/560852833 learned_addr learned my addr 172.21.15.186:0/560852833 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T14:27:49.311 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.303+0000 7ff81bfff700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 0x7ff81c195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:49.312 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 -- 172.21.15.186:0/560852833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 msgr2=0x7ff81c195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:49.312 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 0x7ff81c195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.312 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 -- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 msgr2=0x7ff81c196160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:27:49.312 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff813fff700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:49.312 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c196160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:27:49.313 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 -- 172.21.15.186:0/560852833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814009d00 con 0x7ff81c101990 2023-10-16T14:27:49.313 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff813fff700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:27:49.313 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff8208e8700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b830 tx=0x7ff814004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:49.313 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e040 con 0x7ff81c101990 2023-10-16T14:27:49.313 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81401db50 con 0x7ff81c101990 2023-10-16T14:27:49.314 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c0689a0 con 0x7ff81c101990 2023-10-16T14:27:49.314 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81401bbd0 con 0x7ff81c101990 2023-10-16T14:27:49.314 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.304+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c068ea0 con 0x7ff81c101990 2023-10-16T14:27:49.315 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.305+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c10b680 con 0x7ff81c101990 2023-10-16T14:27:49.315 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.307+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff81401dcc0 con 0x7ff81c101990 2023-10-16T14:27:49.316 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.307+0000 7ff819ffb700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 0x7ff7fc0a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:27:49.316 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.307+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7ff8140cdd60 con 0x7ff81c101990 2023-10-16T14:27:49.316 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.307+0000 7ff81bfff700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 0x7ff7fc0a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:27:49.316 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.308+0000 7ff81bfff700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 0x7ff7fc0a5b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff81c197100 tx=0x7ff80c0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:27:49.316 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.310+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff814098650 con 0x7ff81c101990 2023-10-16T14:27:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:27:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:27:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] boot 2023-10-16T14:27:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: osdmap e26: 4 total, 4 up, 4 in 2023-10-16T14:27:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi178 2023-10-16T14:27:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi178 to 8635M 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:49.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:49 smithi186 ceph-mon[134892]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:27:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:27:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] boot 2023-10-16T14:27:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: osdmap e26: 4 total, 4 up, 4 in 2023-10-16T14:27:49.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:49.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi178 2023-10-16T14:27:49.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi178 to 8635M 2023-10-16T14:27:49.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:49.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:49.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:49 smithi187 ceph-mon[135297]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:49.509 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:27:49.506+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff81c069630 con 0x7ff7fc0a3690 2023-10-16T14:27:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:27:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:27:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: osd.3 [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] boot 2023-10-16T14:27:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: osdmap e26: 4 total, 4 up, 4 in 2023-10-16T14:27:49.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-10-16T14:27:49.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi178 2023-10-16T14:27:49.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:27:49.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi178 to 8635M 2023-10-16T14:27:49.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:49.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:27:49.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:27:49.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:49 smithi178 ceph-mon[98495]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2023-10-16T14:27:50.509 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:50 smithi186 ceph-mon[134892]: osdmap e27: 4 total, 4 up, 4 in 2023-10-16T14:27:50.509 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:50 smithi186 ceph-mon[134892]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:50.509 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:50.509 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:50.510 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:50.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:50 smithi178 ceph-mon[98495]: osdmap e27: 4 total, 4 up, 4 in 2023-10-16T14:27:50.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:50 smithi178 ceph-mon[98495]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:50.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:50.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:50.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:50.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:50 smithi187 ceph-mon[135297]: osdmap e27: 4 total, 4 up, 4 in 2023-10-16T14:27:50.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:50 smithi187 ceph-mon[135297]: from='client.24236 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:27:50.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:27:50.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:27:50.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:51.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:51 smithi178 ceph-mon[98495]: osdmap e28: 4 total, 4 up, 4 in 2023-10-16T14:27:51.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:51 smithi178 ceph-mon[98495]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:51 smithi186 ceph-mon[134892]: osdmap e28: 4 total, 4 up, 4 in 2023-10-16T14:27:51.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:51 smithi186 ceph-mon[134892]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:51 smithi187 ceph-mon[135297]: osdmap e28: 4 total, 4 up, 4 in 2023-10-16T14:27:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:51 smithi187 ceph-mon[135297]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:52.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: osdmap e29: 4 total, 4 up, 4 in 2023-10-16T14:27:52.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/3048470746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]': finished 2023-10-16T14:27:52.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: osdmap e30: 5 total, 4 up, 5 in 2023-10-16T14:27:52.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:52 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:27:52.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: osdmap e29: 4 total, 4 up, 4 in 2023-10-16T14:27:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/3048470746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]': finished 2023-10-16T14:27:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: osdmap e30: 5 total, 4 up, 5 in 2023-10-16T14:27:52.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:52 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:27:52.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: osdmap e29: 4 total, 4 up, 4 in 2023-10-16T14:27:52.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/3048470746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]: dispatch 2023-10-16T14:27:52.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "47410335-85bd-4755-92ee-1203b0cef71f"}]': finished 2023-10-16T14:27:52.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: osdmap e30: 5 total, 4 up, 5 in 2023-10-16T14:27:52.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:52 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:27:53.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:53 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/3906589389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:53.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:53 smithi178 ceph-mon[98495]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:53.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:53 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/3906589389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:53.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:53 smithi186 ceph-mon[134892]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:53.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:53 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/3906589389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:27:53.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:53 smithi187 ceph-mon[135297]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:27:55.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:55 smithi186 ceph-mon[134892]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-10-16T14:27:55.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:55 smithi187 ceph-mon[135297]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-10-16T14:27:56.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:55 smithi178 ceph-mon[98495]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2023-10-16T14:27:57.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:57 smithi187 ceph-mon[135297]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-10-16T14:27:57.993 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:57 smithi186 ceph-mon[134892]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-10-16T14:27:58.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:57 smithi178 ceph-mon[98495]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2023-10-16T14:27:58.894 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:58 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-16T14:27:58.894 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:58 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:58.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:58 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-16T14:27:58.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:58 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:59.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:58 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-10-16T14:27:59.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:58 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:27:59.722 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:59 smithi186 ceph-mon[134892]: Deploying daemon osd.4 on smithi186 2023-10-16T14:27:59.722 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:27:59 smithi186 ceph-mon[134892]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:27:59.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:59 smithi187 ceph-mon[135297]: Deploying daemon osd.4 on smithi186 2023-10-16T14:27:59.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:27:59 smithi187 ceph-mon[135297]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:28:00.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:59 smithi178 ceph-mon[98495]: Deploying daemon osd.4 on smithi186 2023-10-16T14:28:00.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:27:59 smithi178 ceph-mon[98495]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:28:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:01 smithi186 ceph-mon[134892]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-16T14:28:01.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:01 smithi187 ceph-mon[135297]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-16T14:28:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:01 smithi178 ceph-mon[98495]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-10-16T14:28:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:03 smithi186 ceph-mon[134892]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-16T14:28:03.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:03.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:03 smithi178 ceph-mon[98495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-16T14:28:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:03 smithi187 ceph-mon[135297]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-10-16T14:28:05.021 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.017+0000 7ff819ffb700 1 -- 172.21.15.186:0/560852833 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff81c069630 con 0x7ff7fc0a3690 2023-10-16T14:28:05.021 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 4 on host 'smithi186' 2023-10-16T14:28:05.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.019+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 msgr2=0x7ff7fc0a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:05.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.019+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 0x7ff7fc0a5b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff81c197100 tx=0x7ff80c0073d0 comp rx=0 tx=0).stop 2023-10-16T14:28:05.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 msgr2=0x7ff81c19d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:05.023 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b830 tx=0x7ff814004c80 comp rx=0 tx=0).stop 2023-10-16T14:28:05.024 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 shutdown_connections 2023-10-16T14:28:05.024 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7fc0a3690 0x7ff7fc0a5b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:05.024 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff81c101990 0x7ff81c19d1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:05.024 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c0ffe70 0x7ff81c195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:05.024 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 --2- 172.21.15.186:0/560852833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff81c100fd0 0x7ff81c196160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:05.025 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 >> 172.21.15.186:0/560852833 conn(0x7ff81c0fb3f0 msgr2=0x7ff81c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:05.025 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 shutdown_connections 2023-10-16T14:28:05.025 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:05.020+0000 7ff82234b700 1 -- 172.21.15.186:0/560852833 wait complete. 2023-10-16T14:28:05.465 DEBUG:teuthology.orchestra.run.smithi186:osd.4> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.4.service 2023-10-16T14:28:05.467 INFO:tasks.cephadm:Deploying osd.5 on smithi187 with /dev/vg_nvme/lv_3... 2023-10-16T14:28:05.468 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-10-16T14:28:05.497 INFO:journalctl@ceph.osd.4.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:28:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:28:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:05.972 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:28:05.973 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.973 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.973 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:05.973 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:05.973 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.974 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.974 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:05.974 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:06.033 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:28:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:28:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:06.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:06.706 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:28:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:28:06.393+0000 7feb89224640 -1 osd.4 0 log_to_monitors true 2023-10-16T14:28:06.852 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:06 smithi187 ceph-mon[135297]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:06.852 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:06 smithi187 ceph-mon[135297]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:07.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:06 smithi178 ceph-mon[98495]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:07.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:06 smithi178 ceph-mon[98495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:06 smithi186 ceph-mon[134892]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:06 smithi186 ceph-mon[134892]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-10-16T14:28:07.561 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-10-16T14:28:07.561 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-10-16T14:28:07.562 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-10-16T14:28:07.562 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-10-16T14:28:07.562 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113183 s, 926 MB/s 2023-10-16T14:28:07.562 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-10-16T14:28:07.858 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:07.859 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-16T14:28:07.859 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: osdmap e31: 5 total, 4 up, 5 in 2023-10-16T14:28:07.859 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:07.859 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:07.859 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:07 smithi187 ceph-mon[135297]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:07.922 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:28:07.708+0000 7feb7f09c700 -1 osd.4 0 waiting for initial osdmap 2023-10-16T14:28:07.923 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:28:07.715+0000 7feb78e8d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:28:07.923 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:07.923 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-16T14:28:07.923 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: osdmap e31: 5 total, 4 up, 5 in 2023-10-16T14:28:07.924 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:07.924 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:07.924 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:07 smithi186 ceph-mon[134892]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:08.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-10-16T14:28:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: osdmap e31: 5 total, 4 up, 5 in 2023-10-16T14:28:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: from='osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:07 smithi178 ceph-mon[98495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:28:08.178 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi187:vg_nvme/lv_3 2023-10-16T14:28:08.734 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:28:08.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:28:08.987 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: osdmap e32: 5 total, 4 up, 5 in 2023-10-16T14:28:08.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:08.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:08.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:08.988 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:08.989 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:28:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: osdmap e32: 5 total, 4 up, 5 in 2023-10-16T14:28:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:09.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:09.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:09.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:28:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: osdmap e32: 5 total, 4 up, 5 in 2023-10-16T14:28:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:09.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:09.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:09.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:10.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 -- 172.21.15.187:0/4148309859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 msgr2=0x7f65a8101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:10.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 --2- 172.21.15.187:0/4148309859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a8101450 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009990 tx=0x7f65a4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 -- 172.21.15.187:0/4148309859 shutdown_connections 2023-10-16T14:28:10.016 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 --2- 172.21.15.187:0/4148309859 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a8101990 0x7f65a81082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 --2- 172.21.15.187:0/4148309859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a80ffe70 0x7f65a8100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 --2- 172.21.15.187:0/4148309859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a8101450 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 -- 172.21.15.187:0/4148309859 >> 172.21.15.187:0/4148309859 conn(0x7f65a80fb3f0 msgr2=0x7f65a80fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:10.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 -- 172.21.15.187:0/4148309859 shutdown_connections 2023-10-16T14:28:10.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.013+0000 7f65afdba700 1 -- 172.21.15.187:0/4148309859 wait complete. 2023-10-16T14:28:10.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 Processor -- start 2023-10-16T14:28:10.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 -- start start 2023-10-16T14:28:10.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a80ffe70 0x7f65a8196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:10.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:10.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a8101990 0x7f65a819e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:10.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819e830 con 0x7f65a8101990 2023-10-16T14:28:10.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819e970 con 0x7f65a80ffe70 2023-10-16T14:28:10.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65afdba700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a819eb30 con 0x7f65a8100fd0 2023-10-16T14:28:10.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:10.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33640/0 (socket says 172.21.15.187:33640) 2023-10-16T14:28:10.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 -- 172.21.15.187:0/2808289910 learned_addr learned my addr 172.21.15.187:0/2808289910 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:28:10.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ae357700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a8101990 0x7f65a819e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:10.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65adb56700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a80ffe70 0x7f65a8196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:10.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 -- 172.21.15.187:0/2808289910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a80ffe70 msgr2=0x7f65a8196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:10.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a80ffe70 0x7f65a8196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 -- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a8101990 msgr2=0x7f65a819e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:10.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a8101990 0x7f65a819e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 -- 172.21.15.187:0/2808289910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40095f0 con 0x7f65a8100fd0 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.014+0000 7f65ad355700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65a400f740 tx=0x7f65a400f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.015+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a401d070 con 0x7f65a8100fd0 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.015+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65a400fd40 con 0x7f65a8100fd0 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.015+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4005720 con 0x7f65a8100fd0 2023-10-16T14:28:10.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.015+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a819edb0 con 0x7f65a8100fd0 2023-10-16T14:28:10.024 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.015+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a819f220 con 0x7f65a8100fd0 2023-10-16T14:28:10.024 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.016+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a810c820 con 0x7f65a8100fd0 2023-10-16T14:28:10.024 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.017+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f65a4004800 con 0x7f65a8100fd0 2023-10-16T14:28:10.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.017+0000 7f659effd700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 0x7f65940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:10.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.017+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f65a40ccfe0 con 0x7f65a8100fd0 2023-10-16T14:28:10.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.018+0000 7f65adb56700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 0x7f65940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:10.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.018+0000 7f65adb56700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 0x7f65940a3940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:10.026 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.022+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65a4097740 con 0x7f65a8100fd0 2023-10-16T14:28:10.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:28:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:28:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi186 2023-10-16T14:28:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-16T14:28:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] boot 2023-10-16T14:28:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: osdmap e33: 5 total, 5 up, 5 in 2023-10-16T14:28:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi186 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] boot 2023-10-16T14:28:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: osdmap e33: 5 total, 5 up, 5 in 2023-10-16T14:28:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:28:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:28:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi186 2023-10-16T14:28:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi186 to 8584M 2023-10-16T14:28:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-10-16T14:28:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: osd.4 [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] boot 2023-10-16T14:28:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: osdmap e33: 5 total, 5 up, 5 in 2023-10-16T14:28:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-10-16T14:28:10.222 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:10.220+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f65a80612f0 con 0x7f65940a1480 2023-10-16T14:28:11.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: osdmap e34: 5 total, 5 up, 5 in 2023-10-16T14:28:11.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:11.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:11.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:11.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:11.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:11 smithi178 ceph-mon[98495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:11.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: osdmap e34: 5 total, 5 up, 5 in 2023-10-16T14:28:11.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:11.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:11.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:11.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:11.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:11 smithi186 ceph-mon[134892]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:11.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: osdmap e34: 5 total, 5 up, 5 in 2023-10-16T14:28:11.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:11.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:11.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:11.383 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:11.384 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:11 smithi187 ceph-mon[135297]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:13.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/1898004813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:13.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:13.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]': finished 2023-10-16T14:28:13.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: osdmap e35: 6 total, 5 up, 6 in 2023-10-16T14:28:13.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:13.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:13 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/1537281250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/1898004813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]': finished 2023-10-16T14:28:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: osdmap e35: 6 total, 5 up, 6 in 2023-10-16T14:28:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:13 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/1537281250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/1898004813' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]: dispatch 2023-10-16T14:28:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038ae36b-4552-4926-af63-dc0498229ee6"}]': finished 2023-10-16T14:28:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: osdmap e35: 6 total, 5 up, 6 in 2023-10-16T14:28:14.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:14.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:13 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/1537281250' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:15 smithi187 ceph-mon[135297]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:15 smithi178 ceph-mon[98495]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:15 smithi186 ceph-mon[134892]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:17.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:17 smithi187 ceph-mon[135297]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:17 smithi178 ceph-mon[98495]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:17 smithi186 ceph-mon[134892]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:19.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:19 smithi187 ceph-mon[135297]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:19.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-16T14:28:19.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:19 smithi178 ceph-mon[98495]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:20.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-16T14:28:20.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:19 smithi186 ceph-mon[134892]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-10-16T14:28:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:20.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:20 smithi187 ceph-mon[135297]: Deploying daemon osd.5 on smithi187 2023-10-16T14:28:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:20 smithi178 ceph-mon[98495]: Deploying daemon osd.5 on smithi187 2023-10-16T14:28:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:20 smithi186 ceph-mon[134892]: Deploying daemon osd.5 on smithi187 2023-10-16T14:28:21.888 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:21 smithi187 ceph-mon[135297]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:22.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:21 smithi178 ceph-mon[98495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:21 smithi186 ceph-mon[134892]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:23.876 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:23 smithi187 ceph-mon[135297]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:23 smithi178 ceph-mon[98495]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:23 smithi186 ceph-mon[134892]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:25.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:25.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:25 smithi187 ceph-mon[135297]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:25.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:25.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:25 smithi178 ceph-mon[98495]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:25.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:25.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:25.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:25 smithi186 ceph-mon[134892]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:26.936 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.932+0000 7f659effd700 1 -- 172.21.15.187:0/2808289910 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f65a80612f0 con 0x7f65940a1480 2023-10-16T14:28:26.936 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 5 on host 'smithi187' 2023-10-16T14:28:26.938 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 msgr2=0x7f65940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:26.938 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 0x7f65940a3940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6598006fd0 tx=0x7f6598008040 comp rx=0 tx=0).stop 2023-10-16T14:28:26.939 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 msgr2=0x7f65a81971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:26.939 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f65a400f740 tx=0x7f65a400f820 comp rx=0 tx=0).stop 2023-10-16T14:28:26.939 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 shutdown_connections 2023-10-16T14:28:26.939 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65940a1480 0x7f65940a3940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:26.939 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65a8101990 0x7f65a819e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:26.940 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65a80ffe70 0x7f65a8196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:26.940 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 --2- 172.21.15.187:0/2808289910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65a8100fd0 0x7f65a81971f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:26.940 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 >> 172.21.15.187:0/2808289910 conn(0x7f65a80fb3f0 msgr2=0x7f65a80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:26.940 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 shutdown_connections 2023-10-16T14:28:26.940 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:28:26.935+0000 7f65afdba700 1 -- 172.21.15.187:0/2808289910 wait complete. 2023-10-16T14:28:27.419 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:27.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:27.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.420 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:27.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.421 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:27.422 DEBUG:teuthology.orchestra.run.smithi187:osd.5> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.5.service 2023-10-16T14:28:27.424 INFO:tasks.cephadm:Deploying osd.6 on smithi178 with /dev/vg_nvme/lv_2... 2023-10-16T14:28:27.425 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-16T14:28:27.467 INFO:journalctl@ceph.osd.5.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:28:27.513 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.513 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.513 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:27.513 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:27.514 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.514 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:27.514 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.514 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.514 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:27.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:27.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:27.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:27.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:28.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:28:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:28:27.915+0000 7f83c7a01640 -1 osd.5 0 log_to_monitors true 2023-10-16T14:28:28.181 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:28:28.452 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:28 smithi178 ceph-mon[98495]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:28.453 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:28 smithi178 ceph-mon[98495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:28.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:28 smithi187 ceph-mon[135297]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:28.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:28 smithi187 ceph-mon[135297]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:28.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:28 smithi186 ceph-mon[134892]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:28.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:28 smithi186 ceph-mon[134892]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-10-16T14:28:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-16T14:28:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: osdmap e36: 6 total, 5 up, 6 in 2023-10-16T14:28:29.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:29.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:29 smithi178 ceph-mon[98495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:29.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-16T14:28:29.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: osdmap e36: 6 total, 5 up, 6 in 2023-10-16T14:28:29.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:29.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:29 smithi186 ceph-mon[134892]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:29.632 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:28:29.212+0000 7f83bd078700 -1 osd.5 0 waiting for initial osdmap 2023-10-16T14:28:29.633 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:28:29.221+0000 7f83b5666700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:28:29.633 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-10-16T14:28:29.633 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: from='osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.634 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: osdmap e36: 6 total, 5 up, 6 in 2023-10-16T14:28:29.634 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:29.634 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:28:29.634 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:29 smithi187 ceph-mon[135297]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-10-16T14:28:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-16T14:28:30.079 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-16T14:28:30.080 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-16T14:28:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-16T14:28:30.080 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0323737 s, 324 MB/s 2023-10-16T14:28:30.080 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-16T14:28:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:28:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: osdmap e37: 6 total, 5 up, 6 in 2023-10-16T14:28:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:30.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:30.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] boot 2023-10-16T14:28:30.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: osdmap e38: 6 total, 6 up, 6 in 2023-10-16T14:28:30.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:28:30.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: osdmap e37: 6 total, 5 up, 6 in 2023-10-16T14:28:30.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:30.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:30.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] boot 2023-10-16T14:28:30.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: osdmap e38: 6 total, 6 up, 6 in 2023-10-16T14:28:30.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.559 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2023-10-16T14:28:30.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:28:30.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: osdmap e37: 6 total, 5 up, 6 in 2023-10-16T14:28:30.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:30.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:30.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:30.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:30.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:30.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: osd.5 [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] boot 2023-10-16T14:28:30.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: osdmap e38: 6 total, 6 up, 6 in 2023-10-16T14:28:30.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-10-16T14:28:31.298 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:28:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:28:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:28:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi187 2023-10-16T14:28:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi187 to 8584M 2023-10-16T14:28:31.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:31.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:31 smithi178 ceph-mon[98495]: osdmap e39: 6 total, 6 up, 6 in 2023-10-16T14:28:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:28:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:28:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi187 2023-10-16T14:28:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi187 to 8584M 2023-10-16T14:28:31.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:31.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:31 smithi186 ceph-mon[134892]: osdmap e39: 6 total, 6 up, 6 in 2023-10-16T14:28:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:28:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:28:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi187 2023-10-16T14:28:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi187 to 8584M 2023-10-16T14:28:31.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:31.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:31 smithi187 ceph-mon[135297]: osdmap e39: 6 total, 6 up, 6 in 2023-10-16T14:28:32.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1318038148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac102170 msgr2=0x7ff3ac1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1318038148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac102170 0x7ff3ac1025d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00099c0 tx=0x7ff3a0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1318038148 shutdown_connections 2023-10-16T14:28:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1318038148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102b10 0x7ff3ac1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1318038148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac100f70 0x7ff3ac101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1318038148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac102170 0x7ff3ac1025d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1318038148 >> 172.21.15.178:0/1318038148 conn(0x7ff3ac0fc4d0 msgr2=0x7ff3ac0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:32.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1318038148 shutdown_connections 2023-10-16T14:28:32.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.816+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1318038148 wait complete. 2023-10-16T14:28:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.817+0000 7ff3b20c7700 1 Processor -- start 2023-10-16T14:28:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 -- start start 2023-10-16T14:28:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac100f70 0x7ff3ac196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:32.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 0x7ff3ac197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac19e780 con 0x7ff3ac100f70 2023-10-16T14:28:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac19e8c0 con 0x7ff3ac102b10 2023-10-16T14:28:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3b20c7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac19ea80 con 0x7ff3ac102170 2023-10-16T14:28:32.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3ab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac100f70 0x7ff3ac196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3abfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:45100/0 (socket says 172.21.15.178:45100) 2023-10-16T14:28:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3abfff700 1 -- 172.21.15.178:0/1994650827 learned_addr learned my addr 172.21.15.178:0/1994650827 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:28:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.818+0000 7ff3aaffd700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 0x7ff3ac197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:32.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 -- 172.21.15.178:0/1994650827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 msgr2=0x7ff3ac197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 0x7ff3ac197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 -- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac100f70 msgr2=0x7ff3ac196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac100f70 0x7ff3ac196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 -- 172.21.15.178:0/1994650827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00095f0 con 0x7ff3ac102b10 2023-10-16T14:28:32.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3aaffd700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 0x7ff3ac197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:28:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3abfff700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400ef30 tx=0x7ff3a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4018070 con 0x7ff3ac102b10 2023-10-16T14:28:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3a400f040 con 0x7ff3ac102b10 2023-10-16T14:28:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ac19ed00 con 0x7ff3ac102b10 2023-10-16T14:28:32.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a400cdb0 con 0x7ff3ac102b10 2023-10-16T14:28:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.819+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac19f1b0 con 0x7ff3ac102b10 2023-10-16T14:28:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.820+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac108510 con 0x7ff3ac102b10 2023-10-16T14:28:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.822+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff3a4003a00 con 0x7ff3ac102b10 2023-10-16T14:28:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.824+0000 7ff3a8ff9700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 0x7ff3940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.824+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7ff3a40cc440 con 0x7ff3ac102b10 2023-10-16T14:28:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.824+0000 7ff3ab7fe700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 0x7ff3940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.824+0000 7ff3ab7fe700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 0x7ff3940a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac198110 tx=0x7ff39c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:32.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:32.826+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3a4096930 con 0x7ff3ac102b10 2023-10-16T14:28:33.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:33.033+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3ac02d2e0 con 0x7ff3940a1480 2023-10-16T14:28:33.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:33 smithi187 ceph-mon[135297]: osdmap e40: 6 total, 6 up, 6 in 2023-10-16T14:28:33.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:33 smithi187 ceph-mon[135297]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:33.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:33.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:33.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:33.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:33 smithi178 ceph-mon[98495]: osdmap e40: 6 total, 6 up, 6 in 2023-10-16T14:28:33.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:33 smithi178 ceph-mon[98495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:33.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:33.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:33.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:33 smithi186 ceph-mon[134892]: osdmap e40: 6 total, 6 up, 6 in 2023-10-16T14:28:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:33 smithi186 ceph-mon[134892]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:33.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:34.379 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:34 smithi178 ceph-mon[98495]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:34.379 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:34 smithi178 ceph-mon[98495]: osdmap e41: 6 total, 6 up, 6 in 2023-10-16T14:28:34.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:34 smithi187 ceph-mon[135297]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:34.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:34 smithi187 ceph-mon[135297]: osdmap e41: 6 total, 6 up, 6 in 2023-10-16T14:28:34.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:34 smithi186 ceph-mon[134892]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:34.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:34 smithi186 ceph-mon[134892]: osdmap e41: 6 total, 6 up, 6 in 2023-10-16T14:28:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:35 smithi178 ceph-mon[98495]: osdmap e42: 6 total, 6 up, 6 in 2023-10-16T14:28:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:35 smithi178 ceph-mon[98495]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-16T14:28:35.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:35 smithi187 ceph-mon[135297]: osdmap e42: 6 total, 6 up, 6 in 2023-10-16T14:28:35.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:35 smithi187 ceph-mon[135297]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-16T14:28:35.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:35 smithi186 ceph-mon[134892]: osdmap e42: 6 total, 6 up, 6 in 2023-10-16T14:28:35.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:35 smithi186 ceph-mon[134892]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-10-16T14:28:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:36 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]: dispatch 2023-10-16T14:28:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:36 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]': finished 2023-10-16T14:28:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:36 smithi187 ceph-mon[135297]: osdmap e43: 7 total, 6 up, 7 in 2023-10-16T14:28:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:36 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:36.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:36 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]: dispatch 2023-10-16T14:28:36.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:36 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]': finished 2023-10-16T14:28:36.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:36 smithi178 ceph-mon[98495]: osdmap e43: 7 total, 6 up, 7 in 2023-10-16T14:28:36.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:36.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:36 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]: dispatch 2023-10-16T14:28:36.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:36 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4037589157' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73f9efa5-f93a-48b9-83f7-af7901b11cc3"}]': finished 2023-10-16T14:28:36.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:36 smithi186 ceph-mon[134892]: osdmap e43: 7 total, 6 up, 7 in 2023-10-16T14:28:36.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:36 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:37 smithi187 ceph-mon[135297]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-10-16T14:28:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:37 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1508520774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:37.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:37 smithi178 ceph-mon[98495]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-10-16T14:28:37.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:37 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1508520774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:37.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:37 smithi186 ceph-mon[134892]: pgmap v138: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-10-16T14:28:37.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:37 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1508520774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:28:39.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:39 smithi187 ceph-mon[135297]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:28:39.978 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:39 smithi178 ceph-mon[98495]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:28:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:39 smithi186 ceph-mon[134892]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:28:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:41 smithi187 ceph-mon[135297]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:41.544 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:41 smithi178 ceph-mon[98495]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:41 smithi186 ceph-mon[134892]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:43.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:43 smithi187 ceph-mon[135297]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:44.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:43 smithi178 ceph-mon[98495]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:44.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:43 smithi186 ceph-mon[134892]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:44.864 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:44 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-16T14:28:44.864 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:44 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:44.864 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:44 smithi178 ceph-mon[98495]: Deploying daemon osd.6 on smithi178 2023-10-16T14:28:44.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:44 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-16T14:28:44.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:44 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:44.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:44 smithi187 ceph-mon[135297]: Deploying daemon osd.6 on smithi178 2023-10-16T14:28:45.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:44 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-10-16T14:28:45.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:44 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:45.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:44 smithi186 ceph-mon[134892]: Deploying daemon osd.6 on smithi178 2023-10-16T14:28:45.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:45 smithi187 ceph-mon[135297]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:45 smithi178 ceph-mon[98495]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:46.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:45 smithi186 ceph-mon[134892]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:47.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:47 smithi187 ceph-mon[135297]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:48.014 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:47 smithi178 ceph-mon[98495]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:47 smithi186 ceph-mon[134892]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:49.837 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:49 smithi178 ceph-mon[98495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:49.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:49 smithi187 ceph-mon[135297]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:49 smithi186 ceph-mon[134892]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:50.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:50.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:50.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:50 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:51.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:51.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:51.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:50 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:51.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:51.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:51.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:50 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:51.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:51 smithi178 ceph-mon[98495]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:51 smithi187 ceph-mon[135297]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:52.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:51 smithi186 ceph-mon[134892]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.298+0000 7ff3a8ff9700 1 -- 172.21.15.178:0/1994650827 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff3ac02d2e0 con 0x7ff3940a1480 2023-10-16T14:28:53.301 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 6 on host 'smithi178' 2023-10-16T14:28:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.301+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 msgr2=0x7ff3940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.301+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 0x7ff3940a3940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac198110 tx=0x7ff39c009260 comp rx=0 tx=0).stop 2023-10-16T14:28:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.301+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 msgr2=0x7ff3ac19e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.301+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff3a400ef30 tx=0x7ff3a400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:28:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 shutdown_connections 2023-10-16T14:28:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff3940a1480 0x7ff3940a3940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff3ac102b10 0x7ff3ac19e1b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff3ac100f70 0x7ff3ac196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 --2- 172.21.15.178:0/1994650827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff3ac102170 0x7ff3ac197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 >> 172.21.15.178:0/1994650827 conn(0x7ff3ac0fc4d0 msgr2=0x7ff3ac105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 shutdown_connections 2023-10-16T14:28:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:28:53.302+0000 7ff3b20c7700 1 -- 172.21.15.178:0/1994650827 wait complete. 2023-10-16T14:28:53.825 DEBUG:teuthology.orchestra.run.smithi178:osd.6> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.6.service 2023-10-16T14:28:53.832 INFO:tasks.cephadm:Deploying osd.7 on smithi186 with /dev/vg_nvme/lv_2... 2023-10-16T14:28:53.833 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-16T14:28:53.910 INFO:journalctl@ceph.osd.6.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:28:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:54.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:54.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:53 smithi178 ceph-mon[98495]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:54.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:54.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:54.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:54.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:53 smithi186 ceph-mon[134892]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:54.415 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:28:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:54.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:28:54.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:54.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:53 smithi187 ceph-mon[135297]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-10-16T14:28:55.494 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:28:55.283+0000 7f6d15427700 -1 osd.6 0 waiting for initial osdmap 2023-10-16T14:28:55.494 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:28:55.293+0000 7f6d10a1b700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:28:55.494 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-16T14:28:55.495 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: osdmap e44: 7 total, 6 up, 7 in 2023-10-16T14:28:55.495 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:55.495 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.495 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.495 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:55 smithi178 ceph-mon[98495]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:55.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-16T14:28:55.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: osdmap e44: 7 total, 6 up, 7 in 2023-10-16T14:28:55.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:55.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:55 smithi186 ceph-mon[134892]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:55.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-10-16T14:28:55.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: osdmap e44: 7 total, 6 up, 7 in 2023-10-16T14:28:55.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:55.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: from='osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:28:55.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:55 smithi187 ceph-mon[135297]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-10-16T14:28:55.884 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-16T14:28:55.885 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-16T14:28:55.885 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-16T14:28:55.885 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-16T14:28:55.885 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011124 s, 943 MB/s 2023-10-16T14:28:55.885 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-16T14:28:56.272 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_2 2023-10-16T14:28:56.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:56 smithi178 ceph-mon[98495]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:28:56.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:56 smithi178 ceph-mon[98495]: osdmap e45: 7 total, 6 up, 7 in 2023-10-16T14:28:56.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:56 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:56 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:56 smithi186 ceph-mon[134892]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:28:56.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:56 smithi186 ceph-mon[134892]: osdmap e45: 7 total, 6 up, 7 in 2023-10-16T14:28:56.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:56 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:56 smithi187 ceph-mon[135297]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:28:56.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:56 smithi187 ceph-mon[135297]: osdmap e45: 7 total, 6 up, 7 in 2023-10-16T14:28:56.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:56 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:56 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:56.833 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:28:57.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:28:57.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:28:57.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] boot 2023-10-16T14:28:57.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: osdmap e46: 7 total, 7 up, 7 in 2023-10-16T14:28:57.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:57.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:28:57.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:57.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:28:57.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:28:57.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] boot 2023-10-16T14:28:57.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: osdmap e46: 7 total, 7 up, 7 in 2023-10-16T14:28:57.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:57.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:28:57.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:57.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:57.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:28:57.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:28:57.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: osd.6 [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] boot 2023-10-16T14:28:57.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: osdmap e46: 7 total, 7 up, 7 in 2023-10-16T14:28:57.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-10-16T14:28:57.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:28:57.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:28:57.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:57.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:57.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:28:57.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:28:58.197 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 -- 172.21.15.186:0/286619589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c0ffe70 msgr2=0x7f622c100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:58.197 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 --2- 172.21.15.186:0/286619589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c0ffe70 0x7f622c100290 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f621c009990 tx=0x7f621c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.198 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 -- 172.21.15.186:0/286619589 shutdown_connections 2023-10-16T14:28:58.198 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 --2- 172.21.15.186:0/286619589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c1082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.198 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 --2- 172.21.15.186:0/286619589 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c0ffe70 0x7f622c100290 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.198 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 --2- 172.21.15.186:0/286619589 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c100fd0 0x7f622c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 -- 172.21.15.186:0/286619589 >> 172.21.15.186:0/286619589 conn(0x7f622c0fb3f0 msgr2=0x7f622c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 -- 172.21.15.186:0/286619589 shutdown_connections 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.192+0000 7f6233d5b700 1 -- 172.21.15.186:0/286619589 wait complete. 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 Processor -- start 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 -- start start 2023-10-16T14:28:58.199 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 0x7f622c196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:58.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:58.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:58.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c19e760 con 0x7f622c0ffe70 2023-10-16T14:28:58.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c19e8a0 con 0x7f622c100fd0 2023-10-16T14:28:58.200 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6233d5b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c19ea60 con 0x7f622c101990 2023-10-16T14:28:58.201 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:58.201 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47256/0 (socket says 172.21.15.186:47256) 2023-10-16T14:28:58.201 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 -- 172.21.15.186:0/2824539052 learned_addr learned my addr 172.21.15.186:0/2824539052 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T14:28:58.201 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62322f8700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:58.201 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6231af7700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 0x7f622c196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:58.202 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 -- 172.21.15.186:0/2824539052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 msgr2=0x7f622c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:58.202 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.202 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 -- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 msgr2=0x7f622c196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:28:58.202 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 0x7f622c196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:28:58.202 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62312f6700 1 -- 172.21.15.186:0/2824539052 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c0095f0 con 0x7f622c100fd0 2023-10-16T14:28:58.203 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f62322f8700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:28:58.203 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.193+0000 7f6231af7700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 0x7f622c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:28:58.203 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f62312f6700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f622800d950 tx=0x7f622800dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:58.203 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228009b40 con 0x7f622c100fd0 2023-10-16T14:28:58.203 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c19ed40 con 0x7f622c100fd0 2023-10-16T14:28:58.204 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6228009ca0 con 0x7f622c100fd0 2023-10-16T14:28:58.204 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622800f620 con 0x7f622c100fd0 2023-10-16T14:28:58.204 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.194+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c19f180 con 0x7f622c100fd0 2023-10-16T14:28:58.205 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.195+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c10b6f0 con 0x7f622c100fd0 2023-10-16T14:28:58.205 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.196+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6228010460 con 0x7f622c100fd0 2023-10-16T14:28:58.205 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.197+0000 7f6222ffd700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 0x7f62180a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:28:58.206 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.197+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f62280cc390 con 0x7f622c100fd0 2023-10-16T14:28:58.206 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.197+0000 7f6231af7700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 0x7f62180a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:28:58.206 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.198+0000 7f6231af7700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 0x7f62180a3940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f621c000c00 tx=0x7f621c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:28:58.206 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.200+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6228096700 con 0x7f622c100fd0 2023-10-16T14:28:58.401 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:28:58.398+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f622c0fc3c0 con 0x7f62180a1480 2023-10-16T14:28:58.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:58 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi178 2023-10-16T14:28:58.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:58 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi178 to 5757M 2023-10-16T14:28:58.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:58 smithi178 ceph-mon[98495]: osdmap e47: 7 total, 7 up, 7 in 2023-10-16T14:28:58.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:58 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi178 2023-10-16T14:28:58.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:58 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi178 to 5757M 2023-10-16T14:28:58.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:58 smithi186 ceph-mon[134892]: osdmap e47: 7 total, 7 up, 7 in 2023-10-16T14:28:58.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:58 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi178 2023-10-16T14:28:58.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:58 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi178 to 5757M 2023-10-16T14:28:58.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:58 smithi187 ceph-mon[135297]: osdmap e47: 7 total, 7 up, 7 in 2023-10-16T14:28:59.529 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:59 smithi186 ceph-mon[134892]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:59.529 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:59.530 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:59.530 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:59.530 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:28:59 smithi186 ceph-mon[134892]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:28:59.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:59 smithi178 ceph-mon[98495]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:59.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:59.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:59.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:59.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:28:59 smithi178 ceph-mon[98495]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:28:59.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:59 smithi187 ceph-mon[135297]: from='client.24305 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:28:59.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:28:59.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:28:59.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:28:59.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:28:59 smithi187 ceph-mon[135297]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:01.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:01.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/2010125267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]': finished 2023-10-16T14:29:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: osdmap e48: 8 total, 7 up, 8 in 2023-10-16T14:29:01.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:01.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/2010125267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]': finished 2023-10-16T14:29:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: osdmap e48: 8 total, 7 up, 8 in 2023-10-16T14:29:01.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/2010125267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]: dispatch 2023-10-16T14:29:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc91a577-422e-4074-a0f8-35c3be1cac6d"}]': finished 2023-10-16T14:29:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: osdmap e48: 8 total, 7 up, 8 in 2023-10-16T14:29:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/925341793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:02.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/925341793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:02.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/925341793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:03.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:03 smithi187 ceph-mon[135297]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:03.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:03 smithi178 ceph-mon[98495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:03.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:03 smithi186 ceph-mon[134892]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:05 smithi187 ceph-mon[135297]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:05.936 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:05 smithi186 ceph-mon[134892]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:05 smithi178 ceph-mon[98495]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:07 smithi187 ceph-mon[135297]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-16T14:29:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:07.926 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:07 smithi186 ceph-mon[134892]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:07.926 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-16T14:29:07.926 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:07 smithi178 ceph-mon[98495]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-10-16T14:29:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:08.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:08 smithi187 ceph-mon[135297]: Deploying daemon osd.7 on smithi186 2023-10-16T14:29:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:08 smithi178 ceph-mon[98495]: Deploying daemon osd.7 on smithi186 2023-10-16T14:29:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:08 smithi186 ceph-mon[134892]: Deploying daemon osd.7 on smithi186 2023-10-16T14:29:09.774 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:09 smithi186 ceph-mon[134892]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:09.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:09 smithi187 ceph-mon[135297]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:09 smithi178 ceph-mon[98495]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:11.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:11 smithi187 ceph-mon[135297]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:11.433 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:11 smithi186 ceph-mon[134892]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:11.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:11 smithi178 ceph-mon[98495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:12.711 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:12.711 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:12.711 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:13.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:13.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:13.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:13.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:13 smithi186 ceph-mon[134892]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:13 smithi187 ceph-mon[135297]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:13 smithi178 ceph-mon[98495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:14.367 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 7 on host 'smithi186' 2023-10-16T14:29:14.368 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.363+0000 7f6222ffd700 1 -- 172.21.15.186:0/2824539052 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f622c0fc3c0 con 0x7f62180a1480 2023-10-16T14:29:14.369 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 msgr2=0x7f62180a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:14.369 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 0x7f62180a3940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f621c000c00 tx=0x7f621c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:29:14.370 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 msgr2=0x7f622c197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:14.370 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f622800d950 tx=0x7f622800dd10 comp rx=0 tx=0).stop 2023-10-16T14:29:14.370 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 shutdown_connections 2023-10-16T14:29:14.370 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62180a1480 0x7f62180a3940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:14.370 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f622c101990 0x7f622c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:14.371 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f622c0ffe70 0x7f622c196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:14.371 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 --2- 172.21.15.186:0/2824539052 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f622c100fd0 0x7f622c197150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:14.371 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 >> 172.21.15.186:0/2824539052 conn(0x7f622c0fb3f0 msgr2=0x7f622c0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:29:14.371 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 shutdown_connections 2023-10-16T14:29:14.371 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:29:14.366+0000 7f6233d5b700 1 -- 172.21.15.186:0/2824539052 wait complete. 2023-10-16T14:29:14.727 DEBUG:teuthology.orchestra.run.smithi186:osd.7> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.7.service 2023-10-16T14:29:14.730 INFO:tasks.cephadm:Deploying osd.8 on smithi187 with /dev/vg_nvme/lv_2... 2023-10-16T14:29:14.730 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-10-16T14:29:14.759 INFO:journalctl@ceph.osd.7.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:29:15.300 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:29:15.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-mon[134892]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:15.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:15 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:15 smithi187 ceph-mon[135297]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:15.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:15 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:15 smithi178 ceph-mon[98495]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:15.832 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:29:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:29:15.575+0000 7f05dfabf640 -1 osd.7 0 log_to_monitors true 2023-10-16T14:29:16.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:16 smithi187 ceph-mon[135297]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:16 smithi187 ceph-mon[135297]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:16 smithi178 ceph-mon[98495]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:16 smithi178 ceph-mon[98495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:16 smithi186 ceph-mon[134892]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:16 smithi186 ceph-mon[134892]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-10-16T14:29:16.882 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-10-16T14:29:16.882 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-10-16T14:29:16.882 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-10-16T14:29:16.883 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-10-16T14:29:16.883 INFO:teuthology.orchestra.run.smithi187.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110853 s, 946 MB/s 2023-10-16T14:29:16.883 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 2023-10-16T14:29:16.883 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-10-16T14:29:17.354 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi187:vg_nvme/lv_2 2023-10-16T14:29:17.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-16T14:29:17.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: osdmap e49: 8 total, 7 up, 8 in 2023-10-16T14:29:17.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:17.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:17 smithi187 ceph-mon[135297]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:17.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-16T14:29:17.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: osdmap e49: 8 total, 7 up, 8 in 2023-10-16T14:29:17.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:17.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:17 smithi178 ceph-mon[98495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:17.826 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:29:17.386+0000 7f05d5136700 -1 osd.7 0 waiting for initial osdmap 2023-10-16T14:29:17.826 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:29:17.396+0000 7f05cef27700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:29:17.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-10-16T14:29:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: osdmap e49: 8 total, 7 up, 8 in 2023-10-16T14:29:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: from='osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:29:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:17 smithi186 ceph-mon[134892]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-10-16T14:29:17.916 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:29:18.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:29:18.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: osdmap e50: 8 total, 7 up, 8 in 2023-10-16T14:29:18.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi186 2023-10-16T14:29:18.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-16T14:29:18.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] boot 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: osdmap e51: 8 total, 8 up, 8 in 2023-10-16T14:29:18.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:29:18.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: osdmap e50: 8 total, 7 up, 8 in 2023-10-16T14:29:18.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi186 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-16T14:29:18.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:18.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.806 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] boot 2023-10-16T14:29:18.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: osdmap e51: 8 total, 8 up, 8 in 2023-10-16T14:29:18.807 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:29:18.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: osdmap e50: 8 total, 7 up, 8 in 2023-10-16T14:29:18.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:18.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi186 2023-10-16T14:29:18.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi186 to 5722M 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.829 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.829 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.829 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:18.829 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:18.829 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:18.830 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:18.830 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: osd.7 [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] boot 2023-10-16T14:29:18.830 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: osdmap e51: 8 total, 8 up, 8 in 2023-10-16T14:29:18.830 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-10-16T14:29:19.240 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 -- 172.21.15.187:0/2717661877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44100fd0 msgr2=0x7f9b44101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:19.240 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 --2- 172.21.15.187:0/2717661877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44100fd0 0x7f9b44101450 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9b34009990 tx=0x7f9b34009ca0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.240 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 -- 172.21.15.187:0/2717661877 shutdown_connections 2023-10-16T14:29:19.241 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 --2- 172.21.15.187:0/2717661877 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b44101990 0x7f9b441082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.241 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 --2- 172.21.15.187:0/2717661877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b440ffe70 0x7f9b44100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.241 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 --2- 172.21.15.187:0/2717661877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44100fd0 0x7f9b44101450 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.241 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 -- 172.21.15.187:0/2717661877 >> 172.21.15.187:0/2717661877 conn(0x7f9b440fb3f0 msgr2=0x7f9b440fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:29:19.241 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 -- 172.21.15.187:0/2717661877 shutdown_connections 2023-10-16T14:29:19.242 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.236+0000 7f9b4909d700 1 -- 172.21.15.187:0/2717661877 wait complete. 2023-10-16T14:29:19.244 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 Processor -- start 2023-10-16T14:29:19.244 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 -- start start 2023-10-16T14:29:19.244 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b440ffe70 0x7f9b44195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:19.245 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 0x7f9b441960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:19.245 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:19.245 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d700 con 0x7f9b44100fd0 2023-10-16T14:29:19.245 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d840 con 0x7f9b440ffe70 2023-10-16T14:29:19.245 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4909d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4419d9d0 con 0x7f9b44101990 2023-10-16T14:29:19.246 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4359e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:19.246 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4359e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33754/0 (socket says 172.21.15.187:33754) 2023-10-16T14:29:19.246 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b42d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b440ffe70 0x7f9b44195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:19.246 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.237+0000 7f9b4359e700 1 -- 172.21.15.187:0/4028862326 learned_addr learned my addr 172.21.15.187:0/4028862326 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:29:19.246 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4259c700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 0x7f9b441960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:19.247 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 -- 172.21.15.187:0/4028862326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b440ffe70 msgr2=0x7f9b44195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:19.247 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b440ffe70 0x7f9b44195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.247 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 -- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 msgr2=0x7f9b441960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:19.247 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 0x7f9b441960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:19.247 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 -- 172.21.15.187:0/4028862326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b340095f0 con 0x7f9b44101990 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4259c700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 0x7f9b441960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4359e700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00add0 tx=0x7f9b3c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00f070 con 0x7f9b44101990 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b44069290 con 0x7f9b44101990 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b3c007b30 con 0x7f9b44101990 2023-10-16T14:29:19.248 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c005310 con 0x7f9b44101990 2023-10-16T14:29:19.249 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.238+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b44069790 con 0x7f9b44101990 2023-10-16T14:29:19.249 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.239+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4410b6a0 con 0x7f9b44101990 2023-10-16T14:29:19.249 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.240+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9b3c00c940 con 0x7f9b44101990 2023-10-16T14:29:19.249 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.241+0000 7f9b2bfff700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 0x7f9b2c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:19.249 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.241+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7f9b3c0cb7f0 con 0x7f9b44101990 2023-10-16T14:29:19.250 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.241+0000 7f9b42d9d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 0x7f9b2c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:19.250 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.242+0000 7f9b42d9d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 0x7f9b2c0a39c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800bcb0 tx=0x7f9b3800b3a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:29:19.250 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.245+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9b3c095980 con 0x7f9b44101990 2023-10-16T14:29:19.448 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:19.444+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b44069ec0 con 0x7f9b2c0a1500 2023-10-16T14:29:19.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:19 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:29:19.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:19 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:29:19.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:19 smithi187 ceph-mon[135297]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:19.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:19 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:29:19.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:19 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:29:19.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:19 smithi178 ceph-mon[98495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:19.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:19 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:29:19.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:19 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:29:19.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:19 smithi186 ceph-mon[134892]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:20.611 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: osdmap e52: 8 total, 8 up, 8 in 2023-10-16T14:29:20.612 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:20.612 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:20.612 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:20.612 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:20.613 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:20 smithi187 ceph-mon[135297]: osdmap e53: 8 total, 8 up, 8 in 2023-10-16T14:29:20.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: osdmap e52: 8 total, 8 up, 8 in 2023-10-16T14:29:20.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:20.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:20.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:20.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:20.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:20 smithi178 ceph-mon[98495]: osdmap e53: 8 total, 8 up, 8 in 2023-10-16T14:29:20.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: osdmap e52: 8 total, 8 up, 8 in 2023-10-16T14:29:20.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:20.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:20.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:20.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:20.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:20 smithi186 ceph-mon[134892]: osdmap e53: 8 total, 8 up, 8 in 2023-10-16T14:29:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:21 smithi187 ceph-mon[135297]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:21 smithi178 ceph-mon[98495]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:21 smithi186 ceph-mon[134892]: pgmap v170: 1 pgs: 1 remapped+peering; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:22.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: osdmap e54: 8 total, 8 up, 8 in 2023-10-16T14:29:22.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/58937226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]': finished 2023-10-16T14:29:22.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: osdmap e55: 9 total, 8 up, 9 in 2023-10-16T14:29:22.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:22 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:22.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: osdmap e54: 8 total, 8 up, 8 in 2023-10-16T14:29:22.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/58937226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]': finished 2023-10-16T14:29:22.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: osdmap e55: 9 total, 8 up, 9 in 2023-10-16T14:29:22.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:22 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:22.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: osdmap e54: 8 total, 8 up, 8 in 2023-10-16T14:29:22.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/58937226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]: dispatch 2023-10-16T14:29:22.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40ff8899-9c05-4331-8c78-e0d69b8571e6"}]': finished 2023-10-16T14:29:22.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: osdmap e55: 9 total, 8 up, 9 in 2023-10-16T14:29:22.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:22 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:23.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:23 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/134946365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:23.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:23 smithi187 ceph-mon[135297]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:23.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:23 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/134946365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:23.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:23 smithi178 ceph-mon[98495]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:23 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/134946365' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:23 smithi186 ceph-mon[134892]: pgmap v173: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:25 smithi187 ceph-mon[135297]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:26.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:25 smithi178 ceph-mon[98495]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:25 smithi186 ceph-mon[134892]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:27.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:27 smithi187 ceph-mon[135297]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:28.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:27 smithi178 ceph-mon[98495]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:27 smithi186 ceph-mon[134892]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:28.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-16T14:29:28.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:29.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-16T14:29:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-10-16T14:29:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:29.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:29 smithi187 ceph-mon[135297]: Deploying daemon osd.8 on smithi187 2023-10-16T14:29:29.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:29 smithi187 ceph-mon[135297]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:30.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:29 smithi178 ceph-mon[98495]: Deploying daemon osd.8 on smithi187 2023-10-16T14:29:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:29 smithi178 ceph-mon[98495]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:29 smithi186 ceph-mon[134892]: Deploying daemon osd.8 on smithi187 2023-10-16T14:29:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:29 smithi186 ceph-mon[134892]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:31.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:31 smithi187 ceph-mon[135297]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:31 smithi178 ceph-mon[98495]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:31 smithi186 ceph-mon[134892]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:33 smithi187 ceph-mon[135297]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:34.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:33 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:33 smithi178 ceph-mon[98495]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:33 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:33 smithi186 ceph-mon[134892]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:34.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:33 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:35.960 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 8 on host 'smithi187' 2023-10-16T14:29:35.961 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.956+0000 7f9b2bfff700 1 -- 172.21.15.187:0/4028862326 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9b44069ec0 con 0x7f9b2c0a1500 2023-10-16T14:29:35.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 msgr2=0x7f9b2c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:35.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 0x7f9b2c0a39c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800bcb0 tx=0x7f9b3800b3a0 comp rx=0 tx=0).stop 2023-10-16T14:29:35.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 msgr2=0x7f9b4419d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:35.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00add0 tx=0x7f9b3c00c330 comp rx=0 tx=0).stop 2023-10-16T14:29:35.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 shutdown_connections 2023-10-16T14:29:35.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9b2c0a1500 0x7f9b2c0a39c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:35.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b44101990 0x7f9b4419d130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:35.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9b440ffe70 0x7f9b44195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:35.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 --2- 172.21.15.187:0/4028862326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b44100fd0 0x7f9b441960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:35.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 >> 172.21.15.187:0/4028862326 conn(0x7f9b440fb3f0 msgr2=0x7f9b440fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:29:35.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 shutdown_connections 2023-10-16T14:29:35.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:29:35.959+0000 7f9b4909d700 1 -- 172.21.15.187:0/4028862326 wait complete. 2023-10-16T14:29:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:36.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:36.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:36.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:36.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:36.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:36.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:29:36.505 DEBUG:teuthology.orchestra.run.smithi187:osd.8> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.8.service 2023-10-16T14:29:36.508 INFO:tasks.cephadm:Deploying osd.9 on smithi178 with /dev/vg_nvme/lv_1... 2023-10-16T14:29:36.508 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-16T14:29:36.537 INFO:journalctl@ceph.osd.8.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:29:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:36 smithi187 ceph-mon[135297]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:36 smithi187 ceph-mon[135297]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.176 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:29:36 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:29:36.905+0000 7f61cff6c640 -1 osd.8 0 log_to_monitors true 2023-10-16T14:29:37.254 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:29:37.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:36 smithi178 ceph-mon[98495]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:36 smithi178 ceph-mon[98495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:36 smithi186 ceph-mon[134892]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:36 smithi186 ceph-mon[134892]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-10-16T14:29:37.965 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:29:37.949+0000 7f61c55e3700 -1 osd.8 0 waiting for initial osdmap 2023-10-16T14:29:37.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:37.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-16T14:29:37.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:37.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: osdmap e56: 9 total, 8 up, 9 in 2023-10-16T14:29:37.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:37.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-mon[135297]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:38.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:38.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-16T14:29:38.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:38.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: osdmap e56: 9 total, 8 up, 9 in 2023-10-16T14:29:38.092 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:38.093 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:37 smithi178 ceph-mon[98495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-10-16T14:29:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: from='osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: osdmap e56: 9 total, 8 up, 9 in 2023-10-16T14:29:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:37 smithi186 ceph-mon[134892]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:29:38.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:29:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:29:37.961+0000 7f61bf3d4700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:29:39.261 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-16T14:29:39.261 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-16T14:29:39.262 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2023-10-16T14:29:39.262 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2023-10-16T14:29:39.262 INFO:teuthology.orchestra.run.smithi178.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0345267 s, 304 MB/s 2023-10-16T14:29:39.262 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2023-10-16T14:29:39.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:38 smithi178 ceph-mon[98495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:29:39.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:38 smithi178 ceph-mon[98495]: osdmap e57: 9 total, 8 up, 9 in 2023-10-16T14:29:39.299 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.300 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:38 smithi186 ceph-mon[134892]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:29:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:38 smithi186 ceph-mon[134892]: osdmap e57: 9 total, 8 up, 9 in 2023-10-16T14:29:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:38 smithi187 ceph-mon[135297]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:29:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:38 smithi187 ceph-mon[135297]: osdmap e57: 9 total, 8 up, 9 in 2023-10-16T14:29:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.734 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2023-10-16T14:29:39.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:29:39.956 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:29:39.957 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:39.957 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.957 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] boot 2023-10-16T14:29:39.957 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: osdmap e58: 9 total, 9 up, 9 in 2023-10-16T14:29:39.957 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:39.958 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:39.958 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:39.958 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:39.958 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:39.958 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:29:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:29:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] boot 2023-10-16T14:29:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: osdmap e58: 9 total, 9 up, 9 in 2023-10-16T14:29:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:40.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:40.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:29:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:29:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: pgmap v183: 1 pgs: 1 peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-10-16T14:29:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: osd.8 [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] boot 2023-10-16T14:29:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: osdmap e58: 9 total, 9 up, 9 in 2023-10-16T14:29:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-10-16T14:29:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:29:40.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:40.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:29:40.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:29:40.469 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:29:40.988 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:40 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi187 2023-10-16T14:29:40.988 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:40 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi187 to 5722M 2023-10-16T14:29:40.988 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:40 smithi178 ceph-mon[98495]: osdmap e59: 9 total, 9 up, 9 in 2023-10-16T14:29:41.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:40 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi187 2023-10-16T14:29:41.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:40 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi187 to 5722M 2023-10-16T14:29:41.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:40 smithi186 ceph-mon[134892]: osdmap e59: 9 total, 9 up, 9 in 2023-10-16T14:29:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:40 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi187 2023-10-16T14:29:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:40 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi187 to 5722M 2023-10-16T14:29:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:40 smithi187 ceph-mon[135297]: osdmap e59: 9 total, 9 up, 9 in 2023-10-16T14:29:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 -- 172.21.15.178:0/2920617695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 msgr2=0x7f16c81093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 --2- 172.21.15.178:0/2920617695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81093c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f16c000b0d0 tx=0x7f16c000b3e0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 -- 172.21.15.178:0/2920617695 shutdown_connections 2023-10-16T14:29:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 --2- 172.21.15.178:0/2920617695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81093c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 --2- 172.21.15.178:0/2920617695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 0x7f16c8101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 --2- 172.21.15.178:0/2920617695 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 0x7f16c81025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 -- 172.21.15.178:0/2920617695 >> 172.21.15.178:0/2920617695 conn(0x7f16c80fc4d0 msgr2=0x7f16c80fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:29:41.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 -- 172.21.15.178:0/2920617695 shutdown_connections 2023-10-16T14:29:41.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.888+0000 7f16cc837700 1 -- 172.21.15.178:0/2920617695 wait complete. 2023-10-16T14:29:41.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.889+0000 7f16cc837700 1 Processor -- start 2023-10-16T14:29:41.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.889+0000 7f16cc837700 1 -- start start 2023-10-16T14:29:41.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 0x7f16c8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:41.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 0x7f16c81994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:41.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:41.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81a0b10 con 0x7f16c8102b10 2023-10-16T14:29:41.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81a0c50 con 0x7f16c8102170 2023-10-16T14:29:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16cc837700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81a0de0 con 0x7f16c8100f70 2023-10-16T14:29:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36458/0 (socket says 172.21.15.178:36458) 2023-10-16T14:29:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 -- 172.21.15.178:0/894575112 learned_addr learned my addr 172.21.15.178:0/894575112 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:29:41.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c5d9b700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 0x7f16c81994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:41.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c659c700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 0x7f16c8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:41.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 -- 172.21.15.178:0/894575112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 msgr2=0x7f16c8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:41.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 0x7f16c8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 -- 172.21.15.178:0/894575112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 msgr2=0x7f16c81994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:29:41.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 0x7f16c81994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 -- 172.21.15.178:0/894575112 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009d00 con 0x7f16c8102b10 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.890+0000 7f16c6d9d700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009440 tx=0x7f16c0009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.891+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c000e040 con 0x7f16c8102b10 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.891+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16c001dd50 con 0x7f16c8102b10 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.891+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c00123f0 con 0x7f16c8102b10 2023-10-16T14:29:41.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.891+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c81a1060 con 0x7f16c8102b10 2023-10-16T14:29:41.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.891+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c81a14d0 con 0x7f16c8102b10 2023-10-16T14:29:41.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.892+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c81935e0 con 0x7f16c8102b10 2023-10-16T14:29:41.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.894+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f16c001b550 con 0x7f16c8102b10 2023-10-16T14:29:41.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.895+0000 7f16b77fe700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 0x7f16b00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:29:41.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.895+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f16c00ce540 con 0x7f16c8102b10 2023-10-16T14:29:41.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.895+0000 7f16c659c700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 0x7f16b00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:29:41.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.896+0000 7f16c659c700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 0x7f16b00a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f16c8101fd0 tx=0x7f16b800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:29:41.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:41.897+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f16c0098520 con 0x7f16c8102b10 2023-10-16T14:29:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:29:42.110+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f16c802d130 con 0x7f16b00a1500 2023-10-16T14:29:42.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:41 smithi178 ceph-mon[98495]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:42.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:41 smithi178 ceph-mon[98495]: osdmap e60: 9 total, 9 up, 9 in 2023-10-16T14:29:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:41 smithi186 ceph-mon[134892]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:41 smithi186 ceph-mon[134892]: osdmap e60: 9 total, 9 up, 9 in 2023-10-16T14:29:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:41 smithi187 ceph-mon[135297]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:41 smithi187 ceph-mon[135297]: osdmap e60: 9 total, 9 up, 9 in 2023-10-16T14:29:43.255 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:42 smithi178 ceph-mon[98495]: osdmap e61: 9 total, 9 up, 9 in 2023-10-16T14:29:43.255 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:43.256 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:43.256 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:43.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:42 smithi186 ceph-mon[134892]: osdmap e61: 9 total, 9 up, 9 in 2023-10-16T14:29:43.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:43.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:43.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:42 smithi187 ceph-mon[135297]: osdmap e61: 9 total, 9 up, 9 in 2023-10-16T14:29:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:29:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:29:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:44.255 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:43 smithi178 ceph-mon[98495]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:44.256 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:43 smithi178 ceph-mon[98495]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:44.256 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:43 smithi178 ceph-mon[98495]: osdmap e62: 9 total, 9 up, 9 in 2023-10-16T14:29:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:43 smithi186 ceph-mon[134892]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:43 smithi186 ceph-mon[134892]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:43 smithi186 ceph-mon[134892]: osdmap e62: 9 total, 9 up, 9 in 2023-10-16T14:29:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:43 smithi187 ceph-mon[135297]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:29:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:43 smithi187 ceph-mon[135297]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:43 smithi187 ceph-mon[135297]: osdmap e62: 9 total, 9 up, 9 in 2023-10-16T14:29:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-16T14:29:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]: dispatch 2023-10-16T14:29:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]': finished 2023-10-16T14:29:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: osdmap e63: 10 total, 9 up, 10 in 2023-10-16T14:29:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:29:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2833525829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:46.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-16T14:29:46.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]: dispatch 2023-10-16T14:29:46.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]': finished 2023-10-16T14:29:46.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: osdmap e63: 10 total, 9 up, 10 in 2023-10-16T14:29:46.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:29:46.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2833525829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-10-16T14:29:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]: dispatch 2023-10-16T14:29:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1651120189' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22e44478-f2b6-4a84-a740-9bd31070b250"}]': finished 2023-10-16T14:29:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: osdmap e63: 10 total, 9 up, 10 in 2023-10-16T14:29:46.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:29:46.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2833525829' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:29:48.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:47 smithi178 ceph-mon[98495]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-16T14:29:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:47 smithi186 ceph-mon[134892]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-16T14:29:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:47 smithi187 ceph-mon[135297]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-10-16T14:29:49.290 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:49 smithi178 ceph-mon[98495]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-16T14:29:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:49 smithi186 ceph-mon[134892]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-16T14:29:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:49 smithi187 ceph-mon[135297]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-10-16T14:29:51.298 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:51 smithi178 ceph-mon[98495]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:29:51.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:51 smithi187 ceph-mon[135297]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:29:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:51 smithi186 ceph-mon[134892]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-10-16T14:29:53.910 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:53 smithi178 ceph-mon[98495]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-16T14:29:53.910 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-16T14:29:53.911 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:53 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:53.911 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:53 smithi178 ceph-mon[98495]: Deploying daemon osd.9 on smithi178 2023-10-16T14:29:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:53 smithi187 ceph-mon[135297]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-16T14:29:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-16T14:29:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:53 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:53 smithi187 ceph-mon[135297]: Deploying daemon osd.9 on smithi178 2023-10-16T14:29:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:53 smithi186 ceph-mon[134892]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-10-16T14:29:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-10-16T14:29:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:53 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:29:54.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:53 smithi186 ceph-mon[134892]: Deploying daemon osd.9 on smithi178 2023-10-16T14:29:55.894 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:55 smithi178 ceph-mon[98495]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:55.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:55 smithi187 ceph-mon[135297]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:55 smithi186 ceph-mon[134892]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:57.883 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:57 smithi178 ceph-mon[98495]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:57.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:57 smithi187 ceph-mon[135297]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:29:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:57 smithi186 ceph-mon[134892]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:00.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:59 smithi178 ceph-mon[98495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:00.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.163 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.164 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:29:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:59 smithi187 ceph-mon[135297]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:29:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:59 smithi186 ceph-mon[134892]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:29:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:01.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:00 smithi187 ceph-mon[135297]: overall HEALTH_OK 2023-10-16T14:30:01.178 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:00 smithi178 ceph-mon[98495]: overall HEALTH_OK 2023-10-16T14:30:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:00 smithi186 ceph-mon[134892]: overall HEALTH_OK 2023-10-16T14:30:02.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:01 smithi178 ceph-mon[98495]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:02.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:01 smithi187 ceph-mon[135297]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:02.271 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 9 on host 'smithi178' 2023-10-16T14:30:02.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.268+0000 7f16b77fe700 1 -- 172.21.15.178:0/894575112 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f16c802d130 con 0x7f16b00a1500 2023-10-16T14:30:02.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 msgr2=0x7f16b00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:02.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 0x7f16b00a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f16c8101fd0 tx=0x7f16b800b410 comp rx=0 tx=0).stop 2023-10-16T14:30:02.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 msgr2=0x7f16c81a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:02.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f16c0009440 tx=0x7f16c0009470 comp rx=0 tx=0).stop 2023-10-16T14:30:02.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 shutdown_connections 2023-10-16T14:30:02.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16b00a1500 0x7f16b00a39c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:02.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16c8102b10 0x7f16c81a0540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:02.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c8100f70 0x7f16c8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:02.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 --2- 172.21.15.178:0/894575112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16c8102170 0x7f16c81994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:02.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 >> 172.21.15.178:0/894575112 conn(0x7f16c80fc4d0 msgr2=0x7f16c8107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:02.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 shutdown_connections 2023-10-16T14:30:02.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:02.271+0000 7f16cc837700 1 -- 172.21.15.178:0/894575112 wait complete. 2023-10-16T14:30:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:01 smithi186 ceph-mon[134892]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:02.796 DEBUG:teuthology.orchestra.run.smithi178:osd.9> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.9.service 2023-10-16T14:30:02.801 INFO:tasks.cephadm:Deploying osd.10 on smithi186 with /dev/vg_nvme/lv_1... 2023-10-16T14:30:02.802 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-16T14:30:02.883 INFO:journalctl@ceph.osd.9.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:30:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:03.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:03.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:03.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:03 smithi178 ceph-mon[98495]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:03.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:03.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:03 smithi186 ceph-mon[134892]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:03.376 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:30:03.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:03 smithi187 ceph-mon[135297]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-10-16T14:30:04.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-mon[98495]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-16T14:30:04.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-mon[98495]: osdmap e64: 10 total, 9 up, 10 in 2023-10-16T14:30:04.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:04.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-mon[98495]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-mon[98495]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.554 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:30:04.255+0000 7f1dce992700 -1 osd.9 0 waiting for initial osdmap 2023-10-16T14:30:04.555 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:30:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:30:04.275+0000 7f1dc8783700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:30:04.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:04 smithi186 ceph-mon[134892]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-16T14:30:04.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:04 smithi186 ceph-mon[134892]: osdmap e64: 10 total, 9 up, 10 in 2023-10-16T14:30:04.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:04 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:04.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:04 smithi186 ceph-mon[134892]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:04 smithi186 ceph-mon[134892]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:04 smithi187 ceph-mon[135297]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-10-16T14:30:04.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:04 smithi187 ceph-mon[135297]: osdmap e64: 10 total, 9 up, 10 in 2023-10-16T14:30:04.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:04 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:04.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:04 smithi187 ceph-mon[135297]: from='osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:04 smithi187 ceph-mon[135297]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2023-10-16T14:30:04.882 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-16T14:30:04.883 INFO:teuthology.orchestra.run.smithi186.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-16T14:30:04.883 INFO:teuthology.orchestra.run.smithi186.stderr: stderr: 10+0 records in 2023-10-16T14:30:04.883 INFO:teuthology.orchestra.run.smithi186.stderr:10+0 records out 2023-10-16T14:30:04.883 INFO:teuthology.orchestra.run.smithi186.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147662 s, 710 MB/s 2023-10-16T14:30:04.883 INFO:teuthology.orchestra.run.smithi186.stderr:--> Zapping successful for: 2023-10-16T14:30:05.234 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi186:vg_nvme/lv_1 2023-10-16T14:30:05.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:05 smithi178 ceph-mon[98495]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:30:05.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:05 smithi178 ceph-mon[98495]: osdmap e65: 10 total, 9 up, 10 in 2023-10-16T14:30:05.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:05 smithi178 ceph-mon[98495]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:05 smithi186 ceph-mon[134892]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:30:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:05 smithi186 ceph-mon[134892]: osdmap e65: 10 total, 9 up, 10 in 2023-10-16T14:30:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:05 smithi186 ceph-mon[134892]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:05 smithi187 ceph-mon[135297]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2023-10-16T14:30:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:05 smithi187 ceph-mon[135297]: osdmap e65: 10 total, 9 up, 10 in 2023-10-16T14:30:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:05 smithi187 ceph-mon[135297]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-10-16T14:30:05.796 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:30:06.519 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:30:06.519 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:30:06.520 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.520 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] boot 2023-10-16T14:30:06.520 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: osdmap e66: 10 total, 10 up, 10 in 2023-10-16T14:30:06.520 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:30:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:30:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] boot 2023-10-16T14:30:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: osdmap e66: 10 total, 10 up, 10 in 2023-10-16T14:30:06.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:30:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:30:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: osd.9 [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] boot 2023-10-16T14:30:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: osdmap e66: 10 total, 10 up, 10 in 2023-10-16T14:30:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-10-16T14:30:06.961 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 -- 172.21.15.186:0/2439005289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c80ffe50 msgr2=0x7f77c8100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:06.961 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 --2- 172.21.15.186:0/2439005289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c80ffe50 0x7f77c8100270 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009990 tx=0x7f77c4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.961 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 -- 172.21.15.186:0/2439005289 shutdown_connections 2023-10-16T14:30:06.961 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 --2- 172.21.15.186:0/2439005289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101a00 0x7f77c8108320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 --2- 172.21.15.186:0/2439005289 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c80ffe50 0x7f77c8100270 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 --2- 172.21.15.186:0/2439005289 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c8101040 0x7f77c81014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 -- 172.21.15.186:0/2439005289 >> 172.21.15.186:0/2439005289 conn(0x7f77c80fb440 msgr2=0x7f77c80fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:06.962 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 -- 172.21.15.186:0/2439005289 shutdown_connections 2023-10-16T14:30:06.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.955+0000 7f77cf262700 1 -- 172.21.15.186:0/2439005289 wait complete. 2023-10-16T14:30:06.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 Processor -- start 2023-10-16T14:30:06.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 -- start start 2023-10-16T14:30:06.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80ffe50 0x7f77c8069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:06.963 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101040 0x7f77c819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:06.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:06.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c819e7c0 con 0x7f77c80ffe50 2023-10-16T14:30:06.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c819e930 con 0x7f77c8101a00 2023-10-16T14:30:06.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cf262700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c819eaf0 con 0x7f77c8101040 2023-10-16T14:30:06.964 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:06.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.186:47400/0 (socket says 172.21.15.186:47400) 2023-10-16T14:30:06.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 -- 172.21.15.186:0/3547128931 learned_addr learned my addr 172.21.15.186:0/3547128931 (peer_addr_for_me v2:172.21.15.186:0/0) 2023-10-16T14:30:06.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77bffff700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101040 0x7f77c819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:06.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77ccffe700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80ffe50 0x7f77c8069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:06.965 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 -- 172.21.15.186:0/3547128931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101040 msgr2=0x7f77c819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:06.966 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101040 0x7f77c819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.966 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 -- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80ffe50 msgr2=0x7f77c8069e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:06.967 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80ffe50 0x7f77c8069e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:06.967 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.956+0000 7f77cd7ff700 1 -- 172.21.15.186:0/3547128931 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40095f0 con 0x7f77c8101a00 2023-10-16T14:30:06.967 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77cd7ff700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f77c0007ee0 tx=0x7f77c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:06.967 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0017070 con 0x7f77c8101a00 2023-10-16T14:30:06.967 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77c000f040 con 0x7f77c8101a00 2023-10-16T14:30:06.968 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c819edd0 con 0x7f77c8101a00 2023-10-16T14:30:06.968 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c0005740 con 0x7f77c8101a00 2023-10-16T14:30:06.968 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.957+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c819f220 con 0x7f77c8101a00 2023-10-16T14:30:06.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.958+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c810b6d0 con 0x7f77c8101a00 2023-10-16T14:30:06.969 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.959+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f77c000dc60 con 0x7f77c8101a00 2023-10-16T14:30:06.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.959+0000 7f77bdffb700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 0x7f77b00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:06.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.960+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f77c00cbd00 con 0x7f77c8101a00 2023-10-16T14:30:06.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.960+0000 7f77ccffe700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 0x7f77b00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:06.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.960+0000 7f77ccffe700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 0x7f77b00a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77c4006010 tx=0x7f77c400b540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:06.970 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:06.964+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f77c0095b60 con 0x7f77c8101a00 2023-10-16T14:30:07.166 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:07.162+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f77c80612f0 con 0x7f77b00a1480 2023-10-16T14:30:07.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: osdmap e67: 10 total, 10 up, 10 in 2023-10-16T14:30:07.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi178 2023-10-16T14:30:07.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi178 to 4317M 2023-10-16T14:30:07.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:07.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:07.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:07.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:07.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:07.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: osdmap e67: 10 total, 10 up, 10 in 2023-10-16T14:30:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi178 2023-10-16T14:30:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi178 to 4317M 2023-10-16T14:30:07.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:07.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:07.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:07.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:07.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:07.579 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:07.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: osdmap e67: 10 total, 10 up, 10 in 2023-10-16T14:30:07.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi178 2023-10-16T14:30:07.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:07.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi178 to 4317M 2023-10-16T14:30:07.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:07.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:07.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:07.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:07.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:07.678 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:07.679 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:08.536 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:08 smithi186 ceph-mon[134892]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:08.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:08 smithi187 ceph-mon[135297]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:08.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:08 smithi178 ceph-mon[98495]: from='client.24380 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi186:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:09.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:09 smithi187 ceph-mon[135297]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:09.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:09 smithi178 ceph-mon[98495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:09.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:09 smithi186 ceph-mon[134892]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:10.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/3248107724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]': finished 2023-10-16T14:30:10.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: osdmap e68: 11 total, 10 up, 11 in 2023-10-16T14:30:10.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:10.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:10 smithi187 ceph-mon[135297]: from='client.? 172.21.15.186:0/1918908950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:10.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/3248107724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]': finished 2023-10-16T14:30:10.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: osdmap e68: 11 total, 10 up, 11 in 2023-10-16T14:30:10.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:10.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:10 smithi178 ceph-mon[98495]: from='client.? 172.21.15.186:0/1918908950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:10.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/3248107724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]: dispatch 2023-10-16T14:30:10.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c36d6fd-c771-438e-916d-793a97beea8a"}]': finished 2023-10-16T14:30:10.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: osdmap e68: 11 total, 10 up, 11 in 2023-10-16T14:30:10.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:10.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:10 smithi186 ceph-mon[134892]: from='client.? 172.21.15.186:0/1918908950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:11.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:11 smithi187 ceph-mon[135297]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:11.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:11 smithi178 ceph-mon[98495]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:11.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:11 smithi186 ceph-mon[134892]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:13 smithi187 ceph-mon[135297]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:13 smithi178 ceph-mon[98495]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:14.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:13 smithi186 ceph-mon[134892]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:15 smithi187 ceph-mon[135297]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:15 smithi178 ceph-mon[98495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:16.055 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:15 smithi186 ceph-mon[134892]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:16.904 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-16T14:30:16.904 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:16.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-16T14:30:16.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-10-16T14:30:17.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:17.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:17 smithi186 ceph-mon[134892]: Deploying daemon osd.10 on smithi186 2023-10-16T14:30:17.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:17 smithi186 ceph-mon[134892]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:17.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:17 smithi187 ceph-mon[135297]: Deploying daemon osd.10 on smithi186 2023-10-16T14:30:17.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:17 smithi187 ceph-mon[135297]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:17 smithi178 ceph-mon[98495]: Deploying daemon osd.10 on smithi186 2023-10-16T14:30:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:17 smithi178 ceph-mon[98495]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:19.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:19 smithi186 ceph-mon[134892]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:19.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:19 smithi187 ceph-mon[135297]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:19 smithi178 ceph-mon[98495]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:21 smithi186 ceph-mon[134892]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:21 smithi178 ceph-mon[98495]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:21 smithi187 ceph-mon[135297]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:23.125 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.122+0000 7f77bdffb700 1 -- 172.21.15.186:0/3547128931 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f77c80612f0 con 0x7f77b00a1480 2023-10-16T14:30:23.126 INFO:teuthology.orchestra.run.smithi186.stdout:Created osd(s) 10 on host 'smithi186' 2023-10-16T14:30:23.128 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.124+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 msgr2=0x7f77b00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:23.128 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.124+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 0x7f77b00a3940 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f77c4006010 tx=0x7f77c400b540 comp rx=0 tx=0).stop 2023-10-16T14:30:23.128 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.124+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 msgr2=0x7f77c819e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:23.128 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.124+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f77c0007ee0 tx=0x7f77c000d3b0 comp rx=0 tx=0).stop 2023-10-16T14:30:23.129 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 shutdown_connections 2023-10-16T14:30:23.129 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f77b00a1480 0x7f77b00a3940 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:23.129 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f77c8101a00 0x7f77c819e020 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:23.129 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77c80ffe50 0x7f77c8069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:23.129 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 --2- 172.21.15.186:0/3547128931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77c8101040 0x7f77c819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:23.130 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 >> 172.21.15.186:0/3547128931 conn(0x7f77c80fb440 msgr2=0x7f77c80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:23.130 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 shutdown_connections 2023-10-16T14:30:23.130 INFO:teuthology.orchestra.run.smithi186.stderr:2023-10-16T14:30:23.125+0000 7f77cf262700 1 -- 172.21.15.186:0/3547128931 wait complete. 2023-10-16T14:30:23.483 DEBUG:teuthology.orchestra.run.smithi186:osd.10> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.10.service 2023-10-16T14:30:23.486 INFO:tasks.cephadm:Deploying osd.11 on smithi187 with /dev/vg_nvme/lv_1... 2023-10-16T14:30:23.486 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-10-16T14:30:23.515 INFO:journalctl@ceph.osd.10.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:30:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:23 smithi186 ceph-mon[134892]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:23 smithi187 ceph-mon[135297]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:23 smithi178 ceph-mon[98495]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:24.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:24.054 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:30:24.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:30:24 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:30:24.437+0000 7fcb45f53640 -1 osd.10 0 log_to_monitors true 2023-10-16T14:30:24.889 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:24 smithi186 ceph-mon[134892]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:24.889 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:24 smithi186 ceph-mon[134892]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:24.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:24 smithi187 ceph-mon[135297]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:24.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:24 smithi187 ceph-mon[135297]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:25.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:24 smithi178 ceph-mon[98495]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:25.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:24 smithi178 ceph-mon[98495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-10-16T14:30:25.671 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-10-16T14:30:25.671 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-10-16T14:30:25.671 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2023-10-16T14:30:25.671 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2023-10-16T14:30:25.672 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111452 s, 941 MB/s 2023-10-16T14:30:25.672 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2023-10-16T14:30:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-16T14:30:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: osdmap e69: 11 total, 10 up, 11 in 2023-10-16T14:30:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:25.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:25.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:25 smithi187 ceph-mon[135297]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:25.967 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:25.967 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-16T14:30:25.968 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: osdmap e69: 11 total, 10 up, 11 in 2023-10-16T14:30:25.968 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:25.968 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:25.968 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-mon[134892]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:25.969 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:30:25.640+0000 7fcb3b5ca700 -1 osd.10 0 waiting for initial osdmap 2023-10-16T14:30:25.969 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:30:25 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:30:25.654+0000 7fcb363bd700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:30:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-10-16T14:30:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: osdmap e69: 11 total, 10 up, 11 in 2023-10-16T14:30:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: from='osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:26.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:25 smithi178 ceph-mon[98495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]: dispatch 2023-10-16T14:30:26.111 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch daemon add osd smithi187:vg_nvme/lv_1 2023-10-16T14:30:26.674 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:30:26.898 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:30:26.898 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: osdmap e70: 11 total, 10 up, 11 in 2023-10-16T14:30:26.898 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:26.899 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:26.899 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:26.899 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:26.899 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:26.899 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:26.900 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:26.901 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:26.901 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:26.901 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:26.901 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:30:27.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: osdmap e70: 11 total, 10 up, 11 in 2023-10-16T14:30:27.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:27.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:27.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:27.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:27.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:27.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi186", "root=default"]}]': finished 2023-10-16T14:30:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: osdmap e70: 11 total, 10 up, 11 in 2023-10-16T14:30:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:27.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:27.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:27.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:27.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:27.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:27.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:30:27.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:30:27.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi186 2023-10-16T14:30:27.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-16T14:30:27.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:27.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] boot 2023-10-16T14:30:27.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: osdmap e71: 11 total, 11 up, 11 in 2023-10-16T14:30:27.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:27 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:27.956 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 -- 172.21.15.187:0/3885905862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 msgr2=0x7fb9c81013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:27.956 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/3885905862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c81013c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009990 tx=0x7fb9bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.956 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 -- 172.21.15.187:0/3885905862 shutdown_connections 2023-10-16T14:30:27.957 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/3885905862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8102b40 0x7fb9c81093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.957 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/3885905862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c81013c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.957 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/3885905862 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c81021a0 0x7fb9c8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.957 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 -- 172.21.15.187:0/3885905862 >> 172.21.15.187:0/3885905862 conn(0x7fb9c80fc540 msgr2=0x7fb9c80fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:27.957 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 -- 172.21.15.187:0/3885905862 shutdown_connections 2023-10-16T14:30:27.958 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.952+0000 7fb9ceef5700 1 -- 172.21.15.187:0/3885905862 wait complete. 2023-10-16T14:30:27.960 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 Processor -- start 2023-10-16T14:30:27.960 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 -- start start 2023-10-16T14:30:27.961 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:27.961 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c81021a0 0x7fb9c8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:27.961 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 0x7fb9c819e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:27.961 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819e750 con 0x7fb9c81021a0 2023-10-16T14:30:27.962 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819e890 con 0x7fb9c8102b40 2023-10-16T14:30:27.962 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ceef5700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819ea20 con 0x7fb9c8100fa0 2023-10-16T14:30:27.962 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ccc91700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:27.962 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ccc91700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:33904/0 (socket says 172.21.15.187:33904) 2023-10-16T14:30:27.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.953+0000 7fb9ccc91700 1 -- 172.21.15.187:0/1736915838 learned_addr learned my addr 172.21.15.187:0/1736915838 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:30:27.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9cd492700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 0x7fb9c819e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:27.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 -- 172.21.15.187:0/1736915838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 msgr2=0x7fb9c819e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:27.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 0x7fb9c819e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.963 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 -- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c81021a0 msgr2=0x7fb9c8197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:27.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c81021a0 0x7fb9c8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:27.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 -- 172.21.15.187:0/1736915838 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0095f0 con 0x7fb9c8100fa0 2023-10-16T14:30:27.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9cd492700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 0x7fb9c819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:30:27.964 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ccc91700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000c00 tx=0x7fb9bc00f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:27.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc01d070 con 0x7fb9c8100fa0 2023-10-16T14:30:27.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9bc00fdf0 con 0x7fb9c8100fa0 2023-10-16T14:30:27.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c819eca0 con 0x7fb9c8100fa0 2023-10-16T14:30:27.965 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc017790 con 0x7fb9c8100fa0 2023-10-16T14:30:27.966 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.954+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c819f0b0 con 0x7fb9c8100fa0 2023-10-16T14:30:27.966 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.955+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c81084f0 con 0x7fb9c8100fa0 2023-10-16T14:30:27.966 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.956+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb9bc022470 con 0x7fb9c8100fa0 2023-10-16T14:30:27.966 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.957+0000 7fb9c5ffb700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 0x7fb9b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:27.966 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.957+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fb9bc0cdd10 con 0x7fb9c8100fa0 2023-10-16T14:30:27.967 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.957+0000 7fb9c7fff700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 0x7fb9b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:27.967 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.958+0000 7fb9c7fff700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 0x7fb9b80a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400a990 tx=0x7fb9b4005c80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:27.968 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:27.961+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb9bc0d3020 con 0x7fb9c8100fa0 2023-10-16T14:30:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:30:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:30:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi186 2023-10-16T14:30:28.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-16T14:30:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] boot 2023-10-16T14:30:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: osdmap e71: 11 total, 11 up, 11 in 2023-10-16T14:30:28.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:27 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:30:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:30:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi186 2023-10-16T14:30:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi186 to 4292M 2023-10-16T14:30:28.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-10-16T14:30:28.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:28.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: osd.10 [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] boot 2023-10-16T14:30:28.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: osdmap e71: 11 total, 11 up, 11 in 2023-10-16T14:30:28.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:27 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-10-16T14:30:28.162 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:28.159+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9c810c7a0 con 0x7fb9b80a1480 2023-10-16T14:30:28.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:28 smithi187 ceph-mon[135297]: osdmap e72: 11 total, 11 up, 11 in 2023-10-16T14:30:28.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:28.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:28.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:28 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:28 smithi178 ceph-mon[98495]: osdmap e72: 11 total, 11 up, 11 in 2023-10-16T14:30:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:29.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:28 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:28 smithi186 ceph-mon[134892]: osdmap e72: 11 total, 11 up, 11 in 2023-10-16T14:30:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-10-16T14:30:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-10-16T14:30:29.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:28 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:29.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:29 smithi187 ceph-mon[135297]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:29.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:29 smithi187 ceph-mon[135297]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:29.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:29 smithi187 ceph-mon[135297]: osdmap e73: 11 total, 11 up, 11 in 2023-10-16T14:30:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:29 smithi178 ceph-mon[98495]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:29 smithi178 ceph-mon[98495]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:29 smithi178 ceph-mon[98495]: osdmap e73: 11 total, 11 up, 11 in 2023-10-16T14:30:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:29 smithi186 ceph-mon[134892]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:30:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:29 smithi186 ceph-mon[134892]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:29 smithi186 ceph-mon[134892]: osdmap e73: 11 total, 11 up, 11 in 2023-10-16T14:30:30.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: osdmap e74: 11 total, 11 up, 11 in 2023-10-16T14:30:30.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/2125747746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:30.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:30.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]': finished 2023-10-16T14:30:30.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: osdmap e75: 12 total, 11 up, 12 in 2023-10-16T14:30:30.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:30 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: osdmap e74: 11 total, 11 up, 11 in 2023-10-16T14:30:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/2125747746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]': finished 2023-10-16T14:30:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: osdmap e75: 12 total, 11 up, 12 in 2023-10-16T14:30:31.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:30 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:31.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: osdmap e74: 11 total, 11 up, 11 in 2023-10-16T14:30:31.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/2125747746' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:31.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]: dispatch 2023-10-16T14:30:31.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc27eb51-d50b-431d-9e60-f53a95da61af"}]': finished 2023-10-16T14:30:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: osdmap e75: 12 total, 11 up, 12 in 2023-10-16T14:30:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:30 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:31 smithi187 ceph-mon[135297]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:31 smithi187 ceph-mon[135297]: from='client.? 172.21.15.187:0/296096924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:32.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:31 smithi178 ceph-mon[98495]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:32.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:31 smithi178 ceph-mon[98495]: from='client.? 172.21.15.187:0/296096924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:31 smithi186 ceph-mon[134892]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:31 smithi186 ceph-mon[134892]: from='client.? 172.21.15.187:0/296096924' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-10-16T14:30:33.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:33 smithi187 ceph-mon[135297]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:33 smithi178 ceph-mon[98495]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:33 smithi186 ceph-mon[134892]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:35.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:35 smithi187 ceph-mon[135297]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:30:36.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:35 smithi178 ceph-mon[98495]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:30:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:35 smithi186 ceph-mon[134892]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-10-16T14:30:37.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:37 smithi187 ceph-mon[135297]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-16T14:30:37.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-16T14:30:37.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:37 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:38.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:37 smithi178 ceph-mon[98495]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-16T14:30:38.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-16T14:30:38.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:37 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:37 smithi186 ceph-mon[134892]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2023-10-16T14:30:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-10-16T14:30:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:37 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:38.916 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:38 smithi187 ceph-mon[135297]: Deploying daemon osd.11 on smithi187 2023-10-16T14:30:39.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:38 smithi178 ceph-mon[98495]: Deploying daemon osd.11 on smithi187 2023-10-16T14:30:39.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:38 smithi186 ceph-mon[134892]: Deploying daemon osd.11 on smithi187 2023-10-16T14:30:39.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:39 smithi187 ceph-mon[135297]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-16T14:30:40.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:39 smithi178 ceph-mon[98495]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-16T14:30:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:39 smithi186 ceph-mon[134892]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2023-10-16T14:30:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:41 smithi187 ceph-mon[135297]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-16T14:30:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:41 smithi178 ceph-mon[98495]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-16T14:30:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:41 smithi186 ceph-mon[134892]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-10-16T14:30:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:43 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:43 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:43 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:43.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:43 smithi187 ceph-mon[135297]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:43 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:43 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:43 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:43.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:43 smithi178 ceph-mon[98495]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:43.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:43 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:43 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:43.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:43 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:43.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:43 smithi186 ceph-mon[134892]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:45.026 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 11 on host 'smithi187' 2023-10-16T14:30:45.026 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.022+0000 7fb9c5ffb700 1 -- 172.21.15.187:0/1736915838 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb9c810c7a0 con 0x7fb9b80a1480 2023-10-16T14:30:45.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 msgr2=0x7fb9b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:45.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 0x7fb9b80a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400a990 tx=0x7fb9b4005c80 comp rx=0 tx=0).stop 2023-10-16T14:30:45.028 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 msgr2=0x7fb9c8196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:45.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000c00 tx=0x7fb9bc00f880 comp rx=0 tx=0).stop 2023-10-16T14:30:45.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 shutdown_connections 2023-10-16T14:30:45.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb9b80a1480 0x7fb9b80a3940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:45.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb9c8102b40 0x7fb9c819e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:45.029 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb9c8100fa0 0x7fb9c8196bc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:45.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 --2- 172.21.15.187:0/1736915838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c81021a0 0x7fb9c8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:45.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.024+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 >> 172.21.15.187:0/1736915838 conn(0x7fb9c80fc540 msgr2=0x7fb9c8105630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:45.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.025+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 shutdown_connections 2023-10-16T14:30:45.030 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:30:45.025+0000 7fb9ceef5700 1 -- 172.21.15.187:0/1736915838 wait complete. 2023-10-16T14:30:45.460 DEBUG:teuthology.orchestra.run.smithi187:osd.11> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.11.service 2023-10-16T14:30:45.462 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-10-16T14:30:45.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-16T14:30:45.495 INFO:journalctl@ceph.osd.11.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:30:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:45 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:46.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:45 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:46.200 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:30:46.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-10-16T14:30:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:46.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:45 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:30:46.586 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:30:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:30:46.238+0000 7f2f40a02640 -1 osd.11 0 log_to_monitors true 2023-10-16T14:30:46.894 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:46 smithi187 ceph-mon[135297]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:46.895 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:46 smithi187 ceph-mon[135297]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:47.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:46 smithi178 ceph-mon[98495]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:47.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:46 smithi178 ceph-mon[98495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:46 smithi186 ceph-mon[134892]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:46 smithi186 ceph-mon[134892]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-10-16T14:30:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 -- 172.21.15.178:0/4077194694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102170 msgr2=0x7fdc9c1025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/4077194694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102170 0x7fdc9c1025d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc880099c0 tx=0x7fdc88009cd0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 -- 172.21.15.178:0/4077194694 shutdown_connections 2023-10-16T14:30:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/4077194694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/4077194694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/4077194694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102170 0x7fdc9c1025d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 -- 172.21.15.178:0/4077194694 >> 172.21.15.178:0/4077194694 conn(0x7fdc9c0fc4d0 msgr2=0x7fdc9c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 -- 172.21.15.178:0/4077194694 shutdown_connections 2023-10-16T14:30:47.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.591+0000 7fdca2fd3700 1 -- 172.21.15.178:0/4077194694 wait complete. 2023-10-16T14:30:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 Processor -- start 2023-10-16T14:30:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 -- start start 2023-10-16T14:30:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:47.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1a0a00 con 0x7fdc9c102b10 2023-10-16T14:30:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1a0b40 con 0x7fdc9c100f70 2023-10-16T14:30:47.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.592+0000 7fdca2fd3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1a0cd0 con 0x7fdc9c102170 2023-10-16T14:30:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54448/0 (socket says 172.21.15.178:54448) 2023-10-16T14:30:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 -- 172.21.15.178:0/739778144 learned_addr learned my addr 172.21.15.178:0/739778144 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:30:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdca0d6f700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:47.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdca1570700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 -- 172.21.15.178:0/739778144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 msgr2=0x7fdc9c198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 -- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 msgr2=0x7fdc9c1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 -- 172.21.15.178:0/739778144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880095f0 con 0x7fdc9c102170 2023-10-16T14:30:47.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdca0d6f700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:30:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdca1570700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:30:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc9bfff700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c010 tx=0x7fdc8800f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8801d070 con 0x7fdc9c102170 2023-10-16T14:30:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc88005510 con 0x7fdc9c102170 2023-10-16T14:30:47.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c1a0f50 con 0x7fdc9c102170 2023-10-16T14:30:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.593+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88004cb0 con 0x7fdc9c102170 2023-10-16T14:30:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.594+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c1a1300 con 0x7fdc9c102170 2023-10-16T14:30:47.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.595+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c1935b0 con 0x7fdc9c102170 2023-10-16T14:30:47.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.596+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc8800fbf0 con 0x7fdc9c102170 2023-10-16T14:30:47.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.598+0000 7fdc99ffb700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 0x7fdc8c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:47.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.598+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fdc880cd690 con 0x7fdc9c102170 2023-10-16T14:30:47.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.598+0000 7fdca0d6f700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 0x7fdc8c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:47.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.598+0000 7fdca0d6f700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 0x7fdc8c0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c101fd0 tx=0x7fdc90009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:47.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.600+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdc880973d0 con 0x7fdc9c102170 2023-10-16T14:30:47.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.798+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fdc9c10c7c0 con 0x7fdc9c102170 2023-10-16T14:30:47.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.799+0000 7fdc99ffb700 1 -- 172.21.15.178:0/739778144 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7fdc880260a0 con 0x7fdc9c102170 2023-10-16T14:30:47.802 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:30:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.801+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 msgr2=0x7fdc8c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:47.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.801+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 0x7fdc8c0a3940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c101fd0 tx=0x7fdc90009260 comp rx=0 tx=0).stop 2023-10-16T14:30:47.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.801+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 msgr2=0x7fdc9c1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:47.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.801+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdc8800c010 tx=0x7fdc8800f740 comp rx=0 tx=0).stop 2023-10-16T14:30:47.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 shutdown_connections 2023-10-16T14:30:47.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc8c0a1480 0x7fdc8c0a3940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc9c102b10 0x7fdc9c1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc9c100f70 0x7fdc9c198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 --2- 172.21.15.178:0/739778144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc9c102170 0x7fdc9c1993b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 >> 172.21.15.178:0/739778144 conn(0x7fdc9c0fc4d0 msgr2=0x7fdc9c107ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 shutdown_connections 2023-10-16T14:30:47.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:47.802+0000 7fdca2fd3700 1 -- 172.21.15.178:0/739778144 wait complete. 2023-10-16T14:30:48.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:48.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-16T14:30:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: osdmap e76: 12 total, 11 up, 12 in 2023-10-16T14:30:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/739778144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:48.177 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:30:47.837+0000 7f2f36079700 -1 osd.11 0 waiting for initial osdmap 2023-10-16T14:30:48.177 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:30:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:30:47.853+0000 7f2f3066b700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T14:30:48.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-16T14:30:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: osdmap e76: 12 total, 11 up, 12 in 2023-10-16T14:30:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:47 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/739778144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-10-16T14:30:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: osdmap e76: 12 total, 11 up, 12 in 2023-10-16T14:30:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: from='osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2023-10-16T14:30:48.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:47 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/739778144' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:48.382 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1697466626,"num_in_osds":12,"osd_in_since":1697466630,"num_remapped_pgs":0} 2023-10-16T14:30:49.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:30:49.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: osdmap e77: 12 total, 11 up, 12 in 2023-10-16T14:30:49.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:49.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:49.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:48 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:30:49.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: osdmap e77: 12 total, 11 up, 12 in 2023-10-16T14:30:49.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:49.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:49.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:48 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2023-10-16T14:30:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: osdmap e77: 12 total, 11 up, 12 in 2023-10-16T14:30:49.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:30:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:30:49.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:30:49.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:48 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:30:49.383 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd stat -f json 2023-10-16T14:30:50.115 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:30:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: purged_snaps scrub starts 2023-10-16T14:30:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: purged_snaps scrub ok 2023-10-16T14:30:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: Detected new or changed devices on smithi187 2023-10-16T14:30:50.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi187 to 4292M 2023-10-16T14:30:50.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:50.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] boot 2023-10-16T14:30:50.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: osdmap e78: 12 total, 12 up, 12 in 2023-10-16T14:30:50.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:49 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:50.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: purged_snaps scrub starts 2023-10-16T14:30:50.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: purged_snaps scrub ok 2023-10-16T14:30:50.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: Detected new or changed devices on smithi187 2023-10-16T14:30:50.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi187 to 4292M 2023-10-16T14:30:50.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:50.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] boot 2023-10-16T14:30:50.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: osdmap e78: 12 total, 12 up, 12 in 2023-10-16T14:30:50.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:49 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: purged_snaps scrub starts 2023-10-16T14:30:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: purged_snaps scrub ok 2023-10-16T14:30:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: Detected new or changed devices on smithi187 2023-10-16T14:30:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi187 to 4292M 2023-10-16T14:30:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-10-16T14:30:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: osd.11 [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] boot 2023-10-16T14:30:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: osdmap e78: 12 total, 12 up, 12 in 2023-10-16T14:30:50.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:49 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-10-16T14:30:50.923 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:50 smithi178 ceph-mon[98495]: osdmap e79: 12 total, 12 up, 12 in 2023-10-16T14:30:51.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:50 smithi187 ceph-mon[135297]: osdmap e79: 12 total, 12 up, 12 in 2023-10-16T14:30:51.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:50 smithi186 ceph-mon[134892]: osdmap e79: 12 total, 12 up, 12 in 2023-10-16T14:30:51.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 -- 172.21.15.178:0/3052955143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 msgr2=0x7f95941093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:51.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 --2- 172.21.15.178:0/3052955143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95941093c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b0d0 tx=0x7f958c00b3e0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 -- 172.21.15.178:0/3052955143 shutdown_connections 2023-10-16T14:30:51.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 --2- 172.21.15.178:0/3052955143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95941093c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 --2- 172.21.15.178:0/3052955143 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 --2- 172.21.15.178:0/3052955143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 0x7f95941025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 -- 172.21.15.178:0/3052955143 >> 172.21.15.178:0/3052955143 conn(0x7f95940fc4d0 msgr2=0x7f95940fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:51.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 -- 172.21.15.178:0/3052955143 shutdown_connections 2023-10-16T14:30:51.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.507+0000 7f959bfdd700 1 -- 172.21.15.178:0/3052955143 wait complete. 2023-10-16T14:30:51.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 Processor -- start 2023-10-16T14:30:51.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 -- start start 2023-10-16T14:30:51.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:51.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 0x7f95940732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:51.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:51.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9594073f60 con 0x7f9594102b10 2023-10-16T14:30:51.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940740d0 con 0x7f9594100f70 2023-10-16T14:30:51.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.509+0000 7f959bfdd700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95941a0d80 con 0x7f9594102170 2023-10-16T14:30:51.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f9599d79700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:51.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:51.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f9599578700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 0x7f95940732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:51.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37308/0 (socket says 172.21.15.178:37308) 2023-10-16T14:30:51.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f9599d79700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53168/0 (socket says 172.21.15.178:53168) 2023-10-16T14:30:51.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 -- 172.21.15.178:0/2066497962 learned_addr learned my addr 172.21.15.178:0/2066497962 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:30:51.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 -- 172.21.15.178:0/2066497962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 msgr2=0x7f95940732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:51.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 0x7f95940732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 -- 172.21.15.178:0/2066497962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 msgr2=0x7f9594074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:51.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 -- 172.21.15.178:0/2066497962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c009d00 con 0x7f9594102b10 2023-10-16T14:30:51.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f959a57a700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f958c008eb0 tx=0x7f958c008ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:51.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00e040 con 0x7f9594102b10 2023-10-16T14:30:51.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f958c00bdb0 con 0x7f9594102b10 2023-10-16T14:30:51.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.510+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c01d470 con 0x7f9594102b10 2023-10-16T14:30:51.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.511+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95941a1000 con 0x7f9594102b10 2023-10-16T14:30:51.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.511+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95941a1470 con 0x7f9594102b10 2023-10-16T14:30:51.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.512+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95941084d0 con 0x7f9594102b10 2023-10-16T14:30:51.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.514+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f958c005020 con 0x7f9594102b10 2023-10-16T14:30:51.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.515+0000 7f958affd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 0x7f95800a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:51.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.515+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f958c0cd7d0 con 0x7f9594102b10 2023-10-16T14:30:51.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.515+0000 7f9599d79700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 0x7f95800a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:51.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.516+0000 7f9599d79700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 0x7f95800a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9594101fd0 tx=0x7f9590009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:51.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.519+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f958c0972c0 con 0x7f9594102b10 2023-10-16T14:30:51.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.718+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f959410c9b0 con 0x7f9594102b10 2023-10-16T14:30:51.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.719+0000 7f958affd700 1 -- 172.21.15.178:0/2066497962 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f958c09a8e0 con 0x7f9594102b10 2023-10-16T14:30:51.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:30:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 msgr2=0x7f95800a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 0x7f95800a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9594101fd0 tx=0x7f9590009260 comp rx=0 tx=0).stop 2023-10-16T14:30:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 msgr2=0x7f95940737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f958c008eb0 tx=0x7f958c008ee0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 shutdown_connections 2023-10-16T14:30:51.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95800a1500 0x7f95800a39c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9594102b10 0x7f95940737e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9594100f70 0x7f9594074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 --2- 172.21.15.178:0/2066497962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9594102170 0x7f95940732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:51.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 >> 172.21.15.178:0/2066497962 conn(0x7f95940fc4d0 msgr2=0x7f9594105610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:51.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 shutdown_connections 2023-10-16T14:30:51.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:51.721+0000 7f959bfdd700 1 -- 172.21.15.178:0/2066497962 wait complete. 2023-10-16T14:30:52.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:51 smithi178 ceph-mon[98495]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:52.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:51 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2066497962' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:52.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:51 smithi187 ceph-mon[135297]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:52.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:51 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2066497962' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:52.302 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1697466648,"num_in_osds":12,"osd_in_since":1697466630,"num_remapped_pgs":0} 2023-10-16T14:30:52.303 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-16T14:30:52.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:51 smithi186 ceph-mon[134892]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:52.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:51 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2066497962' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-10-16T14:30:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:30:53.950 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:53 smithi178 ceph-mon[98495]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:54.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:53 smithi187 ceph-mon[135297]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:54.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:53 smithi186 ceph-mon[134892]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:54.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 -- 172.21.15.178:0/237159872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4101940 msgr2=0x7faee410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:54.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 --2- 172.21.15.178:0/237159872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4101940 0x7faee410ddb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7faedc00b0d0 tx=0x7faedc00b3e0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 -- 172.21.15.178:0/237159872 shutdown_connections 2023-10-16T14:30:54.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 --2- 172.21.15.178:0/237159872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4101940 0x7faee410ddb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 --2- 172.21.15.178:0/237159872 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4106f90 0x7faee4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 --2- 172.21.15.178:0/237159872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 0x7faee4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 -- 172.21.15.178:0/237159872 >> 172.21.15.178:0/237159872 conn(0x7faee40780a0 msgr2=0x7faee40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:54.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 -- 172.21.15.178:0/237159872 shutdown_connections 2023-10-16T14:30:54.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.523+0000 7faeea4e2700 1 -- 172.21.15.178:0/237159872 wait complete. 2023-10-16T14:30:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.524+0000 7faeea4e2700 1 Processor -- start 2023-10-16T14:30:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.524+0000 7faeea4e2700 1 -- start start 2023-10-16T14:30:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 0x7faee4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:54.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4101940 0x7faee4199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:54.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:54.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee419e2c0 con 0x7faee4106f90 2023-10-16T14:30:54.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee419e430 con 0x7faee4101940 2023-10-16T14:30:54.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faeea4e2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee419e5f0 con 0x7faee4100f80 2023-10-16T14:30:54.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37322/0 (socket says 172.21.15.178:37322) 2023-10-16T14:30:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 -- 172.21.15.178:0/392204167 learned_addr learned my addr 172.21.15.178:0/392204167 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:30:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee3fff700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 0x7faee4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee37fe700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4101940 0x7faee4199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:54.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 -- 172.21.15.178:0/392204167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 msgr2=0x7faee4199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:54.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 0x7faee4199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 -- 172.21.15.178:0/392204167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4101940 msgr2=0x7faee4199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:54.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4101940 0x7faee4199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.525+0000 7faee8a7f700 1 -- 172.21.15.178:0/392204167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc009d00 con 0x7faee4106f90 2023-10-16T14:30:54.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faee8a7f700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faedc007820 tx=0x7faedc007850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00e040 con 0x7faee4106f90 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faedc00bcb0 con 0x7faee4106f90 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee419e870 con 0x7faee4106f90 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc0057b0 con 0x7faee4106f90 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.526+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee41a6130 con 0x7faee4106f90 2023-10-16T14:30:54.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.527+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee404ebb0 con 0x7faee4106f90 2023-10-16T14:30:54.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.529+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7faedc004a50 con 0x7faee4106f90 2023-10-16T14:30:54.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.530+0000 7faee17fa700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 0x7faecc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:54.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.531+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faedc0cd580 con 0x7faee4106f90 2023-10-16T14:30:54.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.531+0000 7faee3fff700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 0x7faecc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:54.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.531+0000 7faee3fff700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 0x7faecc0a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faed4009be0 tx=0x7faed4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:54.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.534+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7faedc097070 con 0x7faee4106f90 2023-10-16T14:30:54.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.736+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faee4110fa0 con 0x7faee4106f90 2023-10-16T14:30:54.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.737+0000 7faee17fa700 1 -- 172.21.15.178:0/392204167 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18482 (secure 0 0 0) 0x7faedc09a690 con 0x7faee4106f90 2023-10-16T14:30:54.740 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:30:54.740 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":79,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:30:49.840789+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:30:54.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 msgr2=0x7faecc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:54.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 0x7faecc0a39c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faed4009be0 tx=0x7faed4009260 comp rx=0 tx=0).stop 2023-10-16T14:30:54.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 msgr2=0x7faee419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:54.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faedc007820 tx=0x7faedc007850 comp rx=0 tx=0).stop 2023-10-16T14:30:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 shutdown_connections 2023-10-16T14:30:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faecc0a1500 0x7faecc0a39c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faee4106f90 0x7faee419dcb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faee4100f80 0x7faee4199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 --2- 172.21.15.178:0/392204167 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7faee4101940 0x7faee4199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:54.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 >> 172.21.15.178:0/392204167 conn(0x7faee40780a0 msgr2=0x7faee410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:54.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 shutdown_connections 2023-10-16T14:30:54.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:54.740+0000 7faeea4e2700 1 -- 172.21.15.178:0/392204167 wait complete. 2023-10-16T14:30:55.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:54 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/392204167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:30:55.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:54 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/392204167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:30:55.216 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T14:30:55.217 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd pool get .mgr pg_num 2023-10-16T14:30:55.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:54 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/392204167' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:30:55.956 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:30:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:55 smithi187 ceph-mon[135297]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:56.218 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:55 smithi178 ceph-mon[98495]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:56.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:55 smithi186 ceph-mon[134892]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:57.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 -- 172.21.15.178:0/2225578812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8101940 msgr2=0x7f01a810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:57.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 --2- 172.21.15.178:0/2225578812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8101940 0x7f01a810ddb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b0d0 tx=0x7f01a000b3e0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 -- 172.21.15.178:0/2225578812 shutdown_connections 2023-10-16T14:30:57.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 --2- 172.21.15.178:0/2225578812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8101940 0x7f01a810ddb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 --2- 172.21.15.178:0/2225578812 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8106f90 0x7f01a8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 --2- 172.21.15.178:0/2225578812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8100f80 0x7f01a8101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 -- 172.21.15.178:0/2225578812 >> 172.21.15.178:0/2225578812 conn(0x7f01a80780a0 msgr2=0x7f01a80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:57.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 -- 172.21.15.178:0/2225578812 shutdown_connections 2023-10-16T14:30:57.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.468+0000 7f01ae54f700 1 -- 172.21.15.178:0/2225578812 wait complete. 2023-10-16T14:30:57.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.469+0000 7f01ae54f700 1 Processor -- start 2023-10-16T14:30:57.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 -- start start 2023-10-16T14:30:57.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 0x7f01a8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:57.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8101940 0x7f01a81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:57.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:57.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a819e220 con 0x7f01a8106f90 2023-10-16T14:30:57.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a819e390 con 0x7f01a8100f80 2023-10-16T14:30:57.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01ae54f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a819e550 con 0x7f01a8101940 2023-10-16T14:30:57.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01acaec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:57.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01a7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 0x7f01a8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:57.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01acaec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37336/0 (socket says 172.21.15.178:37336) 2023-10-16T14:30:57.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01acaec700 1 -- 172.21.15.178:0/1574588208 learned_addr learned my addr 172.21.15.178:0/1574588208 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:30:57.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.470+0000 7f01a77fe700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8101940 0x7f01a81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:57.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 -- 172.21.15.178:0/1574588208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8101940 msgr2=0x7f01a81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:57.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8101940 0x7f01a81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 -- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 msgr2=0x7f01a819dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:57.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 -- 172.21.15.178:0/1574588208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0198009520 con 0x7f01a8100f80 2023-10-16T14:30:57.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01acaec700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:30:57.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a7fff700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 0x7f01a8199260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f019800e9f0 tx=0x7f019800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:57.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019800cb70 con 0x7f01a8100f80 2023-10-16T14:30:57.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0009d00 con 0x7f01a8100f80 2023-10-16T14:30:57.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f019800ee90 con 0x7f01a8100f80 2023-10-16T14:30:57.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.471+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198003cf0 con 0x7f01a8100f80 2023-10-16T14:30:57.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.472+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a81a5ec0 con 0x7f01a8100f80 2023-10-16T14:30:57.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.472+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a804ebb0 con 0x7f01a8100f80 2023-10-16T14:30:57.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.475+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f019800ccd0 con 0x7f01a8100f80 2023-10-16T14:30:57.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.476+0000 7f01a57fa700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 0x7f01900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:30:57.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.476+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0198014070 con 0x7f01a8100f80 2023-10-16T14:30:57.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.476+0000 7f01a77fe700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 0x7f01900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:30:57.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.477+0000 7f01a77fe700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 0x7f01900a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01a819a880 tx=0x7f019c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:30:57.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.478+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0198096770 con 0x7f01a8100f80 2023-10-16T14:30:57.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.675+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01a819a5c0 con 0x7f01a8100f80 2023-10-16T14:30:57.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.676+0000 7f01a57fa700 1 -- 172.21.15.178:0/1574588208 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f019800fe60 con 0x7f01a8100f80 2023-10-16T14:30:57.679 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:30:57.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.678+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 msgr2=0x7f01900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:57.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.678+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 0x7f01900a3940 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f01a819a880 tx=0x7f019c009260 comp rx=0 tx=0).stop 2023-10-16T14:30:57.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.678+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 msgr2=0x7f01a8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:30:57.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.678+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 0x7f01a8199260 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f019800e9f0 tx=0x7f019800ed00 comp rx=0 tx=0).stop 2023-10-16T14:30:57.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 shutdown_connections 2023-10-16T14:30:57.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f01900a1480 0x7f01900a3940 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01a8106f90 0x7f01a819dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f01a8100f80 0x7f01a8199260 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 --2- 172.21.15.178:0/1574588208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01a8101940 0x7f01a81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:30:57.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 >> 172.21.15.178:0/1574588208 conn(0x7f01a80780a0 msgr2=0x7f01a810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:30:57.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 shutdown_connections 2023-10-16T14:30:57.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:30:57.679+0000 7f01ae54f700 1 -- 172.21.15.178:0/1574588208 wait complete. 2023-10-16T14:30:58.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:57 smithi178 ceph-mon[98495]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:58.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:57 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1574588208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:30:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:57 smithi187 ceph-mon[135297]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:57 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1574588208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:30:58.236 INFO:tasks.cephadm:Adding mds.a on smithi178 2023-10-16T14:30:58.236 INFO:tasks.cephadm:Adding mds.d on smithi178 2023-10-16T14:30:58.237 INFO:tasks.cephadm:Adding mds.g on smithi178 2023-10-16T14:30:58.237 INFO:tasks.cephadm:Adding mds.j on smithi178 2023-10-16T14:30:58.237 INFO:tasks.cephadm:Adding mds.b on smithi186 2023-10-16T14:30:58.237 INFO:tasks.cephadm:Adding mds.e on smithi186 2023-10-16T14:30:58.238 INFO:tasks.cephadm:Adding mds.h on smithi186 2023-10-16T14:30:58.238 INFO:tasks.cephadm:Adding mds.k on smithi186 2023-10-16T14:30:58.238 INFO:tasks.cephadm:Adding mds.c on smithi187 2023-10-16T14:30:58.238 INFO:tasks.cephadm:Adding mds.f on smithi187 2023-10-16T14:30:58.239 INFO:tasks.cephadm:Adding mds.i on smithi187 2023-10-16T14:30:58.239 INFO:tasks.cephadm:Adding mds.l on smithi187 2023-10-16T14:30:58.239 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph orch apply mds all '12;smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l' 2023-10-16T14:30:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:57 smithi186 ceph-mon[134892]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:30:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:57 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1574588208' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:30:58.811 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:31:00.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- 172.21.15.187:0/2092923347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68641014b0 msgr2=0x7f68641018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:00.017 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- 172.21.15.187:0/2092923347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68641014b0 0x7f68641018d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6860009990 tx=0x7f6860009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- 172.21.15.187:0/2092923347 shutdown_connections 2023-10-16T14:31:00.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- 172.21.15.187:0/2092923347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6864100ad0 0x7f6864108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- 172.21.15.187:0/2092923347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f68641014b0 0x7f68641018d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- 172.21.15.187:0/2092923347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6864100110 0x7f6864100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.018 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- 172.21.15.187:0/2092923347 >> 172.21.15.187:0/2092923347 conn(0x7f68640fac00 msgr2=0x7f68640fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- 172.21.15.187:0/2092923347 shutdown_connections 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- 172.21.15.187:0/2092923347 wait complete. 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 Processor -- start 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 -- start start 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6864100110 0x7f686410a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:00.019 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:00.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.012+0000 7f686b916700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68641014b0 0x7f68641093a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:00.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f686b916700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6864109b20 con 0x7f6864100110 2023-10-16T14:31:00.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f686b916700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6864109c90 con 0x7f68641014b0 2023-10-16T14:31:00.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f686b916700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68641a0c20 con 0x7f6864100ad0 2023-10-16T14:31:00.020 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:00.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:34028/0 (socket says 172.21.15.187:34028) 2023-10-16T14:31:00.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 -- 172.21.15.187:0/823321844 learned_addr learned my addr 172.21.15.187:0/823321844 (peer_addr_for_me v2:172.21.15.187:0/0) 2023-10-16T14:31:00.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6869eb3700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68641014b0 0x7f68641093a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:00.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 -- 172.21.15.187:0/823321844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68641014b0 msgr2=0x7f68641093a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:00.021 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68641014b0 0x7f68641093a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 -- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6864100110 msgr2=0x7f686410a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6864100110 0x7f686410a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 -- 172.21.15.187:0/823321844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600095f0 con 0x7f6864100ad0 2023-10-16T14:31:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f6868eb1700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f685400c990 tx=0x7f685400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:00.022 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854007b00 con 0x7f6864100ad0 2023-10-16T14:31:00.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6854007c60 con 0x7f6864100ad0 2023-10-16T14:31:00.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68641a0f00 con 0x7f6864100ad0 2023-10-16T14:31:00.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854018660 con 0x7f6864100ad0 2023-10-16T14:31:00.023 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.013+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68641a1370 con 0x7f6864100ad0 2023-10-16T14:31:00.024 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.014+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6864107420 con 0x7f6864100ad0 2023-10-16T14:31:00.024 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.015+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6854004020 con 0x7f6864100ad0 2023-10-16T14:31:00.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.016+0000 7f685a7fc700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 0x7f68500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:00.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.016+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f68540cc920 con 0x7f6864100ad0 2023-10-16T14:31:00.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.016+0000 7f68696b2700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 0x7f68500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:00.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.016+0000 7f68696b2700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 0x7f68500a3940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6860005f50 tx=0x7f6860005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:00.025 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.019+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f68540964b0 con 0x7f6864100ad0 2023-10-16T14:31:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:30:59 smithi187 ceph-mon[135297]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:00.220 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.216+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f686402d3d0 con 0x7f68500a1480 2023-10-16T14:31:00.228 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.225+0000 7f685a7fc700 1 -- 172.21.15.187:0/823321844 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f686402d3d0 con 0x7f68500a1480 2023-10-16T14:31:00.228 INFO:teuthology.orchestra.run.smithi187.stdout:Scheduled mds.all update... 2023-10-16T14:31:00.230 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 msgr2=0x7f68500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:00.230 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 0x7f68500a3940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6860005f50 tx=0x7f6860005e80 comp rx=0 tx=0).stop 2023-10-16T14:31:00.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 msgr2=0x7f6864108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:00.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f685400c990 tx=0x7f685400cd50 comp rx=0 tx=0).stop 2023-10-16T14:31:00.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 shutdown_connections 2023-10-16T14:31:00.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f68500a1480 0x7f68500a3940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.231 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f68641014b0 0x7f68641093a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.232 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6864100110 0x7f686410a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.232 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 --2- 172.21.15.187:0/823321844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6864100ad0 0x7f6864108e60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:00.232 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 >> 172.21.15.187:0/823321844 conn(0x7f68640fac00 msgr2=0x7f6864104560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:00.232 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 shutdown_connections 2023-10-16T14:31:00.232 INFO:teuthology.orchestra.run.smithi187.stderr:2023-10-16T14:31:00.227+0000 7f686b916700 1 -- 172.21.15.187:0/823321844 wait complete. 2023-10-16T14:31:00.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:30:59 smithi178 ceph-mon[98495]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:00.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:30:59 smithi186 ceph-mon[134892]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:00.659 DEBUG:teuthology.orchestra.run.smithi178:mds.a> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.a.service 2023-10-16T14:31:00.665 DEBUG:teuthology.orchestra.run.smithi178:mds.d> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.d.service 2023-10-16T14:31:00.710 DEBUG:teuthology.orchestra.run.smithi178:mds.g> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.g.service 2023-10-16T14:31:00.745 INFO:journalctl@ceph.mds.a.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:31:00.749 DEBUG:teuthology.orchestra.run.smithi178:mds.j> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.j.service 2023-10-16T14:31:00.785 INFO:journalctl@ceph.mds.d.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:31:00.789 DEBUG:teuthology.orchestra.run.smithi186:mds.b> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.b.service 2023-10-16T14:31:00.792 DEBUG:teuthology.orchestra.run.smithi186:mds.e> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.e.service 2023-10-16T14:31:00.821 INFO:journalctl@ceph.mds.b.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:31:00.822 INFO:journalctl@ceph.mds.g.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:31:00.823 DEBUG:teuthology.orchestra.run.smithi186:mds.h> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.h.service 2023-10-16T14:31:00.848 INFO:journalctl@ceph.mds.e.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:31:00.849 DEBUG:teuthology.orchestra.run.smithi186:mds.k> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.k.service 2023-10-16T14:31:00.860 INFO:journalctl@ceph.mds.j.smithi178.stdout:-- Logs begin at Mon 2023-10-16 14:07:29 UTC. -- 2023-10-16T14:31:00.874 INFO:journalctl@ceph.mds.h.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:31:00.875 DEBUG:teuthology.orchestra.run.smithi187:mds.c> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.c.service 2023-10-16T14:31:00.877 DEBUG:teuthology.orchestra.run.smithi187:mds.f> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.f.service 2023-10-16T14:31:00.900 INFO:journalctl@ceph.mds.k.smithi186.stdout:-- Logs begin at Mon 2023-10-16 14:03:15 UTC. -- 2023-10-16T14:31:00.906 INFO:journalctl@ceph.mds.c.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:31:00.907 DEBUG:teuthology.orchestra.run.smithi187:mds.i> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.i.service 2023-10-16T14:31:00.933 INFO:journalctl@ceph.mds.f.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:31:00.934 DEBUG:teuthology.orchestra.run.smithi187:mds.l> sudo journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.l.service 2023-10-16T14:31:00.959 INFO:journalctl@ceph.mds.i.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:31:00.960 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-10-16T14:31:00.961 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:00.985 INFO:journalctl@ceph.mds.l.smithi187.stdout:-- Logs begin at Mon 2023-10-16 14:02:52 UTC. -- 2023-10-16T14:31:01.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4107677756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8100eb0 msgr2=0x7f2eb81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4107677756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8100eb0 0x7f2eb81012d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009990 tx=0x7f2eb4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4107677756 shutdown_connections 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4107677756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8102a50 0x7f2eb8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4107677756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8100eb0 0x7f2eb81012d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4107677756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102090 0x7f2eb8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4107677756 >> 172.21.15.178:0/4107677756 conn(0x7f2eb80fc430 msgr2=0x7f2eb80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:01.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4107677756 shutdown_connections 2023-10-16T14:31:01.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.188+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4107677756 wait complete. 2023-10-16T14:31:01.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 Processor -- start 2023-10-16T14:31:01.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 -- start start 2023-10-16T14:31:01.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 0x7f2eb8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8102090 0x7f2eb8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 0x7f2eb81a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81a08c0 con 0x7f2eb8100eb0 2023-10-16T14:31:01.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81a0a00 con 0x7f2eb8102a50 2023-10-16T14:31:01.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ec05ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb81a0b90 con 0x7f2eb8102090 2023-10-16T14:31:01.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ebe39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 0x7f2eb8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ebdb9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8102090 0x7f2eb8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ebeb9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 0x7f2eb81a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ebe39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 0x7f2eb8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51196/0 (socket says 172.21.15.178:51196) 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.190+0000 7f2ebe39b700 1 -- 172.21.15.178:0/4212656942 learned_addr learned my addr 172.21.15.178:0/4212656942 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 -- 172.21.15.178:0/4212656942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8102090 msgr2=0x7f2eb8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8102090 0x7f2eb8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 -- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 msgr2=0x7f2eb8198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 0x7f2eb8198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 -- 172.21.15.178:0/4212656942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40095f0 con 0x7f2eb8102a50 2023-10-16T14:31:01.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ebeb9c700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 0x7f2eb81a02f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000efc0 tx=0x7f2eb000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:01.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0018070 con 0x7f2eb8102a50 2023-10-16T14:31:01.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb81a0e10 con 0x7f2eb8102a50 2023-10-16T14:31:01.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eb000f040 con 0x7f2eb8102a50 2023-10-16T14:31:01.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb000cd40 con 0x7f2eb8102a50 2023-10-16T14:31:01.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.191+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81a1240 con 0x7f2eb8102a50 2023-10-16T14:31:01.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.192+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb810c770 con 0x7f2eb8102a50 2023-10-16T14:31:01.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.194+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2eb0003a00 con 0x7f2eb8102a50 2023-10-16T14:31:01.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.195+0000 7f2eaf7fe700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 0x7f2ea40a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.196+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2eb00cc5a0 con 0x7f2eb8102a50 2023-10-16T14:31:01.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.196+0000 7f2ebe39b700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 0x7f2ea40a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.196+0000 7f2ebe39b700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 0x7f2ea40a39e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b2e0 tx=0x7f2eb4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:01.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.198+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2eb0096090 con 0x7f2eb8102a50 2023-10-16T14:31:01.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:31:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: Saving service mds.all spec with placement smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-10-16T14:31:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:01 smithi178 ceph-mon[98495]: Deploying daemon mds.l on smithi187 2023-10-16T14:31:01.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.405+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb81a1430 con 0x7f2eb8102a50 2023-10-16T14:31:01.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.407+0000 7f2eaf7fe700 1 -- 172.21.15.178:0/4212656942 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18482 (secure 0 0 0) 0x7f2eb00996b0 con 0x7f2eb8102a50 2023-10-16T14:31:01.410 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:01.410 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":79,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:30:49.840789+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 msgr2=0x7f2ea40a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 0x7f2ea40a39e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400b2e0 tx=0x7f2eb4005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 msgr2=0x7f2eb81a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 0x7f2eb81a02f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000efc0 tx=0x7f2eb000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 shutdown_connections 2023-10-16T14:31:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2ea40a1520 0x7f2ea40a39e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2eb8102a50 0x7f2eb81a02f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb8100eb0 0x7f2eb8198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 --2- 172.21.15.178:0/4212656942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2eb8102090 0x7f2eb8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 >> 172.21.15.178:0/4212656942 conn(0x7f2eb80fc430 msgr2=0x7f2eb8107970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:01.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.409+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 shutdown_connections 2023-10-16T14:31:01.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.410+0000 7f2ec05ff700 1 -- 172.21.15.178:0/4212656942 wait complete. 2023-10-16T14:31:01.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T14:31:01.433 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:31:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:31:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: Saving service mds.all spec with placement smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-10-16T14:31:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:01.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:01.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:01.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:01 smithi186 ceph-mon[134892]: Deploying daemon mds.l on smithi187 2023-10-16T14:31:01.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:31:01.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: Saving service mds.all spec with placement smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-10-16T14:31:01.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.587 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:01.588 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:01.589 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:01.589 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:01 smithi187 ceph-mon[135297]: Deploying daemon mds.l on smithi187 2023-10-16T14:31:01.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 -- 172.21.15.178:0/1455268705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440100e80 msgr2=0x7f44401012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 --2- 172.21.15.178:0/1455268705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440100e80 0x7f44401012a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4434009990 tx=0x7f4434009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 -- 172.21.15.178:0/1455268705 shutdown_connections 2023-10-16T14:31:01.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 --2- 172.21.15.178:0/1455268705 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 0x7f4440109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 --2- 172.21.15.178:0/1455268705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440100e80 0x7f44401012a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 --2- 172.21.15.178:0/1455268705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440102060 0x7f44401024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 -- 172.21.15.178:0/1455268705 >> 172.21.15.178:0/1455268705 conn(0x7f44400fc400 msgr2=0x7f44400fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:01.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 -- 172.21.15.178:0/1455268705 shutdown_connections 2023-10-16T14:31:01.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.631+0000 7f4446fff700 1 -- 172.21.15.178:0/1455268705 wait complete. 2023-10-16T14:31:01.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.632+0000 7f4446fff700 1 Processor -- start 2023-10-16T14:31:01.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.632+0000 7f4446fff700 1 -- start start 2023-10-16T14:31:01.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 0x7f444019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019e800 con 0x7f4440100e80 2023-10-16T14:31:01.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019e970 con 0x7f4440102a20 2023-10-16T14:31:01.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4446fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019eb30 con 0x7f4440102060 2023-10-16T14:31:01.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4444d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:58158/0 (socket says 172.21.15.178:58158) 2023-10-16T14:31:01.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4444d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51226/0 (socket says 172.21.15.178:51226) 2023-10-16T14:31:01.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 -- 172.21.15.178:0/2503580560 learned_addr learned my addr 172.21.15.178:0/2503580560 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:01.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f444559c700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 0x7f444019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 -- 172.21.15.178:0/2503580560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 msgr2=0x7f444019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 0x7f444019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 -- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 msgr2=0x7f4440196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f443ffff700 1 -- 172.21.15.178:0/2503580560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442c009520 con 0x7f4440102060 2023-10-16T14:31:01.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.633+0000 7f4444d9b700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:01.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f443ffff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f442c00ea50 tx=0x7f442c00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:01.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00cbd0 con 0x7f4440102060 2023-10-16T14:31:01.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c00cd30 con 0x7f4440102060 2023-10-16T14:31:01.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44340095f0 con 0x7f4440102060 2023-10-16T14:31:01.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c018680 con 0x7f4440102060 2023-10-16T14:31:01.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.634+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444019f190 con 0x7f4440102060 2023-10-16T14:31:01.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.635+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444010c740 con 0x7f4440102060 2023-10-16T14:31:01.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.637+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f442c010430 con 0x7f4440102060 2023-10-16T14:31:01.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.638+0000 7f443dffb700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 0x7f44300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:01.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.638+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f442c014070 con 0x7f4440102060 2023-10-16T14:31:01.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.639+0000 7f4444d9b700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 0x7f44300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:01.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.639+0000 7f4444d9b700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 0x7f44300a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4434000c00 tx=0x7f4434005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:01.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.640+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f442c096980 con 0x7f4440102060 2023-10-16T14:31:01.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.842+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4440107ec0 con 0x7f4440102060 2023-10-16T14:31:01.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.842+0000 7f443dffb700 1 -- 172.21.15.178:0/2503580560 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f442c099fa0 con 0x7f4440102060 2023-10-16T14:31:01.845 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:31:01.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 msgr2=0x7f44300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 0x7f44300a3940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4434000c00 tx=0x7f4434005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 msgr2=0x7f44401971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:01.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f442c00ea50 tx=0x7f442c00ed60 comp rx=0 tx=0).stop 2023-10-16T14:31:01.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 shutdown_connections 2023-10-16T14:31:01.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f44300a1480 0x7f44300a3940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4440102a20 0x7f444019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4440100e80 0x7f4440196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.844+0000 7f4446fff700 1 --2- 172.21.15.178:0/2503580560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4440102060 0x7f44401971d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:01.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.845+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 >> 172.21.15.178:0/2503580560 conn(0x7f44400fc400 msgr2=0x7f44400fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:01.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.845+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 shutdown_connections 2023-10-16T14:31:01.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:01.845+0000 7f4446fff700 1 -- 172.21.15.178:0/2503580560 wait complete. 2023-10-16T14:31:01.867 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:02.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 -- 172.21.15.178:0/1500307456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec102080 msgr2=0x7fa3ec102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:02.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/1500307456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec102080 0x7fa3ec102500 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 -- 172.21.15.178:0/1500307456 shutdown_connections 2023-10-16T14:31:02.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/1500307456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 0x7fa3ec109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/1500307456 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/1500307456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec102080 0x7fa3ec102500 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 -- 172.21.15.178:0/1500307456 >> 172.21.15.178:0/1500307456 conn(0x7fa3ec0fc420 msgr2=0x7fa3ec0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:02.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 -- 172.21.15.178:0/1500307456 shutdown_connections 2023-10-16T14:31:02.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.063+0000 7fa3f13ff700 1 -- 172.21.15.178:0/1500307456 wait complete. 2023-10-16T14:31:02.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 Processor -- start 2023-10-16T14:31:02.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 -- start start 2023-10-16T14:31:02.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:02.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec102080 0x7fa3ec197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 0x7fa3ec19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec19e6d0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec19e810 con 0x7fa3ec102080 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.064+0000 7fa3f13ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec19e9a0 con 0x7fa3ec102a40 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:02.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51242/0 (socket says 172.21.15.178:51242) 2023-10-16T14:31:02.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 -- 172.21.15.178:0/2943839514 learned_addr learned my addr 172.21.15.178:0/2943839514 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:02.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eb7fe700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 0x7fa3ec19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:02.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3ea7fc700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec102080 0x7fa3ec197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:02.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 -- 172.21.15.178:0/2943839514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 msgr2=0x7fa3ec19e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:02.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 0x7fa3ec19e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 -- 172.21.15.178:0/2943839514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec102080 msgr2=0x7fa3ec197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:02.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec102080 0x7fa3ec197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 -- 172.21.15.178:0/2943839514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00095f0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3eaffd700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e970 tx=0x7fa3dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:02.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc009790 con 0x7fa3ec100ea0 2023-10-16T14:31:02.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3dc00eec0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.065+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec19ec80 con 0x7fa3ec100ea0 2023-10-16T14:31:02.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.066+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc003cf0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.066+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec19f190 con 0x7fa3ec100ea0 2023-10-16T14:31:02.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.067+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec10c760 con 0x7fa3ec100ea0 2023-10-16T14:31:02.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.069+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa3dc00ccd0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.070+0000 7fa3d3fff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 0x7fa3d40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:02.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.070+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa3dc014070 con 0x7fa3ec100ea0 2023-10-16T14:31:02.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.070+0000 7fa3ea7fc700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 0x7fa3d40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:02.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.071+0000 7fa3ea7fc700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 0x7fa3d40a3a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b2e0 tx=0x7fa3e0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:02.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.074+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa3dc096580 con 0x7fa3ec100ea0 2023-10-16T14:31:02.282 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4212656942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:02.282 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2503580560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:02.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.327+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3ec10c930 con 0x7fa3ec100ea0 2023-10-16T14:31:02.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.327+0000 7fa3d3fff700 1 -- 172.21.15.178:0/2943839514 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fa3dc099ba0 con 0x7fa3ec100ea0 2023-10-16T14:31:02.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:02.330 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-10-16T14:31:02.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.329+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 msgr2=0x7fa3d40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:02.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.329+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 0x7fa3d40a3a60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000b2e0 tx=0x7fa3e0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.329+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 msgr2=0x7fa3ec196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:02.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.329+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e970 tx=0x7fa3dc00ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:02.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 shutdown_connections 2023-10-16T14:31:02.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa3d40a15a0 0x7fa3d40a3a60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa3ec102a40 0x7fa3ec19e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3ec100ea0 0x7fa3ec196b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 --2- 172.21.15.178:0/2943839514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa3ec102080 0x7fa3ec197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:02.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 >> 172.21.15.178:0/2943839514 conn(0x7fa3ec0fc420 msgr2=0x7fa3ec0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:02.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 shutdown_connections 2023-10-16T14:31:02.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:02.330+0000 7fa3f13ff700 1 -- 172.21.15.178:0/2943839514 wait complete. 2023-10-16T14:31:02.335 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 1 2023-10-16T14:31:02.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4212656942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:02.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2503580560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:02.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4212656942' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:02.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2503580560' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:02.674 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:02 smithi187 systemd[1]: Starting Ceph mds.l for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:02.991 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:02 smithi187 podman[172519]: 2023-10-16T14:31:03.425 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:02 smithi187 ceph-mds[172559]: -- 172.21.15.187:0/3603621477 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557dd5db31e0 con 0x557dd5d88800 2023-10-16T14:31:03.425 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-l[172535]: starting mds.l at 2023-10-16T14:31:03.425 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:02 smithi187 bash[172519]: 8e8bdfcd52696db5ba2a6e6dee2e5d792923afb71b5b7af31362195791762188 2023-10-16T14:31:03.425 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:31:03 smithi187 systemd[1]: Started Ceph mds.l for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2943839514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6832/1854237656,v1:172.21.15.187:6833/1854237656] up:boot 2023-10-16T14:31:03.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: fsmap 1 up:standby 2023-10-16T14:31:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-16T14:31:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:03.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:03.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:03 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:03.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2943839514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:03.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:03.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6832/1854237656,v1:172.21.15.187:6833/1854237656] up:boot 2023-10-16T14:31:03.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: fsmap 1 up:standby 2023-10-16T14:31:03.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-16T14:31:03.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:03.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:03.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:03 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2943839514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6832/1854237656,v1:172.21.15.187:6833/1854237656] up:boot 2023-10-16T14:31:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: fsmap 1 up:standby 2023-10-16T14:31:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-10-16T14:31:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:03.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:03.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:03 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:04.351 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:04.425 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:04 smithi186 ceph-mon[134892]: Deploying daemon mds.k on smithi186 2023-10-16T14:31:04.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:04 smithi178 ceph-mon[98495]: Deploying daemon mds.k on smithi186 2023-10-16T14:31:04.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 -- 172.21.15.178:0/3979940934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 msgr2=0x7f4490109020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:04.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 --2- 172.21.15.178:0/3979940934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f4490109020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f448800b0a0 tx=0x7f448800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 -- 172.21.15.178:0/3979940934 shutdown_connections 2023-10-16T14:31:04.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 --2- 172.21.15.178:0/3979940934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f4490109020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 --2- 172.21.15.178:0/3979940934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490101950 0x7f4490103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 --2- 172.21.15.178:0/3979940934 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490104280 0x7f4490106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 -- 172.21.15.178:0/3979940934 >> 172.21.15.178:0/3979940934 conn(0x7f44900fb320 msgr2=0x7f44900fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:04.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 -- 172.21.15.178:0/3979940934 shutdown_connections 2023-10-16T14:31:04.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.555+0000 7f44981ff700 1 -- 172.21.15.178:0/3979940934 wait complete. 2023-10-16T14:31:04.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.556+0000 7f44981ff700 1 Processor -- start 2023-10-16T14:31:04.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.556+0000 7f44981ff700 1 -- start start 2023-10-16T14:31:04.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.556+0000 7f44981ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:04.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f44981ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490104280 0x7f4490197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:04.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f44981ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f449019e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:04.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f44981ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019e760 con 0x7f4490106bb0 2023-10-16T14:31:04.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f44981ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019e8a0 con 0x7f4490101950 2023-10-16T14:31:04.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f44981ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449019ea30 con 0x7f4490104280 2023-10-16T14:31:04.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:04.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:51752/0 (socket says 172.21.15.178:51752) 2023-10-16T14:31:04.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 -- 172.21.15.178:0/1310714287 learned_addr learned my addr 172.21.15.178:0/1310714287 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:04.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f449679c700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f449019e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:04.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f449579a700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490104280 0x7f4490197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:04.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 -- 172.21.15.178:0/1310714287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490104280 msgr2=0x7f4490197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:04.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490104280 0x7f4490197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 -- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 msgr2=0x7f449019e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:04.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f449019e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 -- 172.21.15.178:0/1310714287 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488009d00 con 0x7f4490101950 2023-10-16T14:31:04.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.557+0000 7f4495f9b700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e9a0 tx=0x7f448c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:04.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.558+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009790 con 0x7f4490101950 2023-10-16T14:31:04.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.558+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448c00cbd0 con 0x7f4490101950 2023-10-16T14:31:04.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.558+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449019ed10 con 0x7f4490101950 2023-10-16T14:31:04.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.558+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c018680 con 0x7f4490101950 2023-10-16T14:31:04.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.558+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449019f1c0 con 0x7f4490101950 2023-10-16T14:31:04.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.559+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44900fcea0 con 0x7f4490101950 2023-10-16T14:31:04.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.561+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f448c00cd40 con 0x7f4490101950 2023-10-16T14:31:04.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.562+0000 7f4486ffd700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 0x7f447c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:04.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.562+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f448c014070 con 0x7f4490101950 2023-10-16T14:31:04.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.562+0000 7f449579a700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 0x7f447c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:04.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.563+0000 7f449579a700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 0x7f447c0a3a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4480005950 tx=0x7f44800058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:04.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.566+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f448c052100 con 0x7f4490101950 2023-10-16T14:31:04.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:04 smithi187 ceph-mon[135297]: Deploying daemon mds.k on smithi186 2023-10-16T14:31:04.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.813+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f449002d090 con 0x7f4490101950 2023-10-16T14:31:04.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.813+0000 7f4486ffd700 1 -- 172.21.15.178:0/1310714287 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f448c096530 con 0x7f4490101950 2023-10-16T14:31:04.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:04.816 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T14:31:04.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 msgr2=0x7f447c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:04.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 0x7f447c0a3a60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4480005950 tx=0x7f44800058e0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 msgr2=0x7f4490196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:04.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f448c00e9a0 tx=0x7f448c00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 shutdown_connections 2023-10-16T14:31:04.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f447c0a15a0 0x7f447c0a3a60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4490106bb0 0x7f449019e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4490101950 0x7f4490196be0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 --2- 172.21.15.178:0/1310714287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4490104280 0x7f4490197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:04.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 >> 172.21.15.178:0/1310714287 conn(0x7f44900fb320 msgr2=0x7f44901025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:04.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 shutdown_connections 2023-10-16T14:31:04.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:04.816+0000 7f44981ff700 1 -- 172.21.15.178:0/1310714287 wait complete. 2023-10-16T14:31:04.821 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 2 2023-10-16T14:31:04.826 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:04 smithi186 systemd[1]: Starting Ceph mds.k for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:05.247 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:04 smithi186 podman[169945]: 2023-10-16T14:31:05.248 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:05 smithi186 bash[169945]: 3f0981aa9daa0c4bda1f0d5bd5128241f715b5db3a9ab11a94c95ecdeaa03f0c 2023-10-16T14:31:05.248 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:05 smithi186 ceph-mds[169985]: -- 172.21.15.186:0/1552387026 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55747de351e0 con 0x55747de0b000 2023-10-16T14:31:05.248 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-k[169961]: starting mds.k at 2023-10-16T14:31:05.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:05 smithi178 ceph-mon[98495]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:05.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1310714287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:05.576 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:31:05 smithi186 systemd[1]: Started Ceph mds.k for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:05 smithi186 ceph-mon[134892]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:05.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1310714287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:05 smithi187 ceph-mon[135297]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:05.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1310714287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:06.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:06.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:06.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: Deploying daemon mds.j on smithi178 2023-10-16T14:31:06.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6832/3571012275,v1:172.21.15.186:6833/3571012275] up:boot 2023-10-16T14:31:06.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: fsmap 2 up:standby 2023-10-16T14:31:06.555 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-16T14:31:06.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:06.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:06.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:06.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: Deploying daemon mds.j on smithi178 2023-10-16T14:31:06.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6832/3571012275,v1:172.21.15.186:6833/3571012275] up:boot 2023-10-16T14:31:06.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: fsmap 2 up:standby 2023-10-16T14:31:06.578 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-16T14:31:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: Deploying daemon mds.j on smithi178 2023-10-16T14:31:06.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6832/3571012275,v1:172.21.15.186:6833/3571012275] up:boot 2023-10-16T14:31:06.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: fsmap 2 up:standby 2023-10-16T14:31:06.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-10-16T14:31:06.838 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:07.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 -- 172.21.15.178:0/3418139260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678101760 msgr2=0x7f2678103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 --2- 172.21.15.178:0/3418139260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678101760 0x7f2678103b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f26740098f0 tx=0x7f2674009c00 comp rx=0 tx=0).stop 2023-10-16T14:31:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 -- 172.21.15.178:0/3418139260 shutdown_connections 2023-10-16T14:31:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 --2- 172.21.15.178:0/3418139260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26781069c0 0x7f2678108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 --2- 172.21.15.178:0/3418139260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678101760 0x7f2678103b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 --2- 172.21.15.178:0/3418139260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2678104090 0x7f2678106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.047+0000 7f267f39d700 1 -- 172.21.15.178:0/3418139260 >> 172.21.15.178:0/3418139260 conn(0x7f26780fb150 msgr2=0x7f26780fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.048+0000 7f267f39d700 1 -- 172.21.15.178:0/3418139260 shutdown_connections 2023-10-16T14:31:07.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.048+0000 7f267f39d700 1 -- 172.21.15.178:0/3418139260 wait complete. 2023-10-16T14:31:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.049+0000 7f267f39d700 1 Processor -- start 2023-10-16T14:31:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 -- start start 2023-10-16T14:31:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:07.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 0x7f2678196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26781069c0 0x7f267819dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267819e580 con 0x7f26781069c0 2023-10-16T14:31:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267819e6c0 con 0x7f2678101760 2023-10-16T14:31:07.054 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.050+0000 7f267f39d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780ff550 con 0x7f2678104090 2023-10-16T14:31:07.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:51770/0 (socket says 172.21.15.178:51770) 2023-10-16T14:31:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 -- 172.21.15.178:0/1653792827 learned_addr learned my addr 172.21.15.178:0/1653792827 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267db9a700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 0x7f2678196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267eb9c700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26781069c0 0x7f267819dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:07.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 -- 172.21.15.178:0/1653792827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 msgr2=0x7f2678196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 0x7f2678196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 -- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26781069c0 msgr2=0x7f267819dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26781069c0 0x7f267819dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.051+0000 7f267e39b700 1 -- 172.21.15.178:0/1653792827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2674009520 con 0x7f2678101760 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.052+0000 7f267db9a700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 0x7f2678196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:07.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.052+0000 7f267e39b700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f267400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.052+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267401c070 con 0x7f2678101760 2023-10-16T14:31:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.052+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f267400f7d0 con 0x7f2678101760 2023-10-16T14:31:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.052+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674004b10 con 0x7f2678101760 2023-10-16T14:31:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.053+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26780ff7d0 con 0x7f2678101760 2023-10-16T14:31:07.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.053+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780ffc30 con 0x7f2678101760 2023-10-16T14:31:07.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.055+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267810c510 con 0x7f2678101760 2023-10-16T14:31:07.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.058+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2674020070 con 0x7f2678101760 2023-10-16T14:31:07.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.061+0000 7f266f7fe700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 0x7f26640a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:07.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.061+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f26740cd460 con 0x7f2678101760 2023-10-16T14:31:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.062+0000 7f267db9a700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 0x7f26640a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:07.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.063+0000 7f267db9a700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 0x7f26640a3a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2668005fd0 tx=0x7f2668009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:07.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.064+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f267405d100 con 0x7f2678101760 2023-10-16T14:31:07.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.334+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f267810c6f0 con 0x7f2678101760 2023-10-16T14:31:07.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.335+0000 7f266f7fe700 1 -- 172.21.15.178:0/1653792827 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f2674096fd0 con 0x7f2678101760 2023-10-16T14:31:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:07.338 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T14:31:07.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 msgr2=0x7f26640a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:07.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 0x7f26640a3a90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2668005fd0 tx=0x7f2668009310 comp rx=0 tx=0).stop 2023-10-16T14:31:07.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 msgr2=0x7f2678196a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:07.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2674009fd0 tx=0x7f267400b840 comp rx=0 tx=0).stop 2023-10-16T14:31:07.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 shutdown_connections 2023-10-16T14:31:07.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26640a15d0 0x7f26640a3a90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26781069c0 0x7f267819dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2678101760 0x7f2678196a00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 --2- 172.21.15.178:0/1653792827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2678104090 0x7f2678196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:07.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.337+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 >> 172.21.15.178:0/1653792827 conn(0x7f26780fb150 msgr2=0x7f26781023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:07.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.338+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 shutdown_connections 2023-10-16T14:31:07.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:07.338+0000 7f267f39d700 1 -- 172.21.15.178:0/1653792827 wait complete. 2023-10-16T14:31:07.343 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 3 2023-10-16T14:31:07.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:07 smithi178 ceph-mon[98495]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:07 smithi186 ceph-mon[134892]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:07.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:07 smithi187 ceph-mon[135297]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:08.053 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:07 smithi178 systemd[1]: Starting Ceph mds.j for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:08.352 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:08 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1653792827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:08.352 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:08 smithi178 podman[148182]: 2023-10-16T14:31:08.353 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:08 smithi178 ceph-mds[148222]: -- 172.21.15.178:0/2567494524 <== mon.0 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5581321b11e0 con 0x558132188800 2023-10-16T14:31:08.353 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-j[148198]: starting mds.j at 2023-10-16T14:31:08.353 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:08 smithi178 bash[148182]: 562f27405a8a84e911c7f39cd35f59b67581efc03dfa7f9dde642973da794295 2023-10-16T14:31:08.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:08 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1653792827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:08.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:08 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1653792827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:08.802 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:31:08 smithi178 systemd[1]: Started Ceph mds.j for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:09.363 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/3227764024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8074ef0 msgr2=0x7fc6c8073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/3227764024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8074ef0 0x7fc6c8073340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009990 tx=0x7fc6b8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/3227764024 shutdown_connections 2023-10-16T14:31:09.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/3227764024 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8108dc0 0x7fc6c810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/3227764024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8074ef0 0x7fc6c8073340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/3227764024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8073910 0x7fc6c8073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/3227764024 >> 172.21.15.178:0/3227764024 conn(0x7fc6c80fc430 msgr2=0x7fc6c80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/3227764024 shutdown_connections 2023-10-16T14:31:09.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.577+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/3227764024 wait complete. 2023-10-16T14:31:09.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.578+0000 7fc6ce7ff700 1 Processor -- start 2023-10-16T14:31:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 -- start start 2023-10-16T14:31:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 0x7fc6c8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 0x7fc6c81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:09.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a08e0 con 0x7fc6c8108dc0 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a0a50 con 0x7fc6c8073910 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ce7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81a0c10 con 0x7fc6c8074ef0 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ccd9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 0x7fc6c8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 0x7fc6c81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:09.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6ccd9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39534/0 (socket says 172.21.15.178:39534) 2023-10-16T14:31:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.579+0000 7fc6c77fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 0x7fc6c81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:41020/0 (socket says 172.21.15.178:41020) 2023-10-16T14:31:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6ccd9c700 1 -- 172.21.15.178:0/2125959706 learned_addr learned my addr 172.21.15.178:0/2125959706 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 -- 172.21.15.178:0/2125959706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 msgr2=0x7fc6c81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 0x7fc6c81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 -- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 msgr2=0x7fc6c81a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 -- 172.21.15.178:0/2125959706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80095f0 con 0x7fc6c8073910 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6ccd9c700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c7fff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 0x7fc6c8198da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098e0 tx=0x7fc6b8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.580+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b801d070 con 0x7fc6c8073910 2023-10-16T14:31:09.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.581+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6b8021410 con 0x7fc6c8073910 2023-10-16T14:31:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.581+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c81a0e90 con 0x7fc6c8073910 2023-10-16T14:31:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.581+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8005720 con 0x7fc6c8073910 2023-10-16T14:31:09.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.581+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c81a1200 con 0x7fc6c8073910 2023-10-16T14:31:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.582+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c8107470 con 0x7fc6c8073910 2023-10-16T14:31:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.584+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc6b8021580 con 0x7fc6c8073910 2023-10-16T14:31:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.585+0000 7fc6c57fa700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 0x7fc6b00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:09.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.585+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc6b80cdcf0 con 0x7fc6c8073910 2023-10-16T14:31:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.585+0000 7fc6c77fe700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 0x7fc6b00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.586+0000 7fc6c77fe700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 0x7fc6b00a39e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8074c10 tx=0x7fc6bc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:09.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.587+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc6b8097860 con 0x7fc6c8073910 2023-10-16T14:31:09.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:09.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:09.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:09.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: Deploying daemon mds.d on smithi178 2023-10-16T14:31:09.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:09.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.178:6834/1662555012,v1:172.21.15.178:6835/1662555012] up:boot 2023-10-16T14:31:09.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: fsmap 3 up:standby 2023-10-16T14:31:09.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-16T14:31:09.731 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.731 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.731 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.732 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:09.732 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:09.732 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:09.732 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: Deploying daemon mds.d on smithi178 2023-10-16T14:31:09.732 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:09.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.178:6834/1662555012,v1:172.21.15.178:6835/1662555012] up:boot 2023-10-16T14:31:09.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: fsmap 3 up:standby 2023-10-16T14:31:09.733 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-16T14:31:09.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:09.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:09.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:09.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:09.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: Deploying daemon mds.d on smithi178 2023-10-16T14:31:09.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:09.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.178:6834/1662555012,v1:172.21.15.178:6835/1662555012] up:boot 2023-10-16T14:31:09.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: fsmap 3 up:standby 2023-10-16T14:31:09.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-10-16T14:31:09.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.847+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6c802d270 con 0x7fc6c8073910 2023-10-16T14:31:09.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.848+0000 7fc6c57fa700 1 -- 172.21.15.178:0/2125959706 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2916 (secure 0 0 0) 0x7fc6b809ae80 con 0x7fc6c8073910 2023-10-16T14:31:09.851 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:09.851 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T14:31:09.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 msgr2=0x7fc6b00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:09.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 0x7fc6b00a39e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8074c10 tx=0x7fc6bc009260 comp rx=0 tx=0).stop 2023-10-16T14:31:09.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 msgr2=0x7fc6c8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:09.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 0x7fc6c8198da0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098e0 tx=0x7fc6b8009580 comp rx=0 tx=0).stop 2023-10-16T14:31:09.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 shutdown_connections 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc6b00a1520 0x7fc6b00a39e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6c8108dc0 0x7fc6c81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc6c8073910 0x7fc6c8198da0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 --2- 172.21.15.178:0/2125959706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6c8074ef0 0x7fc6c81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 >> 172.21.15.178:0/2125959706 conn(0x7fc6c80fc430 msgr2=0x7fc6c80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:09.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 shutdown_connections 2023-10-16T14:31:09.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:09.851+0000 7fc6ce7ff700 1 -- 172.21.15.178:0/2125959706 wait complete. 2023-10-16T14:31:09.856 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 4 2023-10-16T14:31:10.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:10 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2125959706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:10.803 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:10 smithi178 systemd[1]: Starting Ceph mds.d for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:10.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:10 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2125959706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:10.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:10 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2125959706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:11.418 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:11 smithi178 podman[148573]: 2023-10-16T14:31:11.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:11 smithi178 ceph-mon[98495]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:11.803 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:11 smithi178 bash[148573]: 9f05cdfcc262fafc1cf2e39b8439ba47cb09b18728f17740b356d709d7af2622 2023-10-16T14:31:11.803 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:11 smithi178 ceph-mds[148613]: -- 172.21.15.178:0/3485890167 <== mon.0 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56439c8cb1e0 con 0x56439c8a3000 2023-10-16T14:31:11.804 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-d[148589]: starting mds.d at 2023-10-16T14:31:11.804 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:31:11 smithi178 systemd[1]: Started Ceph mds.d for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:11.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:11 smithi186 ceph-mon[134892]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:11.874 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:11.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:11 smithi187 ceph-mon[135297]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:12.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 -- 172.21.15.178:0/1395852871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 msgr2=0x7f0a7c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:12.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1395852871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c1012d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009990 tx=0x7f0a70009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 -- 172.21.15.178:0/1395852871 shutdown_connections 2023-10-16T14:31:12.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1395852871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 0x7f0a7c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1395852871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c1012d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1395852871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.076+0000 7f0a82bff700 1 -- 172.21.15.178:0/1395852871 >> 172.21.15.178:0/1395852871 conn(0x7f0a7c0fc430 msgr2=0x7f0a7c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:12.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.077+0000 7f0a82bff700 1 -- 172.21.15.178:0/1395852871 shutdown_connections 2023-10-16T14:31:12.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.077+0000 7f0a82bff700 1 -- 172.21.15.178:0/1395852871 wait complete. 2023-10-16T14:31:12.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.077+0000 7f0a82bff700 1 Processor -- start 2023-10-16T14:31:12.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 -- start start 2023-10-16T14:31:12.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:12.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:12.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 0x7f0a7c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:12.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c19a9f0 con 0x7f0a7c100eb0 2023-10-16T14:31:12.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c073d20 con 0x7f0a7c102090 2023-10-16T14:31:12.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a82bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c073ee0 con 0x7f0a7c102a50 2023-10-16T14:31:12.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a8099b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:12.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a7bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:12.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a7bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50948/0 (socket says 172.21.15.178:50948) 2023-10-16T14:31:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a7bfff700 1 -- 172.21.15.178:0/1659497910 learned_addr learned my addr 172.21.15.178:0/1659497910 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a8099b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39570/0 (socket says 172.21.15.178:39570) 2023-10-16T14:31:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.078+0000 7f0a8119c700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 0x7f0a7c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:12.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 -- 172.21.15.178:0/1659497910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 msgr2=0x7f0a7c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:12.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 0x7f0a7c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 -- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 msgr2=0x7f0a7c074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:12.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 -- 172.21.15.178:0/1659497910 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a700095f0 con 0x7f0a7c102090 2023-10-16T14:31:12.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a8099b700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:12.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a7bfff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e970 tx=0x7f0a6800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:12.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68009790 con 0x7f0a7c102090 2023-10-16T14:31:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c1a53a0 con 0x7f0a7c102090 2023-10-16T14:31:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a6800c7e0 con 0x7f0a7c102090 2023-10-16T14:31:12.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c1a5820 con 0x7f0a7c102090 2023-10-16T14:31:12.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.079+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68003be0 con 0x7f0a7c102090 2023-10-16T14:31:12.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.080+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c107db0 con 0x7f0a7c102090 2023-10-16T14:31:12.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.082+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0a68003d40 con 0x7f0a7c102090 2023-10-16T14:31:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.084+0000 7f0a79ffb700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 0x7f0a6c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.084+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0a68014070 con 0x7f0a7c102090 2023-10-16T14:31:12.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.084+0000 7f0a8099b700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 0x7f0a6c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:12.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.085+0000 7f0a8099b700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 0x7f0a6c0a39e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000b2e0 tx=0x7f0a70005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:12.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.087+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0a68096530 con 0x7f0a7c102090 2023-10-16T14:31:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.335+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a7c10c770 con 0x7f0a7c102090 2023-10-16T14:31:12.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.336+0000 7f0a79ffb700 1 -- 172.21.15.178:0/1659497910 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3710 (secure 0 0 0) 0x7f0a68099b50 con 0x7f0a7c102090 2023-10-16T14:31:12.339 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:12.339 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[]} 2023-10-16T14:31:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 msgr2=0x7f0a6c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 0x7f0a6c0a39e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0a7000b2e0 tx=0x7f0a70005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 msgr2=0x7f0a7c0732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e970 tx=0x7f0a6800ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 shutdown_connections 2023-10-16T14:31:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0a6c0a1520 0x7f0a6c0a39e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a7c102a50 0x7f0a7c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a7c100eb0 0x7f0a7c074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 --2- 172.21.15.178:0/1659497910 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0a7c102090 0x7f0a7c0732a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 >> 172.21.15.178:0/1659497910 conn(0x7f0a7c0fc430 msgr2=0x7f0a7c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 shutdown_connections 2023-10-16T14:31:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:12.338+0000 7f0a82bff700 1 -- 172.21.15.178:0/1659497910 wait complete. 2023-10-16T14:31:12.344 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 5 2023-10-16T14:31:12.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.178:6836/3662037973,v1:172.21.15.178:6837/3662037973] up:boot 2023-10-16T14:31:12.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: fsmap 4 up:standby 2023-10-16T14:31:12.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-16T14:31:12.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:12.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: Deploying daemon mds.e on smithi186 2023-10-16T14:31:12.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:12 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1659497910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:12.815 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.178:6836/3662037973,v1:172.21.15.178:6837/3662037973] up:boot 2023-10-16T14:31:12.815 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: fsmap 4 up:standby 2023-10-16T14:31:12.816 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-16T14:31:12.816 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.816 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.816 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.816 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:12.817 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:12.817 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:12.817 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: Deploying daemon mds.e on smithi186 2023-10-16T14:31:12.817 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:12 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1659497910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.178:6836/3662037973,v1:172.21.15.178:6837/3662037973] up:boot 2023-10-16T14:31:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: fsmap 4 up:standby 2023-10-16T14:31:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-10-16T14:31:12.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:12.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:12.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:12.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:12.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: Deploying daemon mds.e on smithi186 2023-10-16T14:31:12.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:12 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1659497910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:13.076 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 systemd[1]: Starting Ceph mds.e for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:13.608 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:13 smithi186 ceph-mon[134892]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:13.609 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 podman[170286]: 2023-10-16T14:31:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:13 smithi178 ceph-mon[98495]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:13.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:13 smithi187 ceph-mon[135297]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:14.076 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 ceph-mds[170326]: -- 172.21.15.186:0/2258927304 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560c817fb1e0 con 0x560c817d1000 2023-10-16T14:31:14.076 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: starting mds.e at 2023-10-16T14:31:14.076 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 bash[170286]: 989d7470f469f07eedc241759bf562be3e885f6bd2cf4e2c4eded6a4cead0842 2023-10-16T14:31:14.076 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:31:13 smithi186 systemd[1]: Started Ceph mds.e for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:14.361 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/248170895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 msgr2=0x7ff0f4103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/248170895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4103d40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0009990 tx=0x7ff0f0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/248170895 shutdown_connections 2023-10-16T14:31:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/248170895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f4109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/248170895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4103d40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/248170895 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 0x7ff0f4106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/248170895 >> 172.21.15.178:0/248170895 conn(0x7ff0f40fb320 msgr2=0x7ff0f40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:14.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/248170895 shutdown_connections 2023-10-16T14:31:14.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.567+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/248170895 wait complete. 2023-10-16T14:31:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 Processor -- start 2023-10-16T14:31:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 -- start start 2023-10-16T14:31:14.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 0x7ff0f4196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f419e610 con 0x7ff0f4106bb0 2023-10-16T14:31:14.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f419e750 con 0x7ff0f4104280 2023-10-16T14:31:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.568+0000 7ff0fb7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f419e8e0 con 0x7ff0f4101950 2023-10-16T14:31:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39594/0 (socket says 172.21.15.178:39594) 2023-10-16T14:31:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 -- 172.21.15.178:0/1212671394 learned_addr learned my addr 172.21.15.178:0/1212671394 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:14.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f959b700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f8d9a700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 0x7ff0f4196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 -- 172.21.15.178:0/1212671394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 msgr2=0x7ff0f4196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 -- 172.21.15.178:0/1212671394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 msgr2=0x7ff0f4196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:14.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 0x7ff0f4196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 -- 172.21.15.178:0/1212671394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f00095f0 con 0x7ff0f4106bb0 2023-10-16T14:31:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0f9d9c700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00ef30 tx=0x7ff0ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec018070 con 0x7ff0f4106bb0 2023-10-16T14:31:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0ec00f040 con 0x7ff0f4106bb0 2023-10-16T14:31:14.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00cdb0 con 0x7ff0f4106bb0 2023-10-16T14:31:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.569+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f419eb60 con 0x7ff0f4106bb0 2023-10-16T14:31:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.570+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f419f020 con 0x7ff0f4106bb0 2023-10-16T14:31:14.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.571+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40fcea0 con 0x7ff0f4106bb0 2023-10-16T14:31:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.573+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff0ec003a00 con 0x7ff0f4106bb0 2023-10-16T14:31:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.575+0000 7ff0ea7fc700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 0x7ff0e00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:14.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.575+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff0ec0cc510 con 0x7ff0f4106bb0 2023-10-16T14:31:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.575+0000 7ff0f959b700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 0x7ff0e00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:14.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.576+0000 7ff0f959b700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 0x7ff0e00a39e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000b680 tx=0x7ff0f0005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:14.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.579+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff0ec096000 con 0x7ff0f4106bb0 2023-10-16T14:31:14.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.827+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0f410c6b0 con 0x7ff0f4106bb0 2023-10-16T14:31:14.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.827+0000 7ff0ea7fc700 1 -- 172.21.15.178:0/1212671394 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4504 (secure 0 0 0) 0x7ff0ec099620 con 0x7ff0f4106bb0 2023-10-16T14:31:14.831 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:14.831 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2023-10-16T14:31:14.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 msgr2=0x7ff0e00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:14.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 0x7ff0e00a39e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000b680 tx=0x7ff0f0005fd0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 msgr2=0x7ff0f419e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:14.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00ef30 tx=0x7ff0ec00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 shutdown_connections 2023-10-16T14:31:14.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0e00a1520 0x7ff0e00a39e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0f4106bb0 0x7ff0f419e040 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4101950 0x7ff0f4196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 --2- 172.21.15.178:0/1212671394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff0f4104280 0x7ff0f4196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:14.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 >> 172.21.15.178:0/1212671394 conn(0x7ff0f40fb320 msgr2=0x7ff0f41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:14.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 shutdown_connections 2023-10-16T14:31:14.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:14.830+0000 7ff0fb7ff700 1 -- 172.21.15.178:0/1212671394 wait complete. 2023-10-16T14:31:14.836 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 6 2023-10-16T14:31:14.966 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:14.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:14.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:14.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:14.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:14.967 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:14.968 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: Deploying daemon mds.f on smithi187 2023-10-16T14:31:14.968 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6834/1810043383,v1:172.21.15.186:6835/1810043383] up:boot 2023-10-16T14:31:14.968 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: fsmap 5 up:standby 2023-10-16T14:31:14.968 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:31:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:15.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:15.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:15.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: Deploying daemon mds.f on smithi187 2023-10-16T14:31:15.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6834/1810043383,v1:172.21.15.186:6835/1810043383] up:boot 2023-10-16T14:31:15.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: fsmap 5 up:standby 2023-10-16T14:31:15.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:31:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:15.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:15.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:15.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:15.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: Deploying daemon mds.f on smithi187 2023-10-16T14:31:15.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6834/1810043383,v1:172.21.15.186:6835/1810043383] up:boot 2023-10-16T14:31:15.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: fsmap 5 up:standby 2023-10-16T14:31:15.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:31:15.638 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 systemd[1]: Starting Ceph mds.f for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:15.893 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 podman[172861]: 2023-10-16T14:31:15.894 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:15 smithi187 ceph-mon[135297]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:15.894 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:15 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1212671394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:15 smithi178 ceph-mon[98495]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:15 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1212671394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:15 smithi186 ceph-mon[134892]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:15 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1212671394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:16.175 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 ceph-mds[172901]: -- 172.21.15.187:0/2461119734 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c9f257f1e0 con 0x55c9f2554800 2023-10-16T14:31:16.175 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: starting mds.f at 2023-10-16T14:31:16.175 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 bash[172861]: 3cc719f8eae2b70d68bf063b3f9af8e86d38442d5a193d5d4dbfebe4498e8350 2023-10-16T14:31:16.175 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:31:15 smithi187 systemd[1]: Started Ceph mds.f for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:16.855 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:17.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.059+0000 7fe15bbff700 1 -- 172.21.15.178:0/1439379998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 msgr2=0x7fe154103d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:17.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.059+0000 7fe15bbff700 1 --2- 172.21.15.178:0/1439379998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154103d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe150009990 tx=0x7fe150009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 -- 172.21.15.178:0/1439379998 shutdown_connections 2023-10-16T14:31:17.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 --2- 172.21.15.178:0/1439379998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe154106c00 0x7fe154109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 --2- 172.21.15.178:0/1439379998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154103d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 --2- 172.21.15.178:0/1439379998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe1541042d0 0x7fe1541066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 -- 172.21.15.178:0/1439379998 >> 172.21.15.178:0/1439379998 conn(0x7fe1540fb3b0 msgr2=0x7fe1540fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:17.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 -- 172.21.15.178:0/1439379998 shutdown_connections 2023-10-16T14:31:17.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.060+0000 7fe15bbff700 1 -- 172.21.15.178:0/1439379998 wait complete. 2023-10-16T14:31:17.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 Processor -- start 2023-10-16T14:31:17.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 -- start start 2023-10-16T14:31:17.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:17.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1541042d0 0x7fe154199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:17.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:17.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541a0890 con 0x7fe154106c00 2023-10-16T14:31:17.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541a0a00 con 0x7fe1541042d0 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15bbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541a0bc0 con 0x7fe1541019a0 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15a19c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15a19c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39614/0 (socket says 172.21.15.178:39614) 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15a19c700 1 -- 172.21.15.178:0/4043841249 learned_addr learned my addr 172.21.15.178:0/4043841249 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15999b700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:17.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.061+0000 7fe15919a700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1541042d0 0x7fe154199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:17.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 -- 172.21.15.178:0/4043841249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 msgr2=0x7fe154198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:17.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 -- 172.21.15.178:0/4043841249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1541042d0 msgr2=0x7fe154199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:17.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1541042d0 0x7fe154199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 -- 172.21.15.178:0/4043841249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500095f0 con 0x7fe154106c00 2023-10-16T14:31:17.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15999b700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:17.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15a19c700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007ee0 tx=0x7fe14c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:17.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c017070 con 0x7fe154106c00 2023-10-16T14:31:17.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14c00f040 con 0x7fe154106c00 2023-10-16T14:31:17.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1541a0ea0 con 0x7fe154106c00 2023-10-16T14:31:17.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c005580 con 0x7fe154106c00 2023-10-16T14:31:17.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.062+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1541a12c0 con 0x7fe154106c00 2023-10-16T14:31:17.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.063+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1541087d0 con 0x7fe154106c00 2023-10-16T14:31:17.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.065+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe14c00db20 con 0x7fe154106c00 2023-10-16T14:31:17.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.067+0000 7fe14affd700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 0x7fe1400a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:17.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.067+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe14c0cc070 con 0x7fe154106c00 2023-10-16T14:31:17.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.067+0000 7fe15999b700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 0x7fe1400a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:17.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.067+0000 7fe15999b700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 0x7fe1400a39e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b2e0 tx=0x7fe150005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:17.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.071+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe14c0ff100 con 0x7fe154106c00 2023-10-16T14:31:17.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:17.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:17.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:17.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6834/2797759968,v1:172.21.15.187:6835/2797759968] up:boot 2023-10-16T14:31:17.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: fsmap 6 up:standby 2023-10-16T14:31:17.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:16 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:31:17.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:17.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:17.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:17.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6834/2797759968,v1:172.21.15.187:6835/2797759968] up:boot 2023-10-16T14:31:17.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: fsmap 6 up:standby 2023-10-16T14:31:17.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:16 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:31:17.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.319+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe15410c750 con 0x7fe154106c00 2023-10-16T14:31:17.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.320+0000 7fe14affd700 1 -- 172.21.15.178:0/4043841249 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5298 (secure 0 0 0) 0x7fe14c095ad0 con 0x7fe154106c00 2023-10-16T14:31:17.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:17.323 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-10-16T14:31:17.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 msgr2=0x7fe1400a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:17.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 0x7fe1400a39e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b2e0 tx=0x7fe150005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 msgr2=0x7fe1541a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe14c007ee0 tx=0x7fe14c00d3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 shutdown_connections 2023-10-16T14:31:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1400a1520 0x7fe1400a39e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe154106c00 0x7fe1541a02c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe1541019a0 0x7fe154198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 --2- 172.21.15.178:0/4043841249 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe1541042d0 0x7fe154199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 >> 172.21.15.178:0/4043841249 conn(0x7fe1540fb3b0 msgr2=0x7fe154100020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 shutdown_connections 2023-10-16T14:31:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:17.322+0000 7fe15bbff700 1 -- 172.21.15.178:0/4043841249 wait complete. 2023-10-16T14:31:17.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:17.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:17.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:17.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:17.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6834/2797759968,v1:172.21.15.187:6835/2797759968] up:boot 2023-10-16T14:31:17.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: fsmap 6 up:standby 2023-10-16T14:31:17.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:16 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:31:17.330 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 7 2023-10-16T14:31:17.812 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:17 smithi187 systemd[1]: Starting Ceph mds.c for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:18.080 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:17 smithi187 podman[173202]: 2023-10-16T14:31:18.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:17 smithi187 ceph-mon[135297]: Deploying daemon mds.c on smithi187 2023-10-16T14:31:18.080 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:17 smithi187 ceph-mon[135297]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:18.081 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:17 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4043841249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:17 smithi178 ceph-mon[98495]: Deploying daemon mds.c on smithi187 2023-10-16T14:31:18.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:17 smithi178 ceph-mon[98495]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:18.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:17 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4043841249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:18.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:17 smithi186 ceph-mon[134892]: Deploying daemon mds.c on smithi187 2023-10-16T14:31:18.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:17 smithi186 ceph-mon[134892]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:18.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:17 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4043841249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:18.425 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:18 smithi187 ceph-mds[173242]: -- 172.21.15.187:0/3799222816 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a2225391e0 con 0x55a22250e800 2023-10-16T14:31:18.425 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: starting mds.c at 2023-10-16T14:31:18.425 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:18 smithi187 bash[173202]: 90fd5d93f0222b640b902bb33647b868e8f453d59fb7a76de69c262733a42570 2023-10-16T14:31:18.425 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:31:18 smithi187 systemd[1]: Started Ceph mds.c for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:19.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:19.412 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.412 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.412 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.412 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:19.413 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:19.413 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:19.413 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: Deploying daemon mds.h on smithi186 2023-10-16T14:31:19.413 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:19.413 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6836/4191845467,v1:172.21.15.187:6837/4191845467] up:boot 2023-10-16T14:31:19.414 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: fsmap 7 up:standby 2023-10-16T14:31:19.414 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:19 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:31:19.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:19.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:19.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:19.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: Deploying daemon mds.h on smithi186 2023-10-16T14:31:19.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:19.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6836/4191845467,v1:172.21.15.187:6837/4191845467] up:boot 2023-10-16T14:31:19.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: fsmap 7 up:standby 2023-10-16T14:31:19.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:19 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:31:19.448 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.448 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.448 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:19.448 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:19.449 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:19.449 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:19.449 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: Deploying daemon mds.h on smithi186 2023-10-16T14:31:19.449 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:19.449 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6836/4191845467,v1:172.21.15.187:6837/4191845467] up:boot 2023-10-16T14:31:19.450 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: fsmap 7 up:standby 2023-10-16T14:31:19.450 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:19 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:31:19.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 -- 172.21.15.178:0/3370984929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0100e90 msgr2=0x7f78b01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:19.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 --2- 172.21.15.178:0/3370984929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0100e90 0x7f78b01012b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f78a00098f0 tx=0x7f78a0009c00 comp rx=0 tx=0).stop 2023-10-16T14:31:19.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 -- 172.21.15.178:0/3370984929 shutdown_connections 2023-10-16T14:31:19.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 --2- 172.21.15.178:0/3370984929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102a30 0x7f78b0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 --2- 172.21.15.178:0/3370984929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0100e90 0x7f78b01012b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 --2- 172.21.15.178:0/3370984929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0102070 0x7f78b01024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 -- 172.21.15.178:0/3370984929 >> 172.21.15.178:0/3370984929 conn(0x7f78b00fc430 msgr2=0x7f78b00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:19.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 -- 172.21.15.178:0/3370984929 shutdown_connections 2023-10-16T14:31:19.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.549+0000 7f78b63ff700 1 -- 172.21.15.178:0/3370984929 wait complete. 2023-10-16T14:31:19.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.550+0000 7f78b63ff700 1 Processor -- start 2023-10-16T14:31:19.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.550+0000 7f78b63ff700 1 -- start start 2023-10-16T14:31:19.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 0x7f78b0198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:19.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:19.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0102a30 0x7f78b01a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:19.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a08e0 con 0x7f78b0102a30 2023-10-16T14:31:19.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a0a50 con 0x7f78b0102070 2023-10-16T14:31:19.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b63ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b01a0c10 con 0x7f78b0100e90 2023-10-16T14:31:19.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:19.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:54152/0 (socket says 172.21.15.178:54152) 2023-10-16T14:31:19.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 -- 172.21.15.178:0/2125748633 learned_addr learned my addr 172.21.15.178:0/2125748633 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:19.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78b499c700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0102a30 0x7f78b01a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:19.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78a7fff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 0x7f78b0198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:19.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 -- 172.21.15.178:0/2125748633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 msgr2=0x7f78b0198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 0x7f78b0198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 -- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0102a30 msgr2=0x7f78b01a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0102a30 0x7f78b01a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78affff700 1 -- 172.21.15.178:0/2125748633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0009520 con 0x7f78b0102070 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.551+0000 7f78a7fff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 0x7f78b0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:19.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78affff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f789800e970 tx=0x7f789800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:19.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898009790 con 0x7f78b0102070 2023-10-16T14:31:19.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b01a0ef0 con 0x7f78b0102070 2023-10-16T14:31:19.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f789800eec0 con 0x7f78b0102070 2023-10-16T14:31:19.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898003cf0 con 0x7f78b0102070 2023-10-16T14:31:19.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.552+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b01a1320 con 0x7f78b0102070 2023-10-16T14:31:19.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.553+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b0193540 con 0x7f78b0102070 2023-10-16T14:31:19.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.555+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f789800ccd0 con 0x7f78b0102070 2023-10-16T14:31:19.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.556+0000 7f78adffb700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 0x7f789c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:19.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.556+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7898014070 con 0x7f78b0102070 2023-10-16T14:31:19.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.556+0000 7f78a7fff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 0x7f789c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:19.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.557+0000 7f78a7fff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 0x7f789c0a39e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78b019a270 tx=0x7f78a0019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:19.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.560+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f78980969b0 con 0x7f78b0102070 2023-10-16T14:31:19.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.807+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b002d1c0 con 0x7f78b0102070 2023-10-16T14:31:19.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.808+0000 7f78adffb700 1 -- 172.21.15.178:0/2125748633 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6092 (secure 0 0 0) 0x7f7898099fd0 con 0x7f78b0102070 2023-10-16T14:31:19.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:19.812 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-10-16T14:31:19.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 msgr2=0x7f789c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:19.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 0x7f789c0a39e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f78b019a270 tx=0x7f78a0019040 comp rx=0 tx=0).stop 2023-10-16T14:31:19.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 msgr2=0x7f78b01992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:19.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f789800e970 tx=0x7f789800ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:19.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 shutdown_connections 2023-10-16T14:31:19.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f789c0a1520 0x7f789c0a39e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78b0102a30 0x7f78b01a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78b0100e90 0x7f78b0198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 --2- 172.21.15.178:0/2125748633 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f78b0102070 0x7f78b01992e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:19.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 >> 172.21.15.178:0/2125748633 conn(0x7f78b00fc430 msgr2=0x7f78b01079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:19.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 shutdown_connections 2023-10-16T14:31:19.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:19.811+0000 7f78b63ff700 1 -- 172.21.15.178:0/2125748633 wait complete. 2023-10-16T14:31:19.817 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 8 2023-10-16T14:31:19.994 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:19 smithi186 systemd[1]: Starting Ceph mds.h for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:20.254 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2125748633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:20.254 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:19 smithi186 podman[170627]: 2023-10-16T14:31:20.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2125748633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:20.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:20 smithi186 ceph-mds[170668]: -- 172.21.15.186:0/151342771 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b3fa5e91e0 con 0x55b3fa5bec00 2023-10-16T14:31:20.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: starting mds.h at 2023-10-16T14:31:20.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:20 smithi186 bash[170627]: c1a4aa4afb05604e87ca3fd6a2349abac1359ba2ca90e0d83c295aaa99c52077 2023-10-16T14:31:20.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:31:20 smithi186 systemd[1]: Started Ceph mds.h for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:20.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2125748633' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:21.631 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.631 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.631 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.631 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:21.632 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:21.632 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:21.632 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: Deploying daemon mds.i on smithi187 2023-10-16T14:31:21.632 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:21.632 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:boot 2023-10-16T14:31:21.633 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: fsmap 8 up:standby 2023-10-16T14:31:21.633 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:21 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:31:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: Deploying daemon mds.i on smithi187 2023-10-16T14:31:21.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:boot 2023-10-16T14:31:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: fsmap 8 up:standby 2023-10-16T14:31:21.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:21 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:31:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: Deploying daemon mds.i on smithi187 2023-10-16T14:31:21.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:21.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:boot 2023-10-16T14:31:21.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: fsmap 8 up:standby 2023-10-16T14:31:21.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:21 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:31:21.835 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:22.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 -- 172.21.15.178:0/4180542656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0100eb0 msgr2=0x7fe7c01012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:22.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/4180542656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0100eb0 0x7fe7c01012d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009990 tx=0x7fe7b0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 -- 172.21.15.178:0/4180542656 shutdown_connections 2023-10-16T14:31:22.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/4180542656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102a50 0x7fe7c0109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/4180542656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0100eb0 0x7fe7c01012d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/4180542656 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0102090 0x7fe7c0102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 -- 172.21.15.178:0/4180542656 >> 172.21.15.178:0/4180542656 conn(0x7fe7c00fc430 msgr2=0x7fe7c00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:22.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 -- 172.21.15.178:0/4180542656 shutdown_connections 2023-10-16T14:31:22.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.038+0000 7fe7c49ff700 1 -- 172.21.15.178:0/4180542656 wait complete. 2023-10-16T14:31:22.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.039+0000 7fe7c49ff700 1 Processor -- start 2023-10-16T14:31:22.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 -- start start 2023-10-16T14:31:22.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0100eb0 0x7fe7c0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:22.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 0x7fe7c00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0073f80 con 0x7fe7c0102a50 2023-10-16T14:31:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00740f0 con 0x7fe7c0100eb0 2023-10-16T14:31:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7c49ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c01a0bb0 con 0x7fe7c0102090 2023-10-16T14:31:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7bdd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 0x7fe7c00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7bed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7bed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44586/0 (socket says 172.21.15.178:44586) 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7be59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0100eb0 0x7fe7c0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7bed9d700 1 -- 172.21.15.178:0/483831114 learned_addr learned my addr 172.21.15.178:0/483831114 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.040+0000 7fe7bdd9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 0x7fe7c00732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50236/0 (socket says 172.21.15.178:50236) 2023-10-16T14:31:22.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 -- 172.21.15.178:0/483831114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 msgr2=0x7fe7c00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:22.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 0x7fe7c00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 -- 172.21.15.178:0/483831114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0100eb0 msgr2=0x7fe7c0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:22.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0100eb0 0x7fe7c0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 -- 172.21.15.178:0/483831114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00095f0 con 0x7fe7c0102a50 2023-10-16T14:31:22.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7bed9d700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800ef30 tx=0x7fe7b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8018070 con 0x7fe7c0102a50 2023-10-16T14:31:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b800f040 con 0x7fe7c0102a50 2023-10-16T14:31:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c01a0e30 con 0x7fe7c0102a50 2023-10-16T14:31:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800cd00 con 0x7fe7c0102a50 2023-10-16T14:31:22.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.041+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c01a12b0 con 0x7fe7c0102a50 2023-10-16T14:31:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.042+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c0107db0 con 0x7fe7c0102a50 2023-10-16T14:31:22.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.044+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe7b8003a00 con 0x7fe7c0102a50 2023-10-16T14:31:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.045+0000 7fe7af7fe700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 0x7fe7a80a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.046+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe7b80cc3f0 con 0x7fe7c0102a50 2023-10-16T14:31:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.046+0000 7fe7be59c700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 0x7fe7a80a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:22.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.046+0000 7fe7be59c700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 0x7fe7a80a3a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b2e0 tx=0x7fe7b0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:22.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.050+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe7b8095ee0 con 0x7fe7c0102a50 2023-10-16T14:31:22.168 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:21 smithi187 systemd[1]: Starting Ceph mds.i for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:22.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.296+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7c010c770 con 0x7fe7c0102a50 2023-10-16T14:31:22.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.296+0000 7fe7af7fe700 1 -- 172.21.15.178:0/483831114 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6886 (secure 0 0 0) 0x7fe7b8099500 con 0x7fe7c0102a50 2023-10-16T14:31:22.301 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:22.301 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2023-10-16T14:31:22.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 msgr2=0x7fe7a80a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:22.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 0x7fe7a80a3a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b2e0 tx=0x7fe7b0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 msgr2=0x7fe7c00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:22.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800ef30 tx=0x7fe7b800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 shutdown_connections 2023-10-16T14:31:22.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe7a80a15a0 0x7fe7a80a3a60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7c0102a50 0x7fe7c00737e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe7c0100eb0 0x7fe7c0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 --2- 172.21.15.178:0/483831114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7c0102090 0x7fe7c00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:22.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 >> 172.21.15.178:0/483831114 conn(0x7fe7c00fc430 msgr2=0x7fe7c00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:22.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 shutdown_connections 2023-10-16T14:31:22.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:22.299+0000 7fe7c49ff700 1 -- 172.21.15.178:0/483831114 wait complete. 2023-10-16T14:31:22.307 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 9 2023-10-16T14:31:22.425 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:22 smithi187 podman[173541]: 2023-10-16T14:31:22.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:22 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/483831114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:22.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:22 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/483831114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:22.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:22 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/483831114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:22.925 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:22 smithi187 ceph-mds[173581]: -- 172.21.15.187:0/854735169 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce4ad911e0 con 0x55ce4ad66c00 2023-10-16T14:31:22.925 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: starting mds.i at 2023-10-16T14:31:22.925 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:22 smithi187 bash[173541]: b9fad82e4ea74a50d4c7b1434236562daa9a8eb72022ad39df6fcfc8190a900d 2023-10-16T14:31:22.925 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:31:22 smithi187 systemd[1]: Started Ceph mds.i for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:23.743 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.744 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.744 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.744 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:23.744 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:23.744 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:23.745 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: Deploying daemon mds.g on smithi178 2023-10-16T14:31:23.745 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:23.745 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:boot 2023-10-16T14:31:23.745 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: fsmap 9 up:standby 2023-10-16T14:31:23.746 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:23 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:31:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:23.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: Deploying daemon mds.g on smithi178 2023-10-16T14:31:23.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:23.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:boot 2023-10-16T14:31:23.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: fsmap 9 up:standby 2023-10-16T14:31:23.828 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:23 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:31:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: Deploying daemon mds.g on smithi178 2023-10-16T14:31:23.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:23.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:boot 2023-10-16T14:31:23.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: fsmap 9 up:standby 2023-10-16T14:31:23.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:23 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:31:24.323 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:24.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 -- 172.21.15.178:0/2244840307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c0738b0 msgr2=0x7fdc1c073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:24.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 --2- 172.21.15.178:0/2244840307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c073d30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc10009990 tx=0x7fdc10009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 -- 172.21.15.178:0/2244840307 shutdown_connections 2023-10-16T14:31:24.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 --2- 172.21.15.178:0/2244840307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c108df0 0x7fdc1c10b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 --2- 172.21.15.178:0/2244840307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c074e90 0x7fdc1c0732e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 --2- 172.21.15.178:0/2244840307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c073d30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 -- 172.21.15.178:0/2244840307 >> 172.21.15.178:0/2244840307 conn(0x7fdc1c0fc450 msgr2=0x7fdc1c0fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:24.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 -- 172.21.15.178:0/2244840307 shutdown_connections 2023-10-16T14:31:24.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.621+0000 7fdc215ff700 1 -- 172.21.15.178:0/2244840307 wait complete. 2023-10-16T14:31:24.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 Processor -- start 2023-10-16T14:31:24.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 -- start start 2023-10-16T14:31:24.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:24.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 0x7fdc1c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:24.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:24.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1a08f0 con 0x7fdc1c108df0 2023-10-16T14:31:24.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1a0a30 con 0x7fdc1c074e90 2023-10-16T14:31:24.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc215ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1a0bc0 con 0x7fdc1c0738b0 2023-10-16T14:31:24.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc1b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:24.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.622+0000 7fdc1b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44594/0 (socket says 172.21.15.178:44594) 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 -- 172.21.15.178:0/387226394 learned_addr learned my addr 172.21.15.178:0/387226394 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1a7fc700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 0x7fdc1c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1affd700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 -- 172.21.15.178:0/387226394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c0738b0 msgr2=0x7fdc1c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 -- 172.21.15.178:0/387226394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 msgr2=0x7fdc1c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:24.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 0x7fdc1c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 -- 172.21.15.178:0/387226394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100095f0 con 0x7fdc1c108df0 2023-10-16T14:31:24.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1a7fc700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 0x7fdc1c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:24.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc1b7fe700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ef30 tx=0x7fdc1400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:24.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14018070 con 0x7fdc1c108df0 2023-10-16T14:31:24.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc1400f040 con 0x7fdc1c108df0 2023-10-16T14:31:24.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400cd00 con 0x7fdc1c108df0 2023-10-16T14:31:24.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.623+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c1a0e40 con 0x7fdc1c108df0 2023-10-16T14:31:24.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.624+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c1a1270 con 0x7fdc1c108df0 2023-10-16T14:31:24.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.625+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c107c00 con 0x7fdc1c108df0 2023-10-16T14:31:24.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.627+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdc14003a00 con 0x7fdc1c108df0 2023-10-16T14:31:24.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.628+0000 7fdc03fff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 0x7fdc040a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:24.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.628+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdc140cc380 con 0x7fdc1c108df0 2023-10-16T14:31:24.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.628+0000 7fdc1affd700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 0x7fdc040a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:24.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.629+0000 7fdc1affd700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 0x7fdc040a3a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009c60 tx=0x7fdc0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:24.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.632+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdc14095e70 con 0x7fdc1c108df0 2023-10-16T14:31:24.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.911+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc1c02d020 con 0x7fdc1c108df0 2023-10-16T14:31:24.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.912+0000 7fdc03fff700 1 -- 172.21.15.178:0/387226394 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7679 (secure 0 0 0) 0x7fdc14099490 con 0x7fdc1c108df0 2023-10-16T14:31:24.916 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:24.917 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-16T14:31:24.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 msgr2=0x7fdc040a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:24.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 0x7fdc040a3a60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009c60 tx=0x7fdc0c009260 comp rx=0 tx=0).stop 2023-10-16T14:31:24.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 msgr2=0x7fdc1c1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:24.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400ef30 tx=0x7fdc1400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 shutdown_connections 2023-10-16T14:31:24.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdc040a15a0 0x7fdc040a3a60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc1c108df0 0x7fdc1c1a0320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc1c0738b0 0x7fdc1c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 --2- 172.21.15.178:0/387226394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdc1c074e90 0x7fdc1c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:24.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 >> 172.21.15.178:0/387226394 conn(0x7fdc1c0fc450 msgr2=0x7fdc1c104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:24.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 shutdown_connections 2023-10-16T14:31:24.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:24.915+0000 7fdc215ff700 1 -- 172.21.15.178:0/387226394 wait complete. 2023-10-16T14:31:24.921 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 10 2023-10-16T14:31:25.053 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:24 smithi178 systemd[1]: Starting Ceph mds.g for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:25.574 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:25 smithi178 podman[149188]: 2023-10-16T14:31:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:25 smithi187 ceph-mon[135297]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:25 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/387226394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:25 smithi178 ceph-mon[98495]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:25 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/387226394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:26.053 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:25 smithi178 bash[149188]: 9ea88cecb6018d1f6fbdfb02cd3af26ee219d10d40eaf40744f7025b48a5a336 2023-10-16T14:31:26.054 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:25 smithi178 ceph-mds[149229]: -- 172.21.15.178:0/622637950 <== mon.0 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563c935a71e0 con 0x563c9357ec00 2023-10-16T14:31:26.054 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:25 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-g[149204]: starting mds.g at 2023-10-16T14:31:26.054 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:31:25 smithi178 systemd[1]: Started Ceph mds.g for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:25 smithi186 ceph-mon[134892]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:25 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/387226394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:26.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:26.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:26.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:26.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: Deploying daemon mds.a on smithi178 2023-10-16T14:31:26.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.178:6838/519971243,v1:172.21.15.178:6839/519971243] up:boot 2023-10-16T14:31:26.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: fsmap 10 up:standby 2023-10-16T14:31:26.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:26 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-16T14:31:26.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:26.959 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.960 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.960 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:26.960 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:26.960 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:26.961 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:26.961 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: Deploying daemon mds.a on smithi178 2023-10-16T14:31:26.961 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.178:6838/519971243,v1:172.21.15.178:6839/519971243] up:boot 2023-10-16T14:31:26.961 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: fsmap 10 up:standby 2023-10-16T14:31:26.961 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:26 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-16T14:31:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: Deploying daemon mds.a on smithi178 2023-10-16T14:31:27.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.178:6838/519971243,v1:172.21.15.178:6839/519971243] up:boot 2023-10-16T14:31:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: fsmap 10 up:standby 2023-10-16T14:31:27.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:26 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-10-16T14:31:27.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 -- 172.21.15.178:0/1506619187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102080 msgr2=0x7f4700102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:27.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 --2- 172.21.15.178:0/1506619187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102080 0x7f4700102500 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009990 tx=0x7f46f0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 -- 172.21.15.178:0/1506619187 shutdown_connections 2023-10-16T14:31:27.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 --2- 172.21.15.178:0/1506619187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102a40 0x7f4700109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 --2- 172.21.15.178:0/1506619187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 0x7f47001012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 --2- 172.21.15.178:0/1506619187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102080 0x7f4700102500 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 -- 172.21.15.178:0/1506619187 >> 172.21.15.178:0/1506619187 conn(0x7f47000fc420 msgr2=0x7f47000fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:27.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 -- 172.21.15.178:0/1506619187 shutdown_connections 2023-10-16T14:31:27.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.144+0000 7f47087ff700 1 -- 172.21.15.178:0/1506619187 wait complete. 2023-10-16T14:31:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.145+0000 7f47087ff700 1 Processor -- start 2023-10-16T14:31:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.145+0000 7f47087ff700 1 -- start start 2023-10-16T14:31:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 0x7f4700074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102080 0x7f47000732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4700073f80 con 0x7f4700102a40 2023-10-16T14:31:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47000740f0 con 0x7f4700100ea0 2023-10-16T14:31:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f47087ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001a0ba0 con 0x7f4700102080 2023-10-16T14:31:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4706d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4706d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:44618/0 (socket says 172.21.15.178:44618) 2023-10-16T14:31:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4706d9c700 1 -- 172.21.15.178:0/4094597354 learned_addr learned my addr 172.21.15.178:0/4094597354 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f470659b700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 0x7f4700074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4705d9a700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102080 0x7f47000732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4706d9c700 1 -- 172.21.15.178:0/4094597354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102080 msgr2=0x7f47000732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.146+0000 7f4706d9c700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102080 0x7f47000732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f4706d9c700 1 -- 172.21.15.178:0/4094597354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 msgr2=0x7f4700074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f4706d9c700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 0x7f4700074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f4706d9c700 1 -- 172.21.15.178:0/4094597354 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00095f0 con 0x7f4700102a40 2023-10-16T14:31:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f4706d9c700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46f800ef30 tx=0x7f46f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8018070 con 0x7f4700102a40 2023-10-16T14:31:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47001a0e20 con 0x7f4700102a40 2023-10-16T14:31:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f800f040 con 0x7f4700102a40 2023-10-16T14:31:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f800cdb0 con 0x7f4700102a40 2023-10-16T14:31:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.147+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47001a1250 con 0x7f4700102a40 2023-10-16T14:31:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.148+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700107dd0 con 0x7f4700102a40 2023-10-16T14:31:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.150+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f46f8003a00 con 0x7f4700102a40 2023-10-16T14:31:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.151+0000 7f46f77fe700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 0x7f46ec0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.151+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f46f80cc4d0 con 0x7f4700102a40 2023-10-16T14:31:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.152+0000 7f470659b700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 0x7f46ec0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.152+0000 7f470659b700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 0x7f46ec0a39e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4700101bf0 tx=0x7f46fc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:27.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.156+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f46f8095fc0 con 0x7f4700102a40 2023-10-16T14:31:27.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.409+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f470002d090 con 0x7f4700102a40 2023-10-16T14:31:27.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.410+0000 7f46f77fe700 1 -- 172.21.15.178:0/4094597354 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8471 (secure 0 0 0) 0x7f46f80995e0 con 0x7f4700102a40 2023-10-16T14:31:27.414 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:27.414 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-16T14:31:27.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 msgr2=0x7f46ec0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:27.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 0x7f46ec0a39e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4700101bf0 tx=0x7f46fc009260 comp rx=0 tx=0).stop 2023-10-16T14:31:27.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 msgr2=0x7f47000737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:27.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f46f800ef30 tx=0x7f46f800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 shutdown_connections 2023-10-16T14:31:27.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f46ec0a1520 0x7f46ec0a39e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4700102a40 0x7f47000737e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.412+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4700100ea0 0x7f4700074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.413+0000 7f47087ff700 1 --2- 172.21.15.178:0/4094597354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4700102080 0x7f47000732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:27.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.413+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 >> 172.21.15.178:0/4094597354 conn(0x7f47000fc420 msgr2=0x7f47000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:27.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.413+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 shutdown_connections 2023-10-16T14:31:27.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:27.413+0000 7f47087ff700 1 -- 172.21.15.178:0/4094597354 wait complete. 2023-10-16T14:31:27.419 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 11 2023-10-16T14:31:27.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:27 smithi187 ceph-mon[135297]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:27.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:27 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4094597354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:27.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:27 smithi178 ceph-mon[98495]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:27.984 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:27 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4094597354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:27 smithi186 ceph-mon[134892]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:27 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4094597354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:28.303 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:27 smithi178 systemd[1]: Starting Ceph mds.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:28.748 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:28 smithi178 podman[149579]: 2023-10-16T14:31:29.053 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:28 smithi178 bash[149579]: b8fc1d471b3afb1c077dd6ec0d17ea2a69457a5741056115373241f57b23cf01 2023-10-16T14:31:29.053 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:28 smithi178 ceph-mds[149619]: -- 172.21.15.178:0/132265562 <== mon.0 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3bc17f1e0 con 0x55c3bc156c00 2023-10-16T14:31:29.053 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:28 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-a[149595]: starting mds.a at 2023-10-16T14:31:29.054 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:31:28 smithi178 systemd[1]: Started Ceph mds.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:29.438 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:29.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 -- 172.21.15.178:0/1035289162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 msgr2=0x7f17c8109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:29.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/1035289162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8109370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b0a0 tx=0x7f17c000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 -- 172.21.15.178:0/1035289162 shutdown_connections 2023-10-16T14:31:29.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/1035289162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8109370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/1035289162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8100eb0 0x7f17c81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/1035289162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8102090 0x7f17c8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 -- 172.21.15.178:0/1035289162 >> 172.21.15.178:0/1035289162 conn(0x7f17c80fc430 msgr2=0x7f17c80fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:29.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 -- 172.21.15.178:0/1035289162 shutdown_connections 2023-10-16T14:31:29.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.641+0000 7f17cf7ff700 1 -- 172.21.15.178:0/1035289162 wait complete. 2023-10-16T14:31:29.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 Processor -- start 2023-10-16T14:31:29.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 -- start start 2023-10-16T14:31:29.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 0x7f17c8106cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:29.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:29.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8103840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:29.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8103fe0 con 0x7f17c8102090 2023-10-16T14:31:29.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8104150 con 0x7f17c8102a50 2023-10-16T14:31:29.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cf7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8104310 con 0x7f17c8100eb0 2023-10-16T14:31:29.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17ccd9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:29.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17ccd9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51156/0 (socket says 172.21.15.178:51156) 2023-10-16T14:31:29.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17cdd9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8103840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:29.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.642+0000 7f17ccd9a700 1 -- 172.21.15.178:0/562603626 learned_addr learned my addr 172.21.15.178:0/562603626 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:29.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cd59b700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 0x7f17c8106cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:29.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 -- 172.21.15.178:0/562603626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 msgr2=0x7f17c8106cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:29.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 0x7f17c8106cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 -- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 msgr2=0x7f17c81071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:29.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 -- 172.21.15.178:0/562603626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0009d00 con 0x7f17c8102a50 2023-10-16T14:31:29.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cd59b700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 0x7f17c8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:29.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17ccd9a700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:29.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cdd9c700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8103840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c0007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000e040 con 0x7f17c8102a50 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c81a5340 con 0x7f17c8102a50 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c0004c50 con 0x7f17c8102a50 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c00052a0 con 0x7f17c8102a50 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.643+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c81a57c0 con 0x7f17c8102a50 2023-10-16T14:31:29.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.645+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c810c770 con 0x7f17c8102a50 2023-10-16T14:31:29.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.646+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f17c0005610 con 0x7f17c8102a50 2023-10-16T14:31:29.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.647+0000 7f17be7fc700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 0x7f17b40a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:29.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.648+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17c00cdff0 con 0x7f17c8102a50 2023-10-16T14:31:29.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.648+0000 7f17cd59b700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 0x7f17b40a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:29.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.648+0000 7f17cd59b700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 0x7f17b40a39e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17c8101c00 tx=0x7f17c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:29.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.651+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f17c0097ae0 con 0x7f17c8102a50 2023-10-16T14:31:29.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.899+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17c81a59b0 con 0x7f17c8102a50 2023-10-16T14:31:29.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.901+0000 7f17be7fc700 1 -- 172.21.15.178:0/562603626 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9266 (secure 0 0 0) 0x7f17c009b100 con 0x7f17c8102a50 2023-10-16T14:31:29.904 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:29.905 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-10-16T14:31:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 msgr2=0x7f17b40a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 0x7f17b40a39e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f17c8101c00 tx=0x7f17c4009260 comp rx=0 tx=0).stop 2023-10-16T14:31:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 msgr2=0x7f17c8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:29.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8103840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f17c0009fd0 tx=0x7f17c0007930 comp rx=0 tx=0).stop 2023-10-16T14:31:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 shutdown_connections 2023-10-16T14:31:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f17b40a1520 0x7f17b40a39e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f17c8102a50 0x7f17c8103840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17c8100eb0 0x7f17c8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 --2- 172.21.15.178:0/562603626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17c8102090 0x7f17c81071f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 >> 172.21.15.178:0/562603626 conn(0x7f17c80fc430 msgr2=0x7f17c80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 shutdown_connections 2023-10-16T14:31:29.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:29.903+0000 7f17cf7ff700 1 -- 172.21.15.178:0/562603626 wait complete. 2023-10-16T14:31:29.910 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 12 2023-10-16T14:31:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:30.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:30.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:30.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: Deploying daemon mds.b on smithi186 2023-10-16T14:31:30.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.178:6840/2221064860,v1:172.21.15.178:6841/2221064860] up:boot 2023-10-16T14:31:30.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: fsmap 11 up:standby 2023-10-16T14:31:30.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:29 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-16T14:31:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:30.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:30.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:30.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:30.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: Deploying daemon mds.b on smithi186 2023-10-16T14:31:30.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.178:6840/2221064860,v1:172.21.15.178:6841/2221064860] up:boot 2023-10-16T14:31:30.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: fsmap 11 up:standby 2023-10-16T14:31:30.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:29 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-16T14:31:30.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-10-16T14:31:30.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-10-16T14:31:30.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:30.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: Deploying daemon mds.b on smithi186 2023-10-16T14:31:30.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.178:6840/2221064860,v1:172.21.15.178:6841/2221064860] up:boot 2023-10-16T14:31:30.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: fsmap 11 up:standby 2023-10-16T14:31:30.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:29 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-10-16T14:31:30.576 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 systemd[1]: Starting Ceph mds.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:31:30.905 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:30 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/562603626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:30.906 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 podman[170978]: 2023-10-16T14:31:30.906 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 ceph-mds[171018]: -- 172.21.15.186:0/2329375195 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bf945ff1e0 con 0x55bf945d5000 2023-10-16T14:31:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:30 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/562603626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:31.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:30 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/562603626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:31.326 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: starting mds.b at 2023-10-16T14:31:31.326 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 bash[170978]: a340a1d15ad3d0d103490fd608dec1f74684820f8363944946db2817388b7c33 2023-10-16T14:31:31.326 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:30 smithi186 systemd[1]: Started Ceph mds.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:31:31.927 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:32.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 -- 172.21.15.178:0/483109873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344073910 msgr2=0x7f9344073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 --2- 172.21.15.178:0/483109873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344073910 0x7f9344073d90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9338009990 tx=0x7f9338009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 -- 172.21.15.178:0/483109873 shutdown_connections 2023-10-16T14:31:32.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 --2- 172.21.15.178:0/483109873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934410d2b0 0x7f934410f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 --2- 172.21.15.178:0/483109873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9344074ef0 0x7f9344073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 --2- 172.21.15.178:0/483109873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344073910 0x7f9344073d90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 -- 172.21.15.178:0/483109873 >> 172.21.15.178:0/483109873 conn(0x7f93440fc210 msgr2=0x7f93440fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:32.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.130+0000 7f934359e700 1 -- 172.21.15.178:0/483109873 shutdown_connections 2023-10-16T14:31:32.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.131+0000 7f934359e700 1 -- 172.21.15.178:0/483109873 wait complete. 2023-10-16T14:31:32.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 Processor -- start 2023-10-16T14:31:32.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 -- start start 2023-10-16T14:31:32.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934410d2b0 0x7f93441a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441a4b30 con 0x7f9344074ef0 2023-10-16T14:31:32.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441a4c70 con 0x7f934410d2b0 2023-10-16T14:31:32.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.132+0000 7f934359e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93441a4e00 con 0x7f9344073910 2023-10-16T14:31:32.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f9341d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37814/0 (socket says 172.21.15.178:37814) 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f9341d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51168/0 (socket says 172.21.15.178:51168) 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 -- 172.21.15.178:0/202438935 learned_addr learned my addr 172.21.15.178:0/202438935 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f9342d9d700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934410d2b0 0x7f93441a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 -- 172.21.15.178:0/202438935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934410d2b0 msgr2=0x7f93441a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934410d2b0 0x7f93441a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 -- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 msgr2=0x7f934419d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 -- 172.21.15.178:0/202438935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380095f0 con 0x7f9344073910 2023-10-16T14:31:32.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f9341d9b700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:32.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.133+0000 7f934259c700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f933400e9a0 tx=0x7f933400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:32.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.134+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334009790 con 0x7f9344073910 2023-10-16T14:31:32.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.134+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9334010430 con 0x7f9344073910 2023-10-16T14:31:32.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.134+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93441a50e0 con 0x7f9344073910 2023-10-16T14:31:32.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.134+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334018680 con 0x7f9344073910 2023-10-16T14:31:32.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.134+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93441a5640 con 0x7f9344073910 2023-10-16T14:31:32.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.135+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9344102e00 con 0x7f9344073910 2023-10-16T14:31:32.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.137+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f93340105a0 con 0x7f9344073910 2023-10-16T14:31:32.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.138+0000 7f93337fe700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 0x7f932c0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.138+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9334014070 con 0x7f9344073910 2023-10-16T14:31:32.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.138+0000 7f9341d9b700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 0x7f932c0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.139+0000 7f9341d9b700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 0x7f932c0a3a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f933800b2e0 tx=0x7f9338005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:32.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.140+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9334096500 con 0x7f9344073910 2023-10-16T14:31:32.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:32.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:32.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:boot 2023-10-16T14:31:32.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: fsmap 12 up:standby 2023-10-16T14:31:32.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:31 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:31:32.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:32.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:32.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:boot 2023-10-16T14:31:32.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: fsmap 12 up:standby 2023-10-16T14:31:32.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:31 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:31:32.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.401+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934402d220 con 0x7f9344073910 2023-10-16T14:31:32.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.402+0000 7f93337fe700 1 -- 172.21.15.178:0/202438935 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10061 (secure 0 0 0) 0x7f9334099b20 con 0x7f9344073910 2023-10-16T14:31:32.406 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:32.406 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24593,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-10-16T14:31:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 msgr2=0x7f932c0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 0x7f932c0a3a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f933800b2e0 tx=0x7f9338005960 comp rx=0 tx=0).stop 2023-10-16T14:31:32.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 msgr2=0x7f934419cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f933400e9a0 tx=0x7f933400ecb0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 shutdown_connections 2023-10-16T14:31:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f932c0a1580 0x7f932c0a3a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f934410d2b0 0x7f93441a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9344073910 0x7f934419cfb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 --2- 172.21.15.178:0/202438935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9344074ef0 0x7f934419d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 >> 172.21.15.178:0/202438935 conn(0x7f93440fc210 msgr2=0x7f93440fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 shutdown_connections 2023-10-16T14:31:32.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.404+0000 7f934359e700 1 -- 172.21.15.178:0/202438935 wait complete. 2023-10-16T14:31:32.411 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 13 2023-10-16T14:31:32.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:32.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:32.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:32.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:boot 2023-10-16T14:31:32.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: fsmap 12 up:standby 2023-10-16T14:31:32.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:31 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:31:32.428 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 -- 172.21.15.178:0/2435783325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102090 msgr2=0x7f40b4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 --2- 172.21.15.178:0/2435783325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102090 0x7f40b4102510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f409c009990 tx=0x7f409c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 -- 172.21.15.178:0/2435783325 shutdown_connections 2023-10-16T14:31:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 --2- 172.21.15.178:0/2435783325 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102a50 0x7f40b4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 --2- 172.21.15.178:0/2435783325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 --2- 172.21.15.178:0/2435783325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102090 0x7f40b4102510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 -- 172.21.15.178:0/2435783325 >> 172.21.15.178:0/2435783325 conn(0x7f40b40fc430 msgr2=0x7f40b40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 -- 172.21.15.178:0/2435783325 shutdown_connections 2023-10-16T14:31:32.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.628+0000 7f40b8fff700 1 -- 172.21.15.178:0/2435783325 wait complete. 2023-10-16T14:31:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 Processor -- start 2023-10-16T14:31:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 -- start start 2023-10-16T14:31:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102090 0x7f40b4197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102a50 0x7f40b419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b419e6d0 con 0x7f40b4102a50 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b419e810 con 0x7f40b4102090 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.630+0000 7f40b8fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b419e9a0 con 0x7f40b4100eb0 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37840/0 (socket says 172.21.15.178:37840) 2023-10-16T14:31:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 -- 172.21.15.178:0/1085423754 learned_addr learned my addr 172.21.15.178:0/1085423754 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40b259c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102090 0x7f40b4197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40b2d9d700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102a50 0x7f40b419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 -- 172.21.15.178:0/1085423754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102090 msgr2=0x7f40b4197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102090 0x7f40b4197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 -- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102a50 msgr2=0x7f40b419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102a50 0x7f40b419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 -- 172.21.15.178:0/1085423754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4009520 con 0x7f40b4100eb0 2023-10-16T14:31:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40aa59c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40a400ea50 tx=0x7f40a400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:32.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a400cbd0 con 0x7f40b4100eb0 2023-10-16T14:31:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c0095f0 con 0x7f40b4100eb0 2023-10-16T14:31:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40a400cd30 con 0x7f40b4100eb0 2023-10-16T14:31:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.631+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4018680 con 0x7f40b4100eb0 2023-10-16T14:31:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.632+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b419f050 con 0x7f40b4100eb0 2023-10-16T14:31:32.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.632+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b4107e10 con 0x7f40b4100eb0 2023-10-16T14:31:32.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.634+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f40a4010430 con 0x7f40b4100eb0 2023-10-16T14:31:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.636+0000 7f40abfff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 0x7f40a00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.636+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f40a4014070 con 0x7f40b4100eb0 2023-10-16T14:31:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.636+0000 7f40b259c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 0x7f40a00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.637+0000 7f40b259c700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 0x7f40a00a39e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b2e0 tx=0x7f409c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.638+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f40a4096df0 con 0x7f40b4100eb0 2023-10-16T14:31:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.851+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b410c770 con 0x7f40b4100eb0 2023-10-16T14:31:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.852+0000 7f40abfff700 1 -- 172.21.15.178:0/1085423754 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18482 (secure 0 0 0) 0x7f40a409a410 con 0x7f40b4100eb0 2023-10-16T14:31:32.857 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:32.857 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":79,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:30:49.840789+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 msgr2=0x7f40a00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 0x7f40a00a39e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f409c00b2e0 tx=0x7f409c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 msgr2=0x7f40b4196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f40a400ea50 tx=0x7f40a400ed60 comp rx=0 tx=0).stop 2023-10-16T14:31:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 shutdown_connections 2023-10-16T14:31:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a00a1520 0x7f40a00a39e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40b4102a50 0x7f40b419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40b4100eb0 0x7f40b4196b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 --2- 172.21.15.178:0/1085423754 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40b4102090 0x7f40b4197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 >> 172.21.15.178:0/1085423754 conn(0x7f40b40fc430 msgr2=0x7f40b40fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 shutdown_connections 2023-10-16T14:31:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:32.855+0000 7f40b8fff700 1 -- 172.21.15.178:0/1085423754 wait complete. 2023-10-16T14:31:32.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T14:31:32.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:31:33.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 -- 172.21.15.178:0/3235449845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b81020b0 msgr2=0x7f09b8104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 --2- 172.21.15.178:0/3235449845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b81020b0 0x7f09b8104540 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f09ac009990 tx=0x7f09ac009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 -- 172.21.15.178:0/3235449845 shutdown_connections 2023-10-16T14:31:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 --2- 172.21.15.178:0/3235449845 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b8104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 --2- 172.21.15.178:0/3235449845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b80ff740 0x7f09b8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 --2- 172.21.15.178:0/3235449845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b81020b0 0x7f09b8104540 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 -- 172.21.15.178:0/3235449845 >> 172.21.15.178:0/3235449845 conn(0x7f09b80fb340 msgr2=0x7f09b80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 -- 172.21.15.178:0/3235449845 shutdown_connections 2023-10-16T14:31:33.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.085+0000 7f09be1ff700 1 -- 172.21.15.178:0/3235449845 wait complete. 2023-10-16T14:31:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.087+0000 7f09be1ff700 1 Processor -- start 2023-10-16T14:31:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.087+0000 7f09be1ff700 1 -- start start 2023-10-16T14:31:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.087+0000 7f09be1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b80ff740 0x7f09b819d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.087+0000 7f09be1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b81020b0 0x7f09b819d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.087+0000 7f09be1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09be1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a4d50 con 0x7f09b81020b0 2023-10-16T14:31:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09be1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a4e90 con 0x7f09b8104a80 2023-10-16T14:31:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09be1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b81a5020 con 0x7f09b80ff740 2023-10-16T14:31:33.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b81020b0 0x7f09b819d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50366/0 (socket says 172.21.15.178:50366) 2023-10-16T14:31:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 -- 172.21.15.178:0/2813803475 learned_addr learned my addr 172.21.15.178:0/2813803475 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 -- 172.21.15.178:0/2813803475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b80ff740 msgr2=0x7f09b819d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b80ff740 0x7f09b819d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 -- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b81020b0 msgr2=0x7f09b819d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b81020b0 0x7f09b819d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 -- 172.21.15.178:0/2813803475 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0095f0 con 0x7f09b8104a80 2023-10-16T14:31:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.088+0000 7f09b7fff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ef30 tx=0x7f09b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.089+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0018070 con 0x7f09b8104a80 2023-10-16T14:31:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.089+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b000f040 con 0x7f09b8104a80 2023-10-16T14:31:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.089+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b000cdb0 con 0x7f09b8104a80 2023-10-16T14:31:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.089+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b81a52a0 con 0x7f09b8104a80 2023-10-16T14:31:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.089+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b81a5760 con 0x7f09b8104a80 2023-10-16T14:31:33.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.090+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b80fcec0 con 0x7f09b8104a80 2023-10-16T14:31:33.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.093+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f09b0003a00 con 0x7f09b8104a80 2023-10-16T14:31:33.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.094+0000 7f09b4ff9700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 0x7f09a00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.094+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09b00cce50 con 0x7f09b8104a80 2023-10-16T14:31:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.094+0000 7f09b77fe700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 0x7f09a00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.095+0000 7f09b77fe700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 0x7f09a00a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f09a8005950 tx=0x7f09a80058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:33.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.096+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f09b0096940 con 0x7f09b8104a80 2023-10-16T14:31:33.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:33 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/202438935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:33 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1085423754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.308+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09b8112ea0 con 0x7f09b8104a80 2023-10-16T14:31:33.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.309+0000 7f09b4ff9700 1 -- 172.21.15.178:0/2813803475 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09b0099f60 con 0x7f09b8104a80 2023-10-16T14:31:33.313 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:31:33.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 msgr2=0x7f09a00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 0x7f09a00a39c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f09a8005950 tx=0x7f09a80058e0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 msgr2=0x7f09b81a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f09b000ef30 tx=0x7f09b000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 shutdown_connections 2023-10-16T14:31:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09a00a1500 0x7f09a00a39c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09b8104a80 0x7f09b81a4780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09b80ff740 0x7f09b819d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 --2- 172.21.15.178:0/2813803475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09b81020b0 0x7f09b819d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 >> 172.21.15.178:0/2813803475 conn(0x7f09b80fb340 msgr2=0x7f09b81003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 shutdown_connections 2023-10-16T14:31:33.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.312+0000 7f09be1ff700 1 -- 172.21.15.178:0/2813803475 wait complete. 2023-10-16T14:31:33.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:33 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/202438935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:33 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1085423754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.337 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T14:31:33.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:33 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/202438935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:33 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1085423754' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:33.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.542+0000 7f65e0fff700 1 -- 172.21.15.178:0/3175818158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc104ce0 msgr2=0x7f65dc10b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.542+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3175818158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc104ce0 0x7f65dc10b590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b0a0 tx=0x7f65d400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 -- 172.21.15.178:0/3175818158 shutdown_connections 2023-10-16T14:31:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3175818158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc104ce0 0x7f65dc10b590 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3175818158 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc103150 0x7f65dc103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3175818158 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104340 0x7f65dc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 -- 172.21.15.178:0/3175818158 >> 172.21.15.178:0/3175818158 conn(0x7f65dc0fe6c0 msgr2=0x7f65dc100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:33.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 -- 172.21.15.178:0/3175818158 shutdown_connections 2023-10-16T14:31:33.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.543+0000 7f65e0fff700 1 -- 172.21.15.178:0/3175818158 wait complete. 2023-10-16T14:31:33.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 Processor -- start 2023-10-16T14:31:33.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 -- start start 2023-10-16T14:31:33.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc104340 0x7f65dc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 0x7f65dc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a09a0 con 0x7f65dc104ce0 2023-10-16T14:31:33.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a0ae0 con 0x7f65dc104340 2023-10-16T14:31:33.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65e0fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a0c70 con 0x7f65dc103150 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65da59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65d9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc104340 0x7f65dc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65da59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37872/0 (socket says 172.21.15.178:37872) 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65da59c700 1 -- 172.21.15.178:0/3933531375 learned_addr learned my addr 172.21.15.178:0/3933531375 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.544+0000 7f65dad9d700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 0x7f65dc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 -- 172.21.15.178:0/3933531375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc104340 msgr2=0x7f65dc199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc104340 0x7f65dc199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 -- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 msgr2=0x7f65dc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 0x7f65dc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 -- 172.21.15.178:0/3933531375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009d00 con 0x7f65dc103150 2023-10-16T14:31:33.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65dad9d700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 0x7f65dc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65da59c700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e970 tx=0x7f65cc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc009790 con 0x7f65dc103150 2023-10-16T14:31:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65cc00eec0 con 0x7f65dc103150 2023-10-16T14:31:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc003cf0 con 0x7f65dc103150 2023-10-16T14:31:33.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc1a0f50 con 0x7f65dc103150 2023-10-16T14:31:33.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.545+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1a1370 con 0x7f65dc103150 2023-10-16T14:31:33.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.547+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc10ea20 con 0x7f65dc103150 2023-10-16T14:31:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.549+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f65cc00ccd0 con 0x7f65dc103150 2023-10-16T14:31:33.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.550+0000 7f65cb7fe700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 0x7f65c40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.550+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f65cc014070 con 0x7f65dc103150 2023-10-16T14:31:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.550+0000 7f65d9d9b700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 0x7f65c40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:33.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.551+0000 7f65d9d9b700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 0x7f65c40a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009c60 tx=0x7f65d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:33.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.552+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65cc096940 con 0x7f65dc103150 2023-10-16T14:31:33.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.775+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f65dc1a1750 con 0x7f65dc103150 2023-10-16T14:31:33.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.776+0000 7f65cb7fe700 1 -- 172.21.15.178:0/3933531375 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f65cc099f60 con 0x7f65dc103150 2023-10-16T14:31:33.780 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:33.780 INFO:teuthology.orchestra.run.smithi178.stdout:[ 2023-10-16T14:31:33.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:31:33.781 INFO:teuthology.orchestra.run.smithi178.stdout: "poolnum": 1, 2023-10-16T14:31:33.781 INFO:teuthology.orchestra.run.smithi178.stdout: "poolname": ".mgr" 2023-10-16T14:31:33.781 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:31:33.781 INFO:teuthology.orchestra.run.smithi178.stdout:] 2023-10-16T14:31:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 msgr2=0x7f65c40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 0x7f65c40a3a60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009c60 tx=0x7f65d0009260 comp rx=0 tx=0).stop 2023-10-16T14:31:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 msgr2=0x7f65dc198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:33.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e970 tx=0x7f65cc00ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 shutdown_connections 2023-10-16T14:31:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65c40a15a0 0x7f65c40a3a60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65dc104ce0 0x7f65dc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65dc103150 0x7f65dc198e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 --2- 172.21.15.178:0/3933531375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f65dc104340 0x7f65dc199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 >> 172.21.15.178:0/3933531375 conn(0x7f65dc0fe6c0 msgr2=0x7f65dc100770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 shutdown_connections 2023-10-16T14:31:33.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:33.779+0000 7f65e0fff700 1 -- 172.21.15.178:0/3933531375 wait complete. 2023-10-16T14:31:33.805 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T14:31:33.806 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T14:31:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 -- 172.21.15.178:0/1112925017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79041042c0 msgr2=0x7f79041066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:34.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 --2- 172.21.15.178:0/1112925017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79041042c0 0x7f79041066b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009990 tx=0x7f78f0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 -- 172.21.15.178:0/1112925017 shutdown_connections 2023-10-16T14:31:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 --2- 172.21.15.178:0/1112925017 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7904106bf0 0x7f7904109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 --2- 172.21.15.178:0/1112925017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 0x7f7904103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 --2- 172.21.15.178:0/1112925017 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79041042c0 0x7f79041066b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 -- 172.21.15.178:0/1112925017 >> 172.21.15.178:0/1112925017 conn(0x7f79040fb320 msgr2=0x7f79040fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 -- 172.21.15.178:0/1112925017 shutdown_connections 2023-10-16T14:31:34.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.032+0000 7f790adff700 1 -- 172.21.15.178:0/1112925017 wait complete. 2023-10-16T14:31:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 Processor -- start 2023-10-16T14:31:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 -- start start 2023-10-16T14:31:34.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 0x7f7904196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 0x7f7904197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419e770 con 0x7f7904101990 2023-10-16T14:31:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419e8b0 con 0x7f79041042c0 2023-10-16T14:31:34.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.034+0000 7f790adff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790419ea40 con 0x7f7904106bf0 2023-10-16T14:31:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7908b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 0x7f7904196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37892/0 (socket says 172.21.15.178:37892) 2023-10-16T14:31:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 -- 172.21.15.178:0/1632002912 learned_addr learned my addr 172.21.15.178:0/1632002912 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:34.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7903fff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 0x7f7904197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 -- 172.21.15.178:0/1632002912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 msgr2=0x7f7904197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 0x7f7904197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 -- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 msgr2=0x7f7904196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 0x7f7904196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:34.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 -- 172.21.15.178:0/1632002912 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00095f0 con 0x7f7904106bf0 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7903fff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 0x7f7904197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790939c700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00ef30 tx=0x7f78fc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc018070 con 0x7f7904106bf0 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78fc00f040 con 0x7f7904106bf0 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790419ecc0 con 0x7f7904106bf0 2023-10-16T14:31:34.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.035+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00cd00 con 0x7f7904106bf0 2023-10-16T14:31:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790419f180 con 0x7f7904106bf0 2023-10-16T14:31:34.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79040fcea0 con 0x7f7904106bf0 2023-10-16T14:31:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.038+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f78fc003a00 con 0x7f7904106bf0 2023-10-16T14:31:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.040+0000 7f7901ffb700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 0x7f78f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.040+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f78fc0cc3c0 con 0x7f7904106bf0 2023-10-16T14:31:34.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.040+0000 7f7908b9b700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 0x7f78f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:34.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.040+0000 7f7908b9b700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 0x7f78f40a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009c60 tx=0x7f78f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:34.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.044+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f78fc095f30 con 0x7f7904106bf0 2023-10-16T14:31:34.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:34 smithi178 ceph-mon[98495]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:34.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:34 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2813803475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:34 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3933531375' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-16T14:31:34.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:34.256+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f790410c8f0 con 0x7f7904106bf0 2023-10-16T14:31:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:34 smithi186 ceph-mon[134892]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:34 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2813803475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:34 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3933531375' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-16T14:31:34.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:34 smithi187 ceph-mon[135297]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:34.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:34 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2813803475' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:34.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:34 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3933531375' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-10-16T14:31:35.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.030+0000 7f7901ffb700 1 -- 172.21.15.178:0/1632002912 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f78fc099550 con 0x7f7904106bf0 2023-10-16T14:31:35.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 msgr2=0x7f78f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:35.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 0x7f78f40a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009c60 tx=0x7f78f8009260 comp rx=0 tx=0).stop 2023-10-16T14:31:35.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 msgr2=0x7f790419e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:35.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00ef30 tx=0x7f78fc00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 shutdown_connections 2023-10-16T14:31:35.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f78f40a1480 0x7f78f40a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7904106bf0 0x7f790419e1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7904101990 0x7f7904196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 --2- 172.21.15.178:0/1632002912 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f79041042c0 0x7f7904197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 >> 172.21.15.178:0/1632002912 conn(0x7f79040fb320 msgr2=0x7f7904102600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:35.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 shutdown_connections 2023-10-16T14:31:35.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.036+0000 7f790adff700 1 -- 172.21.15.178:0/1632002912 wait complete. 2023-10-16T14:31:35.043 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_metadata' created 2023-10-16T14:31:35.061 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-10-16T14:31:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 -- 172.21.15.178:0/2179424833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 msgr2=0x7f8010109050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 --2- 172.21.15.178:0/2179424833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f8010109050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f800800b0a0 tx=0x7f800800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 -- 172.21.15.178:0/2179424833 shutdown_connections 2023-10-16T14:31:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 --2- 172.21.15.178:0/2179424833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f8010109050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 --2- 172.21.15.178:0/2179424833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 --2- 172.21.15.178:0/2179424833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 0x7f80101066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 -- 172.21.15.178:0/2179424833 >> 172.21.15.178:0/2179424833 conn(0x7f80100fb370 msgr2=0x7f80100fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:35.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 -- 172.21.15.178:0/2179424833 shutdown_connections 2023-10-16T14:31:35.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.268+0000 7f80173ff700 1 -- 172.21.15.178:0/2179424833 wait complete. 2023-10-16T14:31:35.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 Processor -- start 2023-10-16T14:31:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 -- start start 2023-10-16T14:31:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 0x7f8010199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:35.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80101a09c0 con 0x7f8010101980 2023-10-16T14:31:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80101a0b00 con 0x7f80101042b0 2023-10-16T14:31:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.269+0000 7f80173ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80101a0c90 con 0x7f8010106be0 2023-10-16T14:31:35.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37916/0 (socket says 172.21.15.178:37916) 2023-10-16T14:31:35.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51268/0 (socket says 172.21.15.178:51268) 2023-10-16T14:31:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 -- 172.21.15.178:0/1299634671 learned_addr learned my addr 172.21.15.178:0/1299634671 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801499a700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 0x7f8010199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 -- 172.21.15.178:0/1299634671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 msgr2=0x7f8010199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 0x7f8010199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 -- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 msgr2=0x7f8010198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:35.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:35.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 -- 172.21.15.178:0/1299634671 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008009d00 con 0x7f8010106be0 2023-10-16T14:31:35.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f801599c700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008009fd0 tx=0x7f800800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:35.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.270+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800e040 con 0x7f8010106be0 2023-10-16T14:31:35.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.271+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80080094a0 con 0x7f8010106be0 2023-10-16T14:31:35.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.271+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80101a0f10 con 0x7f8010106be0 2023-10-16T14:31:35.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.271+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800801b6c0 con 0x7f8010106be0 2023-10-16T14:31:35.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.271+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80101a1330 con 0x7f8010106be0 2023-10-16T14:31:35.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.272+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8010108760 con 0x7f8010106be0 2023-10-16T14:31:35.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.274+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f800801d470 con 0x7f8010106be0 2023-10-16T14:31:35.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.275+0000 7f80067fc700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 0x7f7ffc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:35.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.275+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f80080cd960 con 0x7f8010106be0 2023-10-16T14:31:35.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.275+0000 7f801519b700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 0x7f7ffc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:35.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.276+0000 7f801519b700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 0x7f7ffc0a39c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f800c009c60 tx=0x7f800c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:35.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.279+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f80080972e0 con 0x7f8010106be0 2023-10-16T14:31:35.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1632002912' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:35 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1632002912' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:35 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1632002912' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-10-16T14:31:35.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:35.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:35.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:35 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:35.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:35.497+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f801002d0f0 con 0x7f8010106be0 2023-10-16T14:31:36.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.032+0000 7f80067fc700 1 -- 172.21.15.178:0/1299634671 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7f800809a900 con 0x7f8010106be0 2023-10-16T14:31:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.041+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 msgr2=0x7f7ffc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.041+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 0x7f7ffc0a39c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f800c009c60 tx=0x7f800c009260 comp rx=0 tx=0).stop 2023-10-16T14:31:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.041+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 msgr2=0x7f80101a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.041+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8008009fd0 tx=0x7f800800bce0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 shutdown_connections 2023-10-16T14:31:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7ffc0a1500 0x7f7ffc0a39c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8010106be0 0x7f80101a03f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8010101980 0x7f8010198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 --2- 172.21.15.178:0/1299634671 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f80101042b0 0x7f8010199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.042+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 >> 172.21.15.178:0/1299634671 conn(0x7f80100fb370 msgr2=0x7f8010100020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.043+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 shutdown_connections 2023-10-16T14:31:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.043+0000 7f80173ff700 1 -- 172.21.15.178:0/1299634671 wait complete. 2023-10-16T14:31:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_data' created 2023-10-16T14:31:36.076 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T14:31:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 -- 172.21.15.178:0/1462648106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c073910 msgr2=0x7f3b0c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 --2- 172.21.15.178:0/1462648106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c073910 0x7f3b0c073d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009990 tx=0x7f3af8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 -- 172.21.15.178:0/1462648106 shutdown_connections 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 --2- 172.21.15.178:0/1462648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 --2- 172.21.15.178:0/1462648106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 --2- 172.21.15.178:0/1462648106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c073910 0x7f3b0c073d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 -- 172.21.15.178:0/1462648106 >> 172.21.15.178:0/1462648106 conn(0x7f3b0c0fc400 msgr2=0x7f3b0c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 -- 172.21.15.178:0/1462648106 shutdown_connections 2023-10-16T14:31:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.280+0000 7f3b12fff700 1 -- 172.21.15.178:0/1462648106 wait complete. 2023-10-16T14:31:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.281+0000 7f3b12fff700 1 Processor -- start 2023-10-16T14:31:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.281+0000 7f3b12fff700 1 -- start start 2023-10-16T14:31:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 0x7f3b0c19d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c1a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a4e10 con 0x7f3b0c073910 2023-10-16T14:31:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a4f50 con 0x7f3b0c074ef0 2023-10-16T14:31:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b12fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c1a50e0 con 0x7f3b0c10d4a0 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b10d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 0x7f3b0c19d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b1159c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c1a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b10d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 0x7f3b0c19d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51278/0 (socket says 172.21.15.178:51278) 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b10d9b700 1 -- 172.21.15.178:0/124565399 learned_addr learned my addr 172.21.15.178:0/124565399 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 -- 172.21.15.178:0/124565399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 msgr2=0x7f3b0c1a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c1a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 -- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 msgr2=0x7f3b0c19d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 0x7f3b0c19d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b0bfff700 1 -- 172.21.15.178:0/124565399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80095f0 con 0x7f3b0c074ef0 2023-10-16T14:31:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.282+0000 7f3b1159c700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b0bfff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c19d7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b680 tx=0x7f3af8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af801d070 con 0x7f3b0c074ef0 2023-10-16T14:31:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3af8021410 con 0x7f3b0c074ef0 2023-10-16T14:31:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017650 con 0x7f3b0c074ef0 2023-10-16T14:31:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c1a5360 con 0x7f3b0c074ef0 2023-10-16T14:31:36.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.283+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1a5730 con 0x7f3b0c074ef0 2023-10-16T14:31:36.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.284+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c102ff0 con 0x7f3b0c074ef0 2023-10-16T14:31:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.286+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3af8021810 con 0x7f3b0c074ef0 2023-10-16T14:31:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.287+0000 7f3b09ffb700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.287+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f3af80cdfe0 con 0x7f3b0c074ef0 2023-10-16T14:31:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.288+0000 7f3b10d9b700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.288+0000 7f3b10d9b700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 0x7f3afc0a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009d20 tx=0x7f3b00009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.291+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3af80977e0 con 0x7f3b0c074ef0 2023-10-16T14:31:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:36 smithi178 ceph-mon[98495]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:36 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-16T14:31:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:36 smithi178 ceph-mon[98495]: osdmap e80: 12 total, 12 up, 12 in 2023-10-16T14:31:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:36 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1299634671' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:36 smithi186 ceph-mon[134892]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:36 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-16T14:31:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:36 smithi186 ceph-mon[134892]: osdmap e80: 12 total, 12 up, 12 in 2023-10-16T14:31:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:36 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:36.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:36 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1299634671' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:36.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:36 smithi187 ceph-mon[135297]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:36 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-10-16T14:31:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:36 smithi187 ceph-mon[135297]: osdmap e80: 12 total, 12 up, 12 in 2023-10-16T14:31:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:36 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:31:36.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:36 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1299634671' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:36.551+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b0c112d10 con 0x7f3b0c074ef0 2023-10-16T14:31:37.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.053+0000 7f3b09ffb700 1 -- 172.21.15.178:0/124565399 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3af809ae00 con 0x7f3b0c074ef0 2023-10-16T14:31:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.056+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 msgr2=0x7f3afc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:37.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.056+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 0x7f3afc0a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009d20 tx=0x7f3b00009260 comp rx=0 tx=0).stop 2023-10-16T14:31:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.056+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 msgr2=0x7f3b0c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.056+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c19d7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3af800b680 tx=0x7f3af8009580 comp rx=0 tx=0).stop 2023-10-16T14:31:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 shutdown_connections 2023-10-16T14:31:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1480 0x7f3afc0a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c10d4a0 0x7f3b0c1a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b0c073910 0x7f3b0c19d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 --2- 172.21.15.178:0/124565399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b0c074ef0 0x7f3b0c19d7d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 >> 172.21.15.178:0/124565399 conn(0x7f3b0c0fc400 msgr2=0x7f3b0c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:37.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.057+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 shutdown_connections 2023-10-16T14:31:37.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.058+0000 7f3b12fff700 1 -- 172.21.15.178:0/124565399 wait complete. 2023-10-16T14:31:37.064 INFO:teuthology.orchestra.run.smithi178.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T14:31:37.083 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-10-16T14:31:37.083 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-10-16T14:31:37.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 -- 172.21.15.178:0/1413994509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102060 msgr2=0x7f29141024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:37.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 --2- 172.21.15.178:0/1413994509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102060 0x7f29141024e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2904009990 tx=0x7f2904009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 -- 172.21.15.178:0/1413994509 shutdown_connections 2023-10-16T14:31:37.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 --2- 172.21.15.178:0/1413994509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914102a20 0x7f2914109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 --2- 172.21.15.178:0/1413994509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914100e80 0x7f29141012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 --2- 172.21.15.178:0/1413994509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102060 0x7f29141024e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 -- 172.21.15.178:0/1413994509 >> 172.21.15.178:0/1413994509 conn(0x7f29140fc400 msgr2=0x7f29140fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:37.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 -- 172.21.15.178:0/1413994509 shutdown_connections 2023-10-16T14:31:37.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.295+0000 7f291c1ff700 1 -- 172.21.15.178:0/1413994509 wait complete. 2023-10-16T14:31:37.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 Processor -- start 2023-10-16T14:31:37.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 -- start start 2023-10-16T14:31:37.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:37.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914102060 0x7f29141993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:37.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102a20 0x7f29141a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:37.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a0a10 con 0x7f2914102060 2023-10-16T14:31:37.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a0b50 con 0x7f2914102a20 2023-10-16T14:31:37.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291c1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29141a0ce0 con 0x7f2914100e80 2023-10-16T14:31:37.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f2919f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:37.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291979a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914102060 0x7f29141993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:37.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f291a79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102a20 0x7f29141a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:37.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f2919f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:37956/0 (socket says 172.21.15.178:37956) 2023-10-16T14:31:37.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.296+0000 7f2919f9b700 1 -- 172.21.15.178:0/2669954112 learned_addr learned my addr 172.21.15.178:0/2669954112 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 -- 172.21.15.178:0/2669954112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102a20 msgr2=0x7f29141a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102a20 0x7f29141a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 -- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914102060 msgr2=0x7f29141993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914102060 0x7f29141993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 -- 172.21.15.178:0/2669954112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040095f0 con 0x7f2914100e80 2023-10-16T14:31:37.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f2919f9b700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f291000e970 tx=0x7f291000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:37.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910009790 con 0x7f2914100e80 2023-10-16T14:31:37.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29141a0fc0 con 0x7f2914100e80 2023-10-16T14:31:37.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f291000c7e0 con 0x7f2914100e80 2023-10-16T14:31:37.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.297+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2910003c70 con 0x7f2914100e80 2023-10-16T14:31:37.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.298+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29141a13e0 con 0x7f2914100e80 2023-10-16T14:31:37.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.299+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29141934e0 con 0x7f2914100e80 2023-10-16T14:31:37.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:37.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-16T14:31:37.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: osdmap e81: 12 total, 12 up, 12 in 2023-10-16T14:31:37.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/124565399' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-16T14:31:37.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:37 smithi178 ceph-mon[98495]: pgmap v266: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:37.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.301+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f291000cc50 con 0x7f2914100e80 2023-10-16T14:31:37.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.302+0000 7f290affd700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 0x7f29000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:37.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.302+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2910014070 con 0x7f2914100e80 2023-10-16T14:31:37.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.302+0000 7f291979a700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 0x7f29000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:37.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.303+0000 7f291979a700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 0x7f29000a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f290400b680 tx=0x7f2904005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:37.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.306+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f29100964f0 con 0x7f2914100e80 2023-10-16T14:31:37.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-16T14:31:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: osdmap e81: 12 total, 12 up, 12 in 2023-10-16T14:31:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/124565399' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-16T14:31:37.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:37 smithi186 ceph-mon[134892]: pgmap v266: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-10-16T14:31:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-10-16T14:31:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: osdmap e81: 12 total, 12 up, 12 in 2023-10-16T14:31:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/124565399' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-10-16T14:31:37.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-10-16T14:31:37.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:37 smithi187 ceph-mon[135297]: pgmap v266: 97 pgs: 21 creating+peering, 7 active+clean, 69 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:37.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:37.562+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f291410c6f0 con 0x7f2914100e80 2023-10-16T14:31:38.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.023+0000 7f290affd700 1 -- 172.21.15.178:0/2669954112 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2910003dd0 con 0x7f2914100e80 2023-10-16T14:31:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 msgr2=0x7f29000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:38.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 0x7f29000a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f290400b680 tx=0x7f2904005fd0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 msgr2=0x7f2914198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f291000e970 tx=0x7f291000ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 shutdown_connections 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29000a1480 0x7f29000a3940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2914102a20 0x7f29141a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2914100e80 0x7f2914198e90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 --2- 172.21.15.178:0/2669954112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2914102060 0x7f29141993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.026+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 >> 172.21.15.178:0/2669954112 conn(0x7f29140fc400 msgr2=0x7f2914107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:38.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.027+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 shutdown_connections 2023-10-16T14:31:38.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.027+0000 7f291c1ff700 1 -- 172.21.15.178:0/2669954112 wait complete. 2023-10-16T14:31:38.050 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T14:31:38.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3242422246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 msgr2=0x7fbc24073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:38.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3242422246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc24073340 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc20009990 tx=0x7fbc20009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3242422246 shutdown_connections 2023-10-16T14:31:38.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3242422246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 0x7fbc2410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3242422246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc24073340 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3242422246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc24073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3242422246 >> 172.21.15.178:0/3242422246 conn(0x7fbc240fc450 msgr2=0x7fbc240fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:38.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3242422246 shutdown_connections 2023-10-16T14:31:38.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.271+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3242422246 wait complete. 2023-10-16T14:31:38.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 Processor -- start 2023-10-16T14:31:38.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 -- start start 2023-10-16T14:31:38.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:38.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc2419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 0x7fbc241a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc241a4c60 con 0x7fbc24074ef0 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc241a4da0 con 0x7fbc24073910 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2bdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc241a4f30 con 0x7fbc2410d4d0 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc29b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:38.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2939a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc2419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:38.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc29b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50476/0 (socket says 172.21.15.178:50476) 2023-10-16T14:31:38.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc29b9b700 1 -- 172.21.15.178:0/3466805955 learned_addr learned my addr 172.21.15.178:0/3466805955 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:38.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.273+0000 7fbc2a39c700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 0x7fbc241a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:38.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 -- 172.21.15.178:0/3466805955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 msgr2=0x7fbc241a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:38.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 0x7fbc241a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 -- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 msgr2=0x7fbc2419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:38.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc2419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:38.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 -- 172.21.15.178:0/3466805955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009520 con 0x7fbc24073910 2023-10-16T14:31:38.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc29b9b700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc20004730 tx=0x7fbc20004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:38.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2001d070 con 0x7fbc24073910 2023-10-16T14:31:38.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc20022470 con 0x7fbc24073910 2023-10-16T14:31:38.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc200095f0 con 0x7fbc24073910 2023-10-16T14:31:38.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017650 con 0x7fbc24073910 2023-10-16T14:31:38.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.274+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc241a5440 con 0x7fbc24073910 2023-10-16T14:31:38.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.276+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc24103020 con 0x7fbc24073910 2023-10-16T14:31:38.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.277+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc200225e0 con 0x7fbc24073910 2023-10-16T14:31:38.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.278+0000 7fbc1affd700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 0x7fbc100a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:38.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.279+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fbc200ce0f0 con 0x7fbc24073910 2023-10-16T14:31:38.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.279+0000 7fbc2939a700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 0x7fbc100a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:38.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.279+0000 7fbc2939a700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 0x7fbc100a39e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc24074c10 tx=0x7fbc14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:38.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.282+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc200977a0 con 0x7fbc24073910 2023-10-16T14:31:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: osdmap e82: 12 total, 12 up, 12 in 2023-10-16T14:31:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-16T14:31:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-16T14:31:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-16T14:31:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2669954112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: osdmap e82: 12 total, 12 up, 12 in 2023-10-16T14:31:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-16T14:31:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-16T14:31:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-16T14:31:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2669954112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: osdmap e82: 12 total, 12 up, 12 in 2023-10-16T14:31:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-10-16T14:31:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-10-16T14:31:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2669954112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:38.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:38.506+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc24112d40 con 0x7fbc24073910 2023-10-16T14:31:39.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-16T14:31:39.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: osdmap e83: 12 total, 12 up, 12 in 2023-10-16T14:31:39.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:active 2023-10-16T14:31:39.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3466805955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: pgmap v269: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-16T14:31:39.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:39.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:39.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-16T14:31:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: osdmap e83: 12 total, 12 up, 12 in 2023-10-16T14:31:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:active 2023-10-16T14:31:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3466805955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: pgmap v269: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-16T14:31:39.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:39.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:39.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-10-16T14:31:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: osdmap e83: 12 total, 12 up, 12 in 2023-10-16T14:31:39.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] up:active 2023-10-16T14:31:39.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:39.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3466805955' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-10-16T14:31:39.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: pgmap v269: 97 pgs: 21 creating+peering, 32 active+clean, 44 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2023-10-16T14:31:39.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:39.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:31:39.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:31:39.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:31:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.066+0000 7fbc1affd700 1 -- 172.21.15.178:0/3466805955 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbc2009adc0 con 0x7fbc24073910 2023-10-16T14:31:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 msgr2=0x7fbc100a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 0x7fbc100a39e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc24074c10 tx=0x7fbc14009260 comp rx=0 tx=0).stop 2023-10-16T14:31:40.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 msgr2=0x7fbc2419d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:40.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc20004730 tx=0x7fbc20004760 comp rx=0 tx=0).stop 2023-10-16T14:31:40.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 shutdown_connections 2023-10-16T14:31:40.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc100a1520 0x7fbc100a39e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc2410d4d0 0x7fbc241a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc24073910 0x7fbc2419d0e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 --2- 172.21.15.178:0/3466805955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc24074ef0 0x7fbc2419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.071+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 >> 172.21.15.178:0/3466805955 conn(0x7fbc240fc450 msgr2=0x7fbc240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:40.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.072+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 shutdown_connections 2023-10-16T14:31:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.072+0000 7fbc2bdff700 1 -- 172.21.15.178:0/3466805955 wait complete. 2023-10-16T14:31:40.078 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_data_ec' created 2023-10-16T14:31:40.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-10-16T14:31:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:40 smithi178 ceph-mon[98495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-16T14:31:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:40 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:31:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:40 smithi178 ceph-mon[98495]: osdmap e84: 12 total, 12 up, 12 in 2023-10-16T14:31:40.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 -- 172.21.15.178:0/1874132048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102060 msgr2=0x7fbc5c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:40.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 --2- 172.21.15.178:0/1874132048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102060 0x7fbc5c1024e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009990 tx=0x7fbc4c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 -- 172.21.15.178:0/1874132048 shutdown_connections 2023-10-16T14:31:40.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 --2- 172.21.15.178:0/1874132048 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c102a20 0x7fbc5c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 --2- 172.21.15.178:0/1874132048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c100e80 0x7fbc5c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 --2- 172.21.15.178:0/1874132048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102060 0x7fbc5c1024e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.299+0000 7fbc63fff700 1 -- 172.21.15.178:0/1874132048 >> 172.21.15.178:0/1874132048 conn(0x7fbc5c0fc400 msgr2=0x7fbc5c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:40.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.300+0000 7fbc63fff700 1 -- 172.21.15.178:0/1874132048 shutdown_connections 2023-10-16T14:31:40.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.300+0000 7fbc63fff700 1 -- 172.21.15.178:0/1874132048 wait complete. 2023-10-16T14:31:40.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.300+0000 7fbc63fff700 1 Processor -- start 2023-10-16T14:31:40.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 -- start start 2023-10-16T14:31:40.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c100e80 0x7fbc5c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:40.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:40.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102a20 0x7fbc5c19e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:40.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c19e760 con 0x7fbc5c102060 2023-10-16T14:31:40.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c19e8d0 con 0x7fbc5c100e80 2023-10-16T14:31:40.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc63fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c19ea90 con 0x7fbc5c102a20 2023-10-16T14:31:40.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc6159a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:40.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc6159a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52204/0 (socket says 172.21.15.178:52204) 2023-10-16T14:31:40.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc6159a700 1 -- 172.21.15.178:0/878372914 learned_addr learned my addr 172.21.15.178:0/878372914 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:40.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc61d9b700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c100e80 0x7fbc5c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:40.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.301+0000 7fbc6259c700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102a20 0x7fbc5c19e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:40.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 -- 172.21.15.178:0/878372914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102a20 msgr2=0x7fbc5c19e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:40.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102a20 0x7fbc5c19e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 -- 172.21.15.178:0/878372914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c100e80 msgr2=0x7fbc5c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:40.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c100e80 0x7fbc5c196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:40.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 -- 172.21.15.178:0/878372914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009520 con 0x7fbc5c102060 2023-10-16T14:31:40.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc6159a700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005720 tx=0x7fbc4c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:40.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c01d070 con 0x7fbc5c102060 2023-10-16T14:31:40.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc4c021410 con 0x7fbc5c102060 2023-10-16T14:31:40.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c0095f0 con 0x7fbc5c102060 2023-10-16T14:31:40.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c017650 con 0x7fbc5c102060 2023-10-16T14:31:40.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.302+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c19efa0 con 0x7fbc5c102060 2023-10-16T14:31:40.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.303+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c107e80 con 0x7fbc5c102060 2023-10-16T14:31:40.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.305+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc4c021580 con 0x7fbc5c102060 2023-10-16T14:31:40.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.307+0000 7fbc52ffd700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 0x7fbc480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:40.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.307+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fbc4c0ce5a0 con 0x7fbc5c102060 2023-10-16T14:31:40.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.307+0000 7fbc61d9b700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 0x7fbc480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:40.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.307+0000 7fbc61d9b700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 0x7fbc480a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c1980f0 tx=0x7fbc58009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:40.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.309+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc4c097a60 con 0x7fbc5c102060 2023-10-16T14:31:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:40 smithi186 ceph-mon[134892]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-16T14:31:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:40 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:31:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:40 smithi186 ceph-mon[134892]: osdmap e84: 12 total, 12 up, 12 in 2023-10-16T14:31:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:40 smithi187 ceph-mon[135297]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-10-16T14:31:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:40 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:31:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:40 smithi187 ceph-mon[135297]: osdmap e84: 12 total, 12 up, 12 in 2023-10-16T14:31:40.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:40.508+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fbc5c02d230 con 0x7fbc5c102060 2023-10-16T14:31:41.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.064+0000 7fbc52ffd700 1 -- 172.21.15.178:0/878372914 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbc4c09b080 con 0x7fbc5c102060 2023-10-16T14:31:41.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.068+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 msgr2=0x7fbc480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:41.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.068+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 0x7fbc480a3940 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c1980f0 tx=0x7fbc58009260 comp rx=0 tx=0).stop 2023-10-16T14:31:41.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.068+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 msgr2=0x7fbc5c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:41.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.068+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005720 tx=0x7fbc4c009580 comp rx=0 tx=0).stop 2023-10-16T14:31:41.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 shutdown_connections 2023-10-16T14:31:41.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc480a1480 0x7fbc480a3940 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc5c102a20 0x7fbc5c19e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc5c100e80 0x7fbc5c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 --2- 172.21.15.178:0/878372914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc5c102060 0x7fbc5c197160 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 >> 172.21.15.178:0/878372914 conn(0x7fbc5c0fc400 msgr2=0x7fbc5c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:41.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.069+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 shutdown_connections 2023-10-16T14:31:41.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.070+0000 7fbc63fff700 1 -- 172.21.15.178:0/878372914 wait complete. 2023-10-16T14:31:41.078 INFO:teuthology.orchestra.run.smithi178.stderr:set pool 4 allow_ec_overwrites to true 2023-10-16T14:31:41.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-10-16T14:31:41.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 -- 172.21.15.178:0/2050343306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838102a20 msgr2=0x7f9838109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:41.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 --2- 172.21.15.178:0/2050343306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838102a20 0x7f9838109340 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f983000b0a0 tx=0x7f983000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 -- 172.21.15.178:0/2050343306 shutdown_connections 2023-10-16T14:31:41.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 --2- 172.21.15.178:0/2050343306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838102a20 0x7f9838109340 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 --2- 172.21.15.178:0/2050343306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838100e80 0x7f98381012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 --2- 172.21.15.178:0/2050343306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102060 0x7f98381024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 -- 172.21.15.178:0/2050343306 >> 172.21.15.178:0/2050343306 conn(0x7f98380fc400 msgr2=0x7f98380fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:41.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 -- 172.21.15.178:0/2050343306 shutdown_connections 2023-10-16T14:31:41.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.313+0000 7f983fdff700 1 -- 172.21.15.178:0/2050343306 wait complete. 2023-10-16T14:31:41.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 Processor -- start 2023-10-16T14:31:41.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 -- start start 2023-10-16T14:31:41.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:41.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 0x7f9838197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:41.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 0x7f983819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:41.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819e880 con 0x7f9838100e80 2023-10-16T14:31:41.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819e9f0 con 0x7f9838102a20 2023-10-16T14:31:41.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.314+0000 7f983fdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819ebb0 con 0x7f9838102060 2023-10-16T14:31:41.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:41.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52230/0 (socket says 172.21.15.178:52230) 2023-10-16T14:31:41.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 -- 172.21.15.178:0/1732863992 learned_addr learned my addr 172.21.15.178:0/1732863992 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:41.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983d39a700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 0x7f9838197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:41.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983e39c700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 0x7f983819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 -- 172.21.15.178:0/1732863992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 msgr2=0x7f9838197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 0x7f9838197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 -- 172.21.15.178:0/1732863992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 msgr2=0x7f983819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 0x7f983819e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:41.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 -- 172.21.15.178:0/1732863992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9830009d00 con 0x7f9838100e80 2023-10-16T14:31:41.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983d39a700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 0x7f9838197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:41.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983e39c700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 0x7f983819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:41.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f983db9b700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f983400e970 tx=0x7f983400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:41.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834009790 con 0x7f9838100e80 2023-10-16T14:31:41.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.315+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98340044e0 con 0x7f9838100e80 2023-10-16T14:31:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.316+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983819ee90 con 0x7f9838100e80 2023-10-16T14:31:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.316+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834003e30 con 0x7f9838100e80 2023-10-16T14:31:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.316+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983819f2c0 con 0x7f9838100e80 2023-10-16T14:31:41.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.317+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983810c740 con 0x7f9838100e80 2023-10-16T14:31:41.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.318+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9834003f90 con 0x7f9838100e80 2023-10-16T14:31:41.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.320+0000 7f982effd700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 0x7f98240a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:41.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.320+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f9834014070 con 0x7f9838100e80 2023-10-16T14:31:41.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.320+0000 7f983d39a700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 0x7f98240a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:41.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.321+0000 7f983d39a700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 0x7f98240a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9838100ba0 tx=0x7f9828009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:41.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.322+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9834096720 con 0x7f9838100e80 2023-10-16T14:31:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:41 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-16T14:31:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:41 smithi187 ceph-mon[135297]: osdmap e85: 12 total, 12 up, 12 in 2023-10-16T14:31:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:41 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-16T14:31:41.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:41 smithi187 ceph-mon[135297]: pgmap v272: 129 pgs: 20 creating+peering, 81 active+clean, 28 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-16T14:31:41.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:41 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-16T14:31:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:41 smithi178 ceph-mon[98495]: osdmap e85: 12 total, 12 up, 12 in 2023-10-16T14:31:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:41 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-16T14:31:41.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:41 smithi178 ceph-mon[98495]: pgmap v272: 129 pgs: 20 creating+peering, 81 active+clean, 28 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-16T14:31:41.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:41.568+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f983819f540 con 0x7f9838100e80 2023-10-16T14:31:41.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:41 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-10-16T14:31:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:41 smithi186 ceph-mon[134892]: osdmap e85: 12 total, 12 up, 12 in 2023-10-16T14:31:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:41 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-10-16T14:31:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:41 smithi186 ceph-mon[134892]: pgmap v272: 129 pgs: 20 creating+peering, 81 active+clean, 28 unknown; 578 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 9 op/s 2023-10-16T14:31:42.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.181+0000 7f982effd700 1 -- 172.21.15.178:0/1732863992 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9834099d40 con 0x7f9838100e80 2023-10-16T14:31:42.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 msgr2=0x7f98240a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 0x7f98240a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9838100ba0 tx=0x7f9828009260 comp rx=0 tx=0).stop 2023-10-16T14:31:42.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 msgr2=0x7f9838196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f983400e970 tx=0x7f983400ed30 comp rx=0 tx=0).stop 2023-10-16T14:31:42.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 shutdown_connections 2023-10-16T14:31:42.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98240a1500 0x7f98240a39c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9838102a20 0x7f983819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9838100e80 0x7f9838196d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 --2- 172.21.15.178:0/1732863992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9838102060 0x7f9838197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.185+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 >> 172.21.15.178:0/1732863992 conn(0x7f98380fc400 msgr2=0x7f98380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:42.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.186+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 shutdown_connections 2023-10-16T14:31:42.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.186+0000 7f983fdff700 1 -- 172.21.15.178:0/1732863992 wait complete. 2023-10-16T14:31:42.191 INFO:teuthology.orchestra.run.smithi178.stderr:added data pool 4 to fsmap 2023-10-16T14:31:42.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:42 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-16T14:31:42.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:42 smithi186 ceph-mon[134892]: osdmap e86: 12 total, 12 up, 12 in 2023-10-16T14:31:42.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:42 smithi186 ceph-mon[134892]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:42.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:42 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-16T14:31:42.327 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:31:41 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: 2023-10-16T14:31:41.846+0000 7ff450db5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T14:31:42.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.405+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/4181708998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08102090 msgr2=0x7fcf08102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.405+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/4181708998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08102090 0x7fcf08102510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009990 tx=0x7fcefc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/4181708998 shutdown_connections 2023-10-16T14:31:42.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/4181708998 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 0x7fcf08109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/4181708998 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08100eb0 0x7fcf081012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/4181708998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08102090 0x7fcf08102510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/4181708998 >> 172.21.15.178:0/4181708998 conn(0x7fcf080fc430 msgr2=0x7fcf080fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/4181708998 shutdown_connections 2023-10-16T14:31:42.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.406+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/4181708998 wait complete. 2023-10-16T14:31:42.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 Processor -- start 2023-10-16T14:31:42.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 -- start start 2023-10-16T14:31:42.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08102090 0x7fcf080732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 0x7fcf080737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0819aa40 con 0x7fcf08100eb0 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08073d20 con 0x7fcf08102090 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0e3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08073ee0 con 0x7fcf08102a50 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf07fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf07fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52256/0 (socket says 172.21.15.178:52256) 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf07fff700 1 -- 172.21.15.178:0/3634392074 learned_addr learned my addr 172.21.15.178:0/3634392074 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:42.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf0c99c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 0x7fcf080737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.407+0000 7fcf077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08102090 0x7fcf080732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 -- 172.21.15.178:0/3634392074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 msgr2=0x7fcf080737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 0x7fcf080737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 -- 172.21.15.178:0/3634392074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08102090 msgr2=0x7fcf080732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08102090 0x7fcf080732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 -- 172.21.15.178:0/3634392074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0095f0 con 0x7fcf08100eb0 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf07fff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d980 tx=0x7fcef800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef800f890 con 0x7fcf08100eb0 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcef8010460 con 0x7fcf08100eb0 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80195c0 con 0x7fcf08100eb0 2023-10-16T14:31:42.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf081a53f0 con 0x7fcf08100eb0 2023-10-16T14:31:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.408+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081a5810 con 0x7fcf08100eb0 2023-10-16T14:31:42.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.409+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf08107e30 con 0x7fcf08100eb0 2023-10-16T14:31:42.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.411+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcef80105d0 con 0x7fcf08100eb0 2023-10-16T14:31:42.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.412+0000 7fcf057fa700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 0x7fcef00a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.412+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcef80ccc60 con 0x7fcf08100eb0 2023-10-16T14:31:42.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.413+0000 7fcf077fe700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 0x7fcef00a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.413+0000 7fcf077fe700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 0x7fcef00a3a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b5c0 tx=0x7fcefc005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:42.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.417+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcef8096070 con 0x7fcf08100eb0 2023-10-16T14:31:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:42 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-16T14:31:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:42 smithi187 ceph-mon[135297]: osdmap e86: 12 total, 12 up, 12 in 2023-10-16T14:31:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:42 smithi187 ceph-mon[135297]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:42.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:42 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-16T14:31:42.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:42 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/878372914' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-10-16T14:31:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:42 smithi178 ceph-mon[98495]: osdmap e86: 12 total, 12 up, 12 in 2023-10-16T14:31:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:42 smithi178 ceph-mon[98495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-16T14:31:42.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:42 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-10-16T14:31:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.667+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf0810c770 con 0x7fcf08100eb0 2023-10-16T14:31:42.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.668+0000 7fcf057fa700 1 -- 172.21.15.178:0/3634392074 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11279 (secure 0 0 0) 0x7fcef8016090 con 0x7fcf08100eb0 2023-10-16T14:31:42.672 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:42.672 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:31:41.570556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T14:31:42.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 msgr2=0x7fcef00a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 0x7fcef00a3a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00b5c0 tx=0x7fcefc005fd0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 msgr2=0x7fcf08074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcef800d980 tx=0x7fcef800dc90 comp rx=0 tx=0).stop 2023-10-16T14:31:42.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 shutdown_connections 2023-10-16T14:31:42.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcef00a15a0 0x7fcef00a3a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf08102a50 0x7fcf080737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf08100eb0 0x7fcf08074c60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 --2- 172.21.15.178:0/3634392074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf08102090 0x7fcf080732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 >> 172.21.15.178:0/3634392074 conn(0x7fcf080fc430 msgr2=0x7fcf080fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:42.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 shutdown_connections 2023-10-16T14:31:42.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.671+0000 7fcf0e3ff700 1 -- 172.21.15.178:0/3634392074 wait complete. 2023-10-16T14:31:42.677 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 17 2023-10-16T14:31:42.693 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:42.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 -- 172.21.15.178:0/3404844780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc100ea0 msgr2=0x7f14cc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3404844780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc100ea0 0x7f14cc1012c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009990 tx=0x7f14bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 -- 172.21.15.178:0/3404844780 shutdown_connections 2023-10-16T14:31:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3404844780 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc102a40 0x7f14cc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3404844780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc100ea0 0x7f14cc1012c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3404844780 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102080 0x7f14cc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 -- 172.21.15.178:0/3404844780 >> 172.21.15.178:0/3404844780 conn(0x7f14cc0fc420 msgr2=0x7f14cc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.886+0000 7f14d1bff700 1 -- 172.21.15.178:0/3404844780 shutdown_connections 2023-10-16T14:31:42.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.887+0000 7f14d1bff700 1 -- 172.21.15.178:0/3404844780 wait complete. 2023-10-16T14:31:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.887+0000 7f14d1bff700 1 Processor -- start 2023-10-16T14:31:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 -- start start 2023-10-16T14:31:42.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc100ea0 0x7f14cc198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102a40 0x7f14cc1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc1a08e0 con 0x7f14cc102080 2023-10-16T14:31:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc1a0a20 con 0x7f14cc100ea0 2023-10-16T14:31:42.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14d1bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14cc1a0bb0 con 0x7f14cc102a40 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52280/0 (socket says 172.21.15.178:52280) 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 -- 172.21.15.178:0/3756294782 learned_addr learned my addr 172.21.15.178:0/3756294782 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14cbfff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102a40 0x7f14cc1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14cb7fe700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc100ea0 0x7f14cc198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 -- 172.21.15.178:0/3756294782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102a40 msgr2=0x7f14cc1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102a40 0x7f14cc1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 -- 172.21.15.178:0/3756294782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc100ea0 msgr2=0x7f14cc198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc100ea0 0x7f14cc198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.888+0000 7f14caffd700 1 -- 172.21.15.178:0/3756294782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0095f0 con 0x7f14cc102080 2023-10-16T14:31:42.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14caffd700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e970 tx=0x7f14c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0009790 con 0x7f14cc102080 2023-10-16T14:31:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14c000c7e0 con 0x7f14cc102080 2023-10-16T14:31:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14cc1a0e90 con 0x7f14cc102080 2023-10-16T14:31:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0003c70 con 0x7f14cc102080 2023-10-16T14:31:42.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.889+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14cc1a1360 con 0x7f14cc102080 2023-10-16T14:31:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.890+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14cc10c760 con 0x7f14cc102080 2023-10-16T14:31:42.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.894+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f14c000cc50 con 0x7f14cc102080 2023-10-16T14:31:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.895+0000 7f14c8ff9700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 0x7f14b40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.895+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f14c0014070 con 0x7f14cc102080 2023-10-16T14:31:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.895+0000 7f14cb7fe700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 0x7f14b40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:42.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.896+0000 7f14cb7fe700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 0x7f14b40a3a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00b2e0 tx=0x7f14bc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:42.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:42.897+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f14c0096610 con 0x7f14cc102080 2023-10-16T14:31:43.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.100+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14cc10c940 con 0x7f14cc102080 2023-10-16T14:31:43.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.102+0000 7f14c8ff9700 1 -- 172.21.15.178:0/3756294782 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f14c0003dd0 con 0x7f14cc102080 2023-10-16T14:31:43.106 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:43.106 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:43.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 msgr2=0x7f14b40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 0x7f14b40a3a60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00b2e0 tx=0x7f14bc005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 msgr2=0x7f14cc1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f14c000e970 tx=0x7f14c000ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 shutdown_connections 2023-10-16T14:31:43.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f14b40a15a0 0x7f14b40a3a60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14cc102a40 0x7f14cc1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f14cc100ea0 0x7f14cc198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 --2- 172.21.15.178:0/3756294782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14cc102080 0x7f14cc1992a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 >> 172.21.15.178:0/3756294782 conn(0x7f14cc0fc420 msgr2=0x7f14cc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:43.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 shutdown_connections 2023-10-16T14:31:43.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.104+0000 7f14d1bff700 1 -- 172.21.15.178:0/3756294782 wait complete. 2023-10-16T14:31:43.128 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:43.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/199861629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 msgr2=0x7f0f94103550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/199861629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94103550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0f840099c0 tx=0x7f0f84009cd0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/199861629 shutdown_connections 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/199861629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 0x7f0f9410b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/199861629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94103550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/199861629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/199861629 >> 172.21.15.178:0/199861629 conn(0x7f0f940fe660 msgr2=0x7f0f94100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/199861629 shutdown_connections 2023-10-16T14:31:43.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.324+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/199861629 wait complete. 2023-10-16T14:31:43.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 Processor -- start 2023-10-16T14:31:43.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 -- start start 2023-10-16T14:31:43.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 0x7f0f941a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941a08d0 con 0x7f0f94103130 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941a0a10 con 0x7f0f94104320 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.325+0000 7f0f9a9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f941a0ba0 con 0x7f0f94104cc0 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52290/0 (socket says 172.21.15.178:52290) 2023-10-16T14:31:43.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 -- 172.21.15.178:0/4287439701 learned_addr learned my addr 172.21.15.178:0/4287439701 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:43.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f98f9c700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 0x7f0f941a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f8bfff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 -- 172.21.15.178:0/4287439701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 msgr2=0x7f0f941a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 0x7f0f941a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 -- 172.21.15.178:0/4287439701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 msgr2=0x7f0f94199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 -- 172.21.15.178:0/4287439701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f840095f0 con 0x7f0f94103130 2023-10-16T14:31:43.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f8bfff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:43.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f93fff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f840036f0 tx=0x7f0f84009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:43.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401d070 con 0x7f0f94103130 2023-10-16T14:31:43.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f84004010 con 0x7f0f94103130 2023-10-16T14:31:43.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400fe70 con 0x7f0f94103130 2023-10-16T14:31:43.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.326+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941a0e20 con 0x7f0f94103130 2023-10-16T14:31:43.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.327+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941a1190 con 0x7f0f94103130 2023-10-16T14:31:43.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.328+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9410a620 con 0x7f0f94103130 2023-10-16T14:31:43.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.329+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0f8400f560 con 0x7f0f94103130 2023-10-16T14:31:43.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.331+0000 7f0f91ffb700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 0x7f0f800a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.331+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0f840cddf0 con 0x7f0f94103130 2023-10-16T14:31:43.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.331+0000 7f0f8bfff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 0x7f0f800a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.331+0000 7f0f8bfff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 0x7f0f800a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005950 tx=0x7f0f7c0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:43.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.334+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0f84097290 con 0x7f0f94103130 2023-10-16T14:31:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:43 smithi187 ceph-mon[135297]: osdmap e87: 12 total, 12 up, 12 in 2023-10-16T14:31:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:43 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-16T14:31:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:43 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:43 smithi187 ceph-mon[135297]: pgmap v275: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-16T14:31:43.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:43 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3634392074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:43.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.538+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f9410ea00 con 0x7f0f94103130 2023-10-16T14:31:43.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.540+0000 7f0f91ffb700 1 -- 172.21.15.178:0/4287439701 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f0f84079e60 con 0x7f0f94103130 2023-10-16T14:31:43.543 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:43.543 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 msgr2=0x7f0f800a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 0x7f0f800a3a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c005950 tx=0x7f0f7c0058e0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 msgr2=0x7f0f94198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0f840036f0 tx=0x7f0f84009580 comp rx=0 tx=0).stop 2023-10-16T14:31:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 shutdown_connections 2023-10-16T14:31:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0f800a15a0 0x7f0f800a3a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f94104cc0 0x7f0f941a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f94103130 0x7f0f94198d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 --2- 172.21.15.178:0/4287439701 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0f94104320 0x7f0f94199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 >> 172.21.15.178:0/4287439701 conn(0x7f0f940fe660 msgr2=0x7f0f94107760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:43.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.542+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 shutdown_connections 2023-10-16T14:31:43.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.543+0000 7f0f9a9ff700 1 -- 172.21.15.178:0/4287439701 wait complete. 2023-10-16T14:31:43.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:43 smithi178 ceph-mon[98495]: osdmap e87: 12 total, 12 up, 12 in 2023-10-16T14:31:43.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:43 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-16T14:31:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:43 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:43 smithi178 ceph-mon[98495]: pgmap v275: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-16T14:31:43.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:43 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3634392074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:43.579 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:31:43.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:43 smithi186 ceph-mon[134892]: osdmap e87: 12 total, 12 up, 12 in 2023-10-16T14:31:43.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:43 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1732863992' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-10-16T14:31:43.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:43 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:31:43.580 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:43 smithi186 ceph-mon[134892]: pgmap v275: 129 pgs: 4 creating+peering, 106 active+clean, 19 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 9 op/s 2023-10-16T14:31:43.581 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:43 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3634392074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:43.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 -- 172.21.15.178:0/1929459036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064102080 msgr2=0x7f1064102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 --2- 172.21.15.178:0/1929459036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064102080 0x7f1064102500 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1054009990 tx=0x7f1054009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 -- 172.21.15.178:0/1929459036 shutdown_connections 2023-10-16T14:31:43.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 --2- 172.21.15.178:0/1929459036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f1064109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 --2- 172.21.15.178:0/1929459036 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064100ea0 0x7f10641012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 --2- 172.21.15.178:0/1929459036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064102080 0x7f1064102500 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 -- 172.21.15.178:0/1929459036 >> 172.21.15.178:0/1929459036 conn(0x7f10640fc420 msgr2=0x7f10640fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:43.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 -- 172.21.15.178:0/1929459036 shutdown_connections 2023-10-16T14:31:43.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.777+0000 7f106b7ff700 1 -- 172.21.15.178:0/1929459036 wait complete. 2023-10-16T14:31:43.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.778+0000 7f106b7ff700 1 Processor -- start 2023-10-16T14:31:43.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 -- start start 2023-10-16T14:31:43.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 0x7f10641992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f10641a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10641a08e0 con 0x7f1064100ea0 2023-10-16T14:31:43.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10641a0a20 con 0x7f1064102080 2023-10-16T14:31:43.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106b7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10641a0bb0 con 0x7f1064102a40 2023-10-16T14:31:43.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52312/0 (socket says 172.21.15.178:52312) 2023-10-16T14:31:43.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 -- 172.21.15.178:0/4252927529 learned_addr learned my addr 172.21.15.178:0/4252927529 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:43.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f1068d9a700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 0x7f10641992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f1069d9c700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f10641a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 -- 172.21.15.178:0/4252927529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 msgr2=0x7f10641a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f10641a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 -- 172.21.15.178:0/4252927529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 msgr2=0x7f10641992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:43.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 0x7f10641992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:43.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f106959b700 1 -- 172.21.15.178:0/4252927529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540095f0 con 0x7f1064100ea0 2023-10-16T14:31:43.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.779+0000 7f1069d9c700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f10641a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:31:43.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f1068d9a700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 0x7f10641992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:43.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f106959b700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f106000e970 tx=0x7f106000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:43.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060009790 con 0x7f1064100ea0 2023-10-16T14:31:43.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10600044e0 con 0x7f1064100ea0 2023-10-16T14:31:43.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10641a0e90 con 0x7f1064100ea0 2023-10-16T14:31:43.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060003e30 con 0x7f1064100ea0 2023-10-16T14:31:43.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.780+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10641a12c0 con 0x7f1064100ea0 2023-10-16T14:31:43.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.781+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106410c760 con 0x7f1064100ea0 2023-10-16T14:31:43.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.783+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1060003f90 con 0x7f1064100ea0 2023-10-16T14:31:43.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.784+0000 7f105a7fc700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 0x7f10500a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:43.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.784+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1060014070 con 0x7f1064100ea0 2023-10-16T14:31:43.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.785+0000 7f1068d9a700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 0x7f10500a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:43.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.785+0000 7f1068d9a700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 0x7f10500a3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1054000c00 tx=0x7f1054005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:43.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:43.788+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f10600965c0 con 0x7f1064100ea0 2023-10-16T14:31:44.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.036+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10641a1790 con 0x7f1064100ea0 2023-10-16T14:31:44.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.037+0000 7f105a7fc700 1 -- 172.21.15.178:0/4252927529 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11279 (secure 0 0 0) 0x7f1060099be0 con 0x7f1064100ea0 2023-10-16T14:31:44.040 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:44.041 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24515,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24520,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24550,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:31:41.570556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24593},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-10-16T14:31:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 msgr2=0x7f10500a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 0x7f10500a3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1054000c00 tx=0x7f1054005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 msgr2=0x7f1064198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f106000e970 tx=0x7f106000ed30 comp rx=0 tx=0).stop 2023-10-16T14:31:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 shutdown_connections 2023-10-16T14:31:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10500a15a0 0x7f10500a3a60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1064102a40 0x7f10641a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1064100ea0 0x7f1064198d60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 --2- 172.21.15.178:0/4252927529 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1064102080 0x7f10641992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 >> 172.21.15.178:0/4252927529 conn(0x7f10640fc420 msgr2=0x7f10640fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 shutdown_connections 2023-10-16T14:31:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.040+0000 7f106b7ff700 1 -- 172.21.15.178:0/4252927529 wait complete. 2023-10-16T14:31:44.045 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 17 2023-10-16T14:31:44.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:44.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 -- 172.21.15.178:0/3870673005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4074ef0 msgr2=0x7f36c4073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 --2- 172.21.15.178:0/3870673005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4074ef0 0x7f36c4073340 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009990 tx=0x7f36b8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 -- 172.21.15.178:0/3870673005 shutdown_connections 2023-10-16T14:31:44.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 --2- 172.21.15.178:0/3870673005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 0x7f36c410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 --2- 172.21.15.178:0/3870673005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4074ef0 0x7f36c4073340 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 --2- 172.21.15.178:0/3870673005 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4073910 0x7f36c4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 -- 172.21.15.178:0/3870673005 >> 172.21.15.178:0/3870673005 conn(0x7f36c40fc420 msgr2=0x7f36c40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 -- 172.21.15.178:0/3870673005 shutdown_connections 2023-10-16T14:31:44.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.258+0000 7f36cabff700 1 -- 172.21.15.178:0/3870673005 wait complete. 2023-10-16T14:31:44.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.259+0000 7f36cabff700 1 Processor -- start 2023-10-16T14:31:44.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.259+0000 7f36cabff700 1 -- start start 2023-10-16T14:31:44.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4074ef0 0x7f36c419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 0x7f36c41a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c41a4cb0 con 0x7f36c4073910 2023-10-16T14:31:44.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c41a4df0 con 0x7f36c4074ef0 2023-10-16T14:31:44.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36cabff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c41a4f80 con 0x7f36c410d4c0 2023-10-16T14:31:44.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52322/0 (socket says 172.21.15.178:52322) 2023-10-16T14:31:44.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 -- 172.21.15.178:0/3839328156 learned_addr learned my addr 172.21.15.178:0/3839328156 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c3fff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4074ef0 0x7f36c419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c919c700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 0x7f36c41a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 -- 172.21.15.178:0/3839328156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 msgr2=0x7f36c41a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 0x7f36c41a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 -- 172.21.15.178:0/3839328156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4074ef0 msgr2=0x7f36c419d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4074ef0 0x7f36c419d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.260+0000 7f36c899b700 1 -- 172.21.15.178:0/3839328156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b80095f0 con 0x7f36c4073910 2023-10-16T14:31:44.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36c899b700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f36b8004740 tx=0x7f36b8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b801d070 con 0x7f36c4073910 2023-10-16T14:31:44.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36b8004cf0 con 0x7f36c4073910 2023-10-16T14:31:44.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c41a5200 con 0x7f36c4073910 2023-10-16T14:31:44.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80055a0 con 0x7f36c4073910 2023-10-16T14:31:44.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.261+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41a5620 con 0x7f36c4073910 2023-10-16T14:31:44.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.262+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c4103010 con 0x7f36c4073910 2023-10-16T14:31:44.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.264+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f36b800fbc0 con 0x7f36c4073910 2023-10-16T14:31:44.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.266+0000 7f36c1ffb700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 0x7f36b40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.266+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f36b80ce1a0 con 0x7f36c4073910 2023-10-16T14:31:44.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.266+0000 7f36c3fff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 0x7f36b40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.267+0000 7f36c3fff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 0x7f36b40a3a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36c4074c10 tx=0x7f36b0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.270+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f36b805b100 con 0x7f36c4073910 2023-10-16T14:31:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:44 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3756294782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:44 smithi187 ceph-mon[135297]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-16T14:31:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:44 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:31:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:44 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4287439701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:44 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4252927529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.482+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36c4112d30 con 0x7f36c4073910 2023-10-16T14:31:44.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.484+0000 7f36c1ffb700 1 -- 172.21.15.178:0/3839328156 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f36b8097640 con 0x7f36c4073910 2023-10-16T14:31:44.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:44.488 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:44.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 msgr2=0x7f36b40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 0x7f36b40a3a60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f36c4074c10 tx=0x7f36b0009260 comp rx=0 tx=0).stop 2023-10-16T14:31:44.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 msgr2=0x7f36c419d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f36b8004740 tx=0x7f36b8009580 comp rx=0 tx=0).stop 2023-10-16T14:31:44.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 shutdown_connections 2023-10-16T14:31:44.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36b40a15a0 0x7f36b40a3a60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36c410d4c0 0x7f36c41a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c4073910 0x7f36c419d130 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 --2- 172.21.15.178:0/3839328156 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f36c4074ef0 0x7f36c419d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 >> 172.21.15.178:0/3839328156 conn(0x7f36c40fc420 msgr2=0x7f36c40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 shutdown_connections 2023-10-16T14:31:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.486+0000 7f36cabff700 1 -- 172.21.15.178:0/3839328156 wait complete. 2023-10-16T14:31:44.506 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:44 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3756294782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.506 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:44 smithi178 ceph-mon[98495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-16T14:31:44.506 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:44 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:31:44.507 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:44 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4287439701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.507 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:44 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4252927529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T14:31:36.554542+0000', 'modified': '2023-10-16T14:31:41.570556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24593}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6839/2475230127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2475230127}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2475230127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-10-16T14:31:44.509 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T14:31:44.509 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T14:31:44.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:44 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3756294782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:44 smithi186 ceph-mon[134892]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-10-16T14:31:44.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:44 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:31:44.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:44 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4287439701' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:44 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4252927529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:31:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 -- 172.21.15.178:0/2380722731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 msgr2=0x7f45cc102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 --2- 172.21.15.178:0/2380722731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc102500 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 -- 172.21.15.178:0/2380722731 shutdown_connections 2023-10-16T14:31:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 --2- 172.21.15.178:0/2380722731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 0x7f45cc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 --2- 172.21.15.178:0/2380722731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 0x7f45cc1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 --2- 172.21.15.178:0/2380722731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc102500 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 -- 172.21.15.178:0/2380722731 >> 172.21.15.178:0/2380722731 conn(0x7f45cc0fc420 msgr2=0x7f45cc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 -- 172.21.15.178:0/2380722731 shutdown_connections 2023-10-16T14:31:44.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.707+0000 7f45d45ff700 1 -- 172.21.15.178:0/2380722731 wait complete. 2023-10-16T14:31:44.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 Processor -- start 2023-10-16T14:31:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 -- start start 2023-10-16T14:31:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 0x7f45cc196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 0x7f45cc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19e6e0 con 0x7f45cc102080 2023-10-16T14:31:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19e820 con 0x7f45cc100ea0 2023-10-16T14:31:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.708+0000 7f45d45ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45cc19e9b0 con 0x7f45cc102a40 2023-10-16T14:31:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52336/0 (socket says 172.21.15.178:52336) 2023-10-16T14:31:44.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 -- 172.21.15.178:0/3657344559 learned_addr learned my addr 172.21.15.178:0/3657344559 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 0x7f45cc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d239b700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 0x7f45cc196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 -- 172.21.15.178:0/3657344559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 msgr2=0x7f45cc19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 0x7f45cc19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 -- 172.21.15.178:0/3657344559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 msgr2=0x7f45cc196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 0x7f45cc196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 -- 172.21.15.178:0/3657344559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0095f0 con 0x7f45cc102080 2023-10-16T14:31:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.709+0000 7f45c37fe700 1 -- 172.21.15.178:0/3657344559 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc01d070 con 0x7f45cc102080 2023-10-16T14:31:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.710+0000 7f45c37fe700 1 -- 172.21.15.178:0/3657344559 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc022470 con 0x7f45cc102080 2023-10-16T14:31:44.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.710+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45cc19ec30 con 0x7f45cc102080 2023-10-16T14:31:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.710+0000 7f45c37fe700 1 -- 172.21.15.178:0/3657344559 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc017650 con 0x7f45cc102080 2023-10-16T14:31:44.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.710+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc19f0b0 con 0x7f45cc102080 2023-10-16T14:31:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.713+0000 7f45c37fe700 1 -- 172.21.15.178:0/3657344559 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f45bc0225e0 con 0x7f45cc102080 2023-10-16T14:31:44.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 Processor -- start 2023-10-16T14:31:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45c37fe700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 0x7f45b80a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 -- start start 2023-10-16T14:31:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45c37fe700 1 -- 172.21.15.178:0/3657344559 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45bc0ce140 con 0x7f45cc102080 2023-10-16T14:31:44.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 0x7f45b80a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0bee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c02dc20 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c012c10 con 0x55ae6c0bdb40 2023-10-16T14:31:44.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.715+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c010530 con 0x55ae6c0be9e0 2023-10-16T14:31:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52338/0 (socket says 172.21.15.178:52338) 2023-10-16T14:31:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d2b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0bee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d239b700 1 -- 172.21.15.178:0/889361833 learned_addr learned my addr 172.21.15.178:0/889361833 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 0x7f45b80a3a10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45c400f590 tx=0x7f45c400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 -- 172.21.15.178:0/889361833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 msgr2=0x55ae6c0bee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0bee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 -- 172.21.15.178:0/889361833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 msgr2=0x55ae6c0bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 -- 172.21.15.178:0/889361833 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6bfc93a0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d239b700 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0be400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45bc022890 tx=0x7f45bc009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a44c02fe63ca55 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45c0bfe700 1 -- 172.21.15.178:0/889361833 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0a1040 con 0x55ae6c0bdb40 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45c0bfe700 1 -- 172.21.15.178:0/889361833 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc09b2a0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.716+0000 7f45c0bfe700 1 -- 172.21.15.178:0/889361833 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc09a790 con 0x55ae6c0bdb40 2023-10-16T14:31:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 -- 172.21.15.178:0/889361833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 msgr2=0x55ae6c0be400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0be400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f45bc022890 tx=0x7f45bc009310 comp rx=0 tx=0).stop 2023-10-16T14:31:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 -- 172.21.15.178:0/889361833 shutdown_connections 2023-10-16T14:31:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0bee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/889361833 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0be400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 -- 172.21.15.178:0/889361833 >> 172.21.15.178:0/889361833 conn(0x55ae6bfe6d80 msgr2=0x55ae6c0b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 -- 172.21.15.178:0/889361833 shutdown_connections 2023-10-16T14:31:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.717+0000 7f45e7f25b80 1 -- 172.21.15.178:0/889361833 wait complete. 2023-10-16T14:31:44.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 -- start start 2023-10-16T14:31:44.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 Processor -- start 2023-10-16T14:31:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c00be20 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6bfa86c0 con 0x55ae6c0be9e0 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.718+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6bfa8360 con 0x55ae6c0bdb40 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45e7f25b80 10 client.0 _ll_register_callbacks cb 0x55ae6be057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:31:44.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52344/0 (socket says 172.21.15.178:52344) 2023-10-16T14:31:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d1b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 -- 172.21.15.178:0/1752082772 learned_addr learned my addr 172.21.15.178:0/1752082772 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 -- 172.21.15.178:0/1752082772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 msgr2=0x55ae6c0c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 -- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0be9e0 msgr2=0x55ae6c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 -- 172.21.15.178:0/1752082772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c8009960 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45d239b700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45c8012960 tx=0x7f45c8012c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c637fcadf299984 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:31:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.719+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8017070 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45c800f580 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae6c010ea0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45c800f740 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 10 client.14661 fetch_fsmap learned FSMap version 17 2023-10-16T14:31:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f45cc19f0b0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ae6c010ea0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f45c8010920 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 10 client.14661 fetch_fsmap finished waiting for FSMap version 17 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 10 client.14661 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 20 client.14661 populate_metadata read hostname 'smithi178' 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c0d2a10 0x55ae6c0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 4 client.14661 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:31:44.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.720+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae6c088bc0 con 0x55ae6c0d2a10 2023-10-16T14:31:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.721+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c0d2a10 0x55ae6c0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.721+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c0d2a10 0x55ae6c0d1470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e53507dc646d3b7 server_cookie=d142212bd41a81db in_seq=0 out_seq=0 2023-10-16T14:31:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.721+0000 7f45b5ffb700 10 client.14661 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:31:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1752082772 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ae6c088bc0 con 0x55ae6c0d2a10 2023-10-16T14:31:44.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45b5ffb700 10 client.14661 handle_command_reply: tid=0 2023-10-16T14:31:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 client.14661 shutdown 2023-10-16T14:31:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 msgr2=0x55ae6c0c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f45c8012960 tx=0x7f45c8012c70 comp rx=0 tx=0).stop 2023-10-16T14:31:44.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 shutdown_connections 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c0d2a10 0x55ae6c0d1470 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c6d90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1752082772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45b5ffb700 0 client.14661 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:31:44.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 >> 172.21.15.178:0/1752082772 conn(0x55ae6bfe6d80 msgr2=0x55ae6c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.741 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 shutdown_connections 2023-10-16T14:31:44.741 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.722+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1752082772 wait complete. 2023-10-16T14:31:44.741 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.723+0000 7f45e7f25b80 20 client.14661 trim_cache size 0 max 16384 2023-10-16T14:31:44.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.740+0000 7f45e7f25b80 1 Processor -- start 2023-10-16T14:31:44.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 -- start start 2023-10-16T14:31:44.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6be8c8d0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6bfc3720 con 0x55ae6c0bdb40 2023-10-16T14:31:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c0bcde0 con 0x55ae6c0be9e0 2023-10-16T14:31:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52346/0 (socket says 172.21.15.178:52346) 2023-10-16T14:31:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 -- 172.21.15.178:0/247706039 learned_addr learned my addr 172.21.15.178:0/247706039 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 -- 172.21.15.178:0/247706039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 msgr2=0x55ae6c0c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 -- 172.21.15.178:0/247706039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 msgr2=0x55ae6c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d239b700 1 -- 172.21.15.178:0/247706039 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae6be36d90 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.741+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45d239b700 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097c0 tx=0x7f45c8009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9e44c382f1b315 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45c13ff700 1 -- 172.21.15.178:0/247706039 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8017040 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45c13ff700 1 -- 172.21.15.178:0/247706039 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45c80046d0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45c13ff700 1 -- 172.21.15.178:0/247706039 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c80052a0 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 -- 172.21.15.178:0/247706039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 msgr2=0x55ae6c0c1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f45c80097c0 tx=0x7f45c8009400 comp rx=0 tx=0).stop 2023-10-16T14:31:44.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 -- 172.21.15.178:0/247706039 shutdown_connections 2023-10-16T14:31:44.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0c1980 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/247706039 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.742+0000 7f45e7f25b80 1 -- 172.21.15.178:0/247706039 >> 172.21.15.178:0/247706039 conn(0x55ae6bfe6d80 msgr2=0x55ae6c0c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.743+0000 7f45e7f25b80 1 -- 172.21.15.178:0/247706039 shutdown_connections 2023-10-16T14:31:44.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.743+0000 7f45e7f25b80 1 -- 172.21.15.178:0/247706039 wait complete. 2023-10-16T14:31:44.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.743+0000 7f45e7f25b80 1 -- start start 2023-10-16T14:31:44.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.743+0000 7f45e7f25b80 1 Processor -- start 2023-10-16T14:31:44.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c0cb4d0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c0cb640 con 0x55ae6c0bd1e0 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae6c126400 con 0x55ae6c0be9e0 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45e7f25b80 10 client.0 _ll_register_callbacks cb 0x7f45c8012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45d239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45d1b9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45d239b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:54994/0 (socket says 172.21.15.178:54994) 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45d239b700 1 -- 172.21.15.178:0/1830079648 learned_addr learned my addr 172.21.15.178:0/1830079648 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.744+0000 7f45d2b9c700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 -- 172.21.15.178:0/1830079648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 msgr2=0x55ae6c0caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 -- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 msgr2=0x55ae6c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:31:44.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 -- 172.21.15.178:0/1830079648 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45cc19f0b0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d239b700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:44.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45d1b9a700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0caa50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ae6c0c5770 tx=0x7f45bc09af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6e1ff99235e09 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:44.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:31:44.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc0a1040 con 0x55ae6c0bdb40 2023-10-16T14:31:44.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f45bc09f3f0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45bc09b2a0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc0a3620 con 0x55ae6c0bdb40 2023-10-16T14:31:44.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45e7f25b80 10 client.14667 fetch_fsmap learned FSMap version 17 2023-10-16T14:31:44.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55ae6be1fc10 con 0x55ae6c0bdb40 2023-10-16T14:31:44.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45bc09b2a0 con 0x55ae6c0bdb40 2023-10-16T14:31:44.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.745+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f45bc0a3110 con 0x55ae6c0bdb40 2023-10-16T14:31:44.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 10 client.14667 fetch_fsmap finished waiting for FSMap version 17 2023-10-16T14:31:44.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 10 client.14667 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:31:44.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 20 client.14667 populate_metadata read hostname 'smithi178' 2023-10-16T14:31:44.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c12b310 0x55ae6c127770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:44.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 4 client.14667 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:31:44.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae6c004e00 con 0x55ae6c12b310 2023-10-16T14:31:44.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45d239b700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c12b310 0x55ae6c127770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:44.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.746+0000 7f45d239b700 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c12b310 0x55ae6c127770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c223a220da46d685 server_cookie=877fee89a20da6c7 in_seq=0 out_seq=0 2023-10-16T14:31:44.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45b5ffb700 10 client.14667 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:31:44.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45b5ffb700 1 -- 172.21.15.178:0/1830079648 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ae6c004e00 con 0x55ae6c12b310 2023-10-16T14:31:44.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45b5ffb700 10 client.14667 handle_command_reply: tid=0 2023-10-16T14:31:44.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 client.14667 shutdown 2023-10-16T14:31:44.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 msgr2=0x55ae6c0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0caa50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55ae6c0c5770 tx=0x7f45bc09af10 comp rx=0 tx=0).stop 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 shutdown_connections 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ae6c12b310 0x55ae6c127770 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45b5ffb700 0 client.14667 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ae6c0be9e0 0x55ae6c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ae6c0bd1e0 0x55ae6c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 --2- 172.21.15.178:0/1830079648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ae6c0bdb40 0x55ae6c0caa50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 >> 172.21.15.178:0/1830079648 conn(0x55ae6bfe6d80 msgr2=0x55ae6c0c1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 shutdown_connections 2023-10-16T14:31:44.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 1 -- 172.21.15.178:0/1830079648 wait complete. 2023-10-16T14:31:44.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.747+0000 7f45e7f25b80 20 client.14667 trim_cache size 0 max 16384 2023-10-16T14:31:44.765 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:31:44.765 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:31:44.765 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 0, 2023-10-16T14:31:44.765 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24593, 2023-10-16T14:31:44.766 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:31:44.766 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:31:44.766 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:31:44.766 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 7.6952522459999999, 2023-10-16T14:31:44.766 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 17, 2023-10-16T14:31:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 87, 2023-10-16T14:31:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 87, 2023-10-16T14:31:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 13.841089197000001 2023-10-16T14:31:44.767 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:31:44.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 msgr2=0x7f45b80a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 0x7f45b80a3a10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45c400f590 tx=0x7f45c400b040 comp rx=0 tx=0).stop 2023-10-16T14:31:44.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 msgr2=0x7f45cc1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:44.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f45bc000c00 tx=0x7f45bc005720 comp rx=0 tx=0).stop 2023-10-16T14:31:44.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 shutdown_connections 2023-10-16T14:31:44.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f45b80a1550 0x7f45b80a3a10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45cc102a40 0x7f45cc19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f45cc100ea0 0x7f45cc196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 --2- 172.21.15.178:0/3657344559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45cc102080 0x7f45cc1970a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:44.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 >> 172.21.15.178:0/3657344559 conn(0x7f45cc0fc420 msgr2=0x7f45cc1055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:44.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 shutdown_connections 2023-10-16T14:31:44.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:44.748+0000 7f45d45ff700 1 -- 172.21.15.178:0/3657344559 wait complete. 2023-10-16T14:31:44.771 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T14:31:44.772 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:44.773 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-16T14:31:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3839328156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:45.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:45 smithi187 ceph-mon[135297]: pgmap v276: 129 pgs: 4 creating+peering, 114 active+clean, 11 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2023-10-16T14:31:45.506 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:31:45.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3839328156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:45.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:45 smithi178 ceph-mon[98495]: pgmap v276: 129 pgs: 4 creating+peering, 114 active+clean, 11 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2023-10-16T14:31:45.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3839328156' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:45.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:45 smithi186 ceph-mon[134892]: pgmap v276: 129 pgs: 4 creating+peering, 114 active+clean, 11 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 741 B/s wr, 6 op/s 2023-10-16T14:31:47.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 -- 172.21.15.178:0/1069162658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106f90 msgr2=0x7f2d14107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:47.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/1069162658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106f90 0x7f2d14107370 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d040099c0 tx=0x7f2d04009cd0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 -- 172.21.15.178:0/1069162658 shutdown_connections 2023-10-16T14:31:47.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/1069162658 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14101940 0x7f2d1410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/1069162658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14106f90 0x7f2d14107370 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/1069162658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14100f80 0x7f2d14101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 -- 172.21.15.178:0/1069162658 >> 172.21.15.178:0/1069162658 conn(0x7f2d140780a0 msgr2=0x7f2d140784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:47.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 -- 172.21.15.178:0/1069162658 shutdown_connections 2023-10-16T14:31:47.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.164+0000 7f2d1a26b700 1 -- 172.21.15.178:0/1069162658 wait complete. 2023-10-16T14:31:47.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.165+0000 7f2d1a26b700 1 Processor -- start 2023-10-16T14:31:47.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.165+0000 7f2d1a26b700 1 -- start start 2023-10-16T14:31:47.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:47.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 0x7f2d14197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14106f90 0x7f2d1419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1419bfa0 con 0x7f2d14101940 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1419c110 con 0x7f2d14100f80 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d1a26b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1419c2d0 con 0x7f2d14106f90 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d137fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 0x7f2d14197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:47.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:47.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55012/0 (socket says 172.21.15.178:55012) 2023-10-16T14:31:47.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 -- 172.21.15.178:0/784539291 learned_addr learned my addr 172.21.15.178:0/784539291 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:47.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d18808700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14106f90 0x7f2d1419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:47.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 -- 172.21.15.178:0/784539291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14106f90 msgr2=0x7f2d1419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:47.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14106f90 0x7f2d1419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 -- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 msgr2=0x7f2d14197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:47.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 0x7f2d14197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d13fff700 1 -- 172.21.15.178:0/784539291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d040095f0 con 0x7f2d14100f80 2023-10-16T14:31:47.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.166+0000 7f2d137fe700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 0x7f2d14197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:47.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d13fff700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b580 tx=0x7f2d0400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0401d070 con 0x7f2d14100f80 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d04005050 con 0x7f2d14100f80 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1419c550 con 0x7f2d14100f80 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004a40 con 0x7f2d14100f80 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.167+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d141a3e10 con 0x7f2d14100f80 2023-10-16T14:31:47.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.168+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1404ebb0 con 0x7f2d14100f80 2023-10-16T14:31:47.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.170+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2d04005430 con 0x7f2d14100f80 2023-10-16T14:31:47.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.171+0000 7f2d117fa700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 0x7f2cfc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:47.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.172+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2d040cdd00 con 0x7f2d14100f80 2023-10-16T14:31:47.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.172+0000 7f2d137fe700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 0x7f2cfc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:47.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.172+0000 7f2d137fe700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 0x7f2cfc0a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d14198600 tx=0x7f2d0800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:47.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.176+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d040971a0 con 0x7f2d14100f80 2023-10-16T14:31:47.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.417+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d14110ea0 con 0x7f2d14100f80 2023-10-16T14:31:47.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.417+0000 7f2d117fa700 1 -- 172.21.15.178:0/784539291 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2d0409a7c0 con 0x7f2d14100f80 2023-10-16T14:31:47.420 INFO:teuthology.orchestra.run.smithi178.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T14:31:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 msgr2=0x7f2cfc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 0x7f2cfc0a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2d14198600 tx=0x7f2d0800b410 comp rx=0 tx=0).stop 2023-10-16T14:31:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 msgr2=0x7f2d14197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:47.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b580 tx=0x7f2d0400f690 comp rx=0 tx=0).stop 2023-10-16T14:31:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 shutdown_connections 2023-10-16T14:31:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2cfc0a1480 0x7f2cfc0a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d14106f90 0x7f2d1419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d14100f80 0x7f2d14197030 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 --2- 172.21.15.178:0/784539291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d14101940 0x7f2d14197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 >> 172.21.15.178:0/784539291 conn(0x7f2d140780a0 msgr2=0x7f2d140ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 shutdown_connections 2023-10-16T14:31:47.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:47.420+0000 7f2d1a26b700 1 -- 172.21.15.178:0/784539291 wait complete. 2023-10-16T14:31:47.696 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:47 smithi178 ceph-mon[98495]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 1 op/s 2023-10-16T14:31:47.697 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:47 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/784539291' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:31:47.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:47 smithi187 ceph-mon[135297]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 1 op/s 2023-10-16T14:31:47.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:47 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/784539291' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:31:47.986 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-16T14:31:47.986 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T14:31:47.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2023-10-16T14:31:47.987 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-10-16T14:31:47.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:47.987 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-16T14:31:47.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T14:31:47.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T14:31:47.988 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T14:31:47.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:47.989 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2023-10-16T14:31:48.033 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T14:31:48.034 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T14:31:48.034 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T14:31:48.034 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2023-10-16T14:31:48.035 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:31:48.036 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:31:48.036 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2023-10-16T14:31:48.036 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:31:48.036 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:31:48.036 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T14:31:48.037 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:31:48.037 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T14:31:48.037 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 41759sec preferred_lft 41759sec 2023-10-16T14:31:48.037 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::92fb:5c5a:625:714e/64 scope link noprefixroute 2023-10-16T14:31:48.037 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:31:48.038 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T14:31:48.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:48.039 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:31:48.040 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:31:48.040 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2023-10-16T14:31:48.040 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2023-10-16T14:31:48.041 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2023-10-16T14:31:48.041 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T14:31:48.041 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:31:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:47 smithi186 ceph-mon[134892]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 1 op/s 2023-10-16T14:31:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:47 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/784539291' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:31:48.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:48.516 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T14:31:48.586 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-10-16T14:31:48.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:48.591 DEBUG:teuthology.orchestra.run.smithi178:> route 2023-10-16T14:31:48.650 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2023-10-16T14:31:48.651 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T14:31:48.651 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T14:31:48.651 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T14:31:48.651 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T14:31:48.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:48.653 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:31:48.653 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:31:48.654 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T14:31:48.654 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T14:31:48.654 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T14:31:48.655 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:31:49.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:49.369 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-10-16T14:31:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:49.414 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2023-10-16T14:31:49.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:49.498 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:31:49.499 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:31:49.499 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:31:49.499 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-10-16T14:31:49.500 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:31:49.683 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-10-16T14:31:49.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:49.684 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:31:49.684 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:31:49.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-10-16T14:31:49.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T14:31:49.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-10-16T14:31:49.686 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-10-16T14:31:49.686 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-10-16T14:31:49.686 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:31:49.889 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:49 smithi178 ceph-mon[98495]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-10-16T14:31:49.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:49 smithi187 ceph-mon[135297]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-10-16T14:31:49.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:49.990 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:31:49.990 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:31:49.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2023-10-16T14:31:49.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T14:31:49.991 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:31:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:49 smithi186 ceph-mon[134892]: pgmap v278: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2023-10-16T14:31:50.164 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-10-16T14:31:50.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T14:31:50.165 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:50.209 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-10-16T14:31:50.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T14:31:50.210 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:50.292 DEBUG:teuthology.orchestra.run.smithi178:> sudo modprobe fuse 2023-10-16T14:31:50.401 DEBUG:teuthology.orchestra.run.smithi178:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T14:31:50.446 INFO:teuthology.orchestra.run.smithi178.stdout:/sys 2023-10-16T14:31:50.446 INFO:teuthology.orchestra.run.smithi178.stdout:/proc 2023-10-16T14:31:50.446 INFO:teuthology.orchestra.run.smithi178.stdout:/dev 2023-10-16T14:31:50.446 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/security 2023-10-16T14:31:50.447 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/shm 2023-10-16T14:31:50.447 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/pts 2023-10-16T14:31:50.447 INFO:teuthology.orchestra.run.smithi178.stdout:/run 2023-10-16T14:31:50.447 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup 2023-10-16T14:31:50.447 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/systemd 2023-10-16T14:31:50.448 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/pstore 2023-10-16T14:31:50.448 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-16T14:31:50.448 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/devices 2023-10-16T14:31:50.448 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-16T14:31:50.448 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/cpuset 2023-10-16T14:31:50.449 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/freezer 2023-10-16T14:31:50.449 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/pids 2023-10-16T14:31:50.449 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/perf_event 2023-10-16T14:31:50.449 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/hugetlb 2023-10-16T14:31:50.449 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/blkio 2023-10-16T14:31:50.450 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/memory 2023-10-16T14:31:50.450 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/tracing 2023-10-16T14:31:50.450 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/config 2023-10-16T14:31:50.450 INFO:teuthology.orchestra.run.smithi178.stdout:/ 2023-10-16T14:31:50.450 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/selinux 2023-10-16T14:31:50.451 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/mqueue 2023-10-16T14:31:50.451 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/debug 2023-10-16T14:31:50.451 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T14:31:50.451 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/hugepages 2023-10-16T14:31:50.451 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/fuse/connections 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/nfs/rpc_pipefs 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/1000 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/ceph 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c349f995cf4d8a1f4f026e9bd0be70033f397a264cce7788f74d0976bc19b854/merged 2023-10-16T14:31:50.452 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/cc66a8e9bf3f4f8566710fcc0d4eaa140067152f9fa41ea1e3440533e2771ac8/merged 2023-10-16T14:31:50.453 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/0 2023-10-16T14:31:50.453 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T14:31:50.453 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/201374a8fede183aba3f846de2fcd1b62826df48458b3f7290d86ce79a733fb0/merged 2023-10-16T14:31:50.453 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/a8119eca76a8d78ab2fceb3c5c4b2b729b9ae2cff1fd77afa8755486e681eef1/merged 2023-10-16T14:31:50.453 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c15b4e19cbd9162c444ce11ef9964e338f20cc38a27d0173891cfa8bbdc9d25d/merged 2023-10-16T14:31:50.454 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c43aac3828067c8e145ff6b365a92c8f71815e7d091d26bf62672d994e57b94c/merged 2023-10-16T14:31:50.454 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/26dc8dae4056abed73b308f2975f639e76a482e2bc95eb1adfbfd2e72983b075/merged 2023-10-16T14:31:50.454 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/1b0665d5469466142301d1a9d39fac95acd758c32cf83378420665bf90543ceb/merged 2023-10-16T14:31:50.454 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/fd215243147bbebc87b425b857be4bddace79b4dab1e5b7dd13e071ecb8ac95b/merged 2023-10-16T14:31:50.454 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/6892876b869dfc27dbf949837a8624554834e4b099986fad3a8782a58d6945bd/merged 2023-10-16T14:31:50.455 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns 2023-10-16T14:31:50.455 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:31:50.455 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:31:50.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:50.456 DEBUG:teuthology.orchestra.run.smithi178:> ls /sys/fs/fuse/connections 2023-10-16T14:31:50.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T14:31:50.540 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-10-16T14:31:50.585 DEBUG:teuthology.orchestra.run.smithi178:> sudo modprobe fuse 2023-10-16T14:31:50.655 DEBUG:teuthology.orchestra.run.smithi178:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T14:31:50.703 INFO:teuthology.orchestra.run.smithi178.stdout:/sys 2023-10-16T14:31:51.032 INFO:teuthology.orchestra.run.smithi178.stdout:/proc 2023-10-16T14:31:51.033 INFO:teuthology.orchestra.run.smithi178.stdout:/dev 2023-10-16T14:31:51.033 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/security 2023-10-16T14:31:51.033 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/shm 2023-10-16T14:31:51.033 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/pts 2023-10-16T14:31:51.033 INFO:teuthology.orchestra.run.smithi178.stdout:/run 2023-10-16T14:31:51.034 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup 2023-10-16T14:31:51.034 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/systemd 2023-10-16T14:31:51.034 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/pstore 2023-10-16T14:31:51.034 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-10-16T14:31:51.034 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/devices 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/cpuset 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/freezer 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/pids 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/perf_event 2023-10-16T14:31:51.035 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/hugetlb 2023-10-16T14:31:51.036 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/blkio 2023-10-16T14:31:51.036 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup/memory 2023-10-16T14:31:51.036 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/tracing 2023-10-16T14:31:51.036 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/config 2023-10-16T14:31:51.036 INFO:teuthology.orchestra.run.smithi178.stdout:/ 2023-10-16T14:31:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/selinux 2023-10-16T14:31:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/mqueue 2023-10-16T14:31:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/debug 2023-10-16T14:31:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T14:31:51.037 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/hugepages 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/fuse/connections 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/nfs/rpc_pipefs 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/1000 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/ceph 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay 2023-10-16T14:31:51.038 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c349f995cf4d8a1f4f026e9bd0be70033f397a264cce7788f74d0976bc19b854/merged 2023-10-16T14:31:51.039 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/cc66a8e9bf3f4f8566710fcc0d4eaa140067152f9fa41ea1e3440533e2771ac8/merged 2023-10-16T14:31:51.040 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/0 2023-10-16T14:31:51.040 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T14:31:51.041 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/201374a8fede183aba3f846de2fcd1b62826df48458b3f7290d86ce79a733fb0/merged 2023-10-16T14:31:51.041 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/a8119eca76a8d78ab2fceb3c5c4b2b729b9ae2cff1fd77afa8755486e681eef1/merged 2023-10-16T14:31:51.041 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c15b4e19cbd9162c444ce11ef9964e338f20cc38a27d0173891cfa8bbdc9d25d/merged 2023-10-16T14:31:51.042 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c43aac3828067c8e145ff6b365a92c8f71815e7d091d26bf62672d994e57b94c/merged 2023-10-16T14:31:51.042 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/26dc8dae4056abed73b308f2975f639e76a482e2bc95eb1adfbfd2e72983b075/merged 2023-10-16T14:31:51.042 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/1b0665d5469466142301d1a9d39fac95acd758c32cf83378420665bf90543ceb/merged 2023-10-16T14:31:51.042 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/fd215243147bbebc87b425b857be4bddace79b4dab1e5b7dd13e071ecb8ac95b/merged 2023-10-16T14:31:51.043 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/6892876b869dfc27dbf949837a8624554834e4b099986fad3a8782a58d6945bd/merged 2023-10-16T14:31:51.043 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns 2023-10-16T14:31:51.043 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:31:51.044 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:31:51.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:51.046 DEBUG:teuthology.orchestra.run.smithi178:> ls /sys/fs/fuse/connections 2023-10-16T14:31:51.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi178.stderr:2023-10-16T14:31:50.778+0000 7f8fcc7795c0 -1 init, newargv = 0x5568aa366070 newargc=15 2023-10-16T14:31:51.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi178.stderr:ceph-fuse[152070]: starting ceph client 2023-10-16T14:31:51.047 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi178.stderr:ceph-fuse[152070]: starting fuse 2023-10-16T14:31:51.132 INFO:teuthology.orchestra.run.smithi178.stdout:121 2023-10-16T14:31:51.132 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-10-16T14:31:51.133 DEBUG:teuthology.orchestra.run.smithi178:> sudo stdin-killer -- python3 -c ' 2023-10-16T14:31:51.133 DEBUG:teuthology.orchestra.run.smithi178:> import glob 2023-10-16T14:31:51.134 DEBUG:teuthology.orchestra.run.smithi178:> import re 2023-10-16T14:31:51.134 DEBUG:teuthology.orchestra.run.smithi178:> import os 2023-10-16T14:31:51.134 DEBUG:teuthology.orchestra.run.smithi178:> import subprocess 2023-10-16T14:31:51.135 DEBUG:teuthology.orchestra.run.smithi178:> 2023-10-16T14:31:51.135 DEBUG:teuthology.orchestra.run.smithi178:> def _find_admin_socket(client_name): 2023-10-16T14:31:51.135 DEBUG:teuthology.orchestra.run.smithi178:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-10-16T14:31:51.136 DEBUG:teuthology.orchestra.run.smithi178:> files = glob.glob(asok_path) 2023-10-16T14:31:51.136 DEBUG:teuthology.orchestra.run.smithi178:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-10-16T14:31:51.136 DEBUG:teuthology.orchestra.run.smithi178:> 2023-10-16T14:31:51.137 DEBUG:teuthology.orchestra.run.smithi178:> # Given a non-glob path, it better be there 2023-10-16T14:31:51.137 DEBUG:teuthology.orchestra.run.smithi178:> if "*" not in asok_path: 2023-10-16T14:31:51.137 DEBUG:teuthology.orchestra.run.smithi178:> assert(len(files) == 1) 2023-10-16T14:31:51.138 DEBUG:teuthology.orchestra.run.smithi178:> return files[0] 2023-10-16T14:31:51.138 DEBUG:teuthology.orchestra.run.smithi178:> 2023-10-16T14:31:51.138 DEBUG:teuthology.orchestra.run.smithi178:> for f in files: 2023-10-16T14:31:51.139 DEBUG:teuthology.orchestra.run.smithi178:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T14:31:51.139 DEBUG:teuthology.orchestra.run.smithi178:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T14:31:51.139 DEBUG:teuthology.orchestra.run.smithi178:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T14:31:51.140 DEBUG:teuthology.orchestra.run.smithi178:> contents = proc_f.read() 2023-10-16T14:31:51.140 DEBUG:teuthology.orchestra.run.smithi178:> if mountpoint in contents: 2023-10-16T14:31:51.141 DEBUG:teuthology.orchestra.run.smithi178:> return f 2023-10-16T14:31:51.141 DEBUG:teuthology.orchestra.run.smithi178:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T14:31:51.141 DEBUG:teuthology.orchestra.run.smithi178:> 2023-10-16T14:31:51.142 DEBUG:teuthology.orchestra.run.smithi178:> print(_find_admin_socket("client.admin")) 2023-10-16T14:31:51.142 DEBUG:teuthology.orchestra.run.smithi178:> ' 2023-10-16T14:31:51.317 INFO:teuthology.orchestra.run.smithi178.stdout:/var/run/ceph/ceph-client.admin.152070.asok 2023-10-16T14:31:51.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T14:31:51.331 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.152070.asok 2023-10-16T14:31:51.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:51.332 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.152070.asok status 2023-10-16T14:31:51.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:51 smithi178 ceph-mon[98495]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 107 B/s wr, 0 op/s 2023-10-16T14:31:51.560 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:31:51.561 INFO:teuthology.orchestra.run.smithi178.stdout: "metadata": { 2023-10-16T14:31:51.561 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_sha1": "4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0", 2023-10-16T14:31:51.561 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 18.0.0-6730-g4ba8a190 (4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0) reef (dev)", 2023-10-16T14:31:51.561 INFO:teuthology.orchestra.run.smithi178.stdout: "entity_id": "admin", 2023-10-16T14:31:51.562 INFO:teuthology.orchestra.run.smithi178.stdout: "hostname": "smithi178", 2023-10-16T14:31:51.562 INFO:teuthology.orchestra.run.smithi178.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-10-16T14:31:51.562 INFO:teuthology.orchestra.run.smithi178.stdout: "pid": "152070", 2023-10-16T14:31:51.562 INFO:teuthology.orchestra.run.smithi178.stdout: "root": "/" 2023-10-16T14:31:51.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:31:51.563 INFO:teuthology.orchestra.run.smithi178.stdout: "dentry_count": 0, 2023-10-16T14:31:51.563 INFO:teuthology.orchestra.run.smithi178.stdout: "dentry_pinned_count": 0, 2023-10-16T14:31:51.563 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24703, 2023-10-16T14:31:51.563 INFO:teuthology.orchestra.run.smithi178.stdout: "inst": { 2023-10-16T14:31:51.563 INFO:teuthology.orchestra.run.smithi178.stdout: "name": { 2023-10-16T14:31:51.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "client", 2023-10-16T14:31:51.564 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 24703 2023-10-16T14:31:51.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:31:51.564 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": { 2023-10-16T14:31:51.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2023-10-16T14:31:51.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:31:51.565 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 2890693732 2023-10-16T14:31:51.565 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:31:51.565 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:31:51.565 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": { 2023-10-16T14:31:51.566 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2023-10-16T14:31:51.566 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:31:51.566 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 2890693732 2023-10-16T14:31:51.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:31:51.567 INFO:teuthology.orchestra.run.smithi178.stdout: "inst_str": "client.24703 172.21.15.178:0/2890693732", 2023-10-16T14:31:51.567 INFO:teuthology.orchestra.run.smithi178.stdout: "addr_str": "172.21.15.178:0/2890693732", 2023-10-16T14:31:51.567 INFO:teuthology.orchestra.run.smithi178.stdout: "inode_count": 1, 2023-10-16T14:31:51.567 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_epoch": 17, 2023-10-16T14:31:51.567 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_epoch": 87, 2023-10-16T14:31:51.568 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_epoch_barrier": 0, 2023-10-16T14:31:51.568 INFO:teuthology.orchestra.run.smithi178.stdout: "blocklisted": false, 2023-10-16T14:31:51.568 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs" 2023-10-16T14:31:51.568 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:31:51.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:51.576 DEBUG:teuthology.orchestra.run.smithi178:> stat --file-system '--printf=%T 2023-10-16T14:31:51.577 DEBUG:teuthology.orchestra.run.smithi178:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:51.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:51 smithi186 ceph-mon[134892]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 107 B/s wr, 0 op/s 2023-10-16T14:31:51.622 INFO:teuthology.orchestra.run.smithi178.stdout:fuseblk 2023-10-16T14:31:51.623 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:51.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:51.623 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:51 smithi187 ceph-mon[135297]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 107 B/s wr, 0 op/s 2023-10-16T14:31:51.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:51.735 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-10-16T14:31:51.867 INFO:teuthology.orchestra.run.smithi178.stdout:# file: . 2023-10-16T14:31:51.867 INFO:teuthology.orchestra.run.smithi178.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-10-16T14:31:51.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:51.869 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:31:51.869 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:31:51.914 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:52.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 -- 172.21.15.178:0/1015300945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 msgr2=0x7f94f4102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/1015300945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f4102550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009990 tx=0x7f94e4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 -- 172.21.15.178:0/1015300945 shutdown_connections 2023-10-16T14:31:52.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/1015300945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 0x7f94f41093b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/1015300945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 0x7f94f4101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/1015300945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f4102550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.150+0000 7f94fb5ff700 1 -- 172.21.15.178:0/1015300945 >> 172.21.15.178:0/1015300945 conn(0x7f94f40fc4c0 msgr2=0x7f94f40fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:52.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.151+0000 7f94fb5ff700 1 -- 172.21.15.178:0/1015300945 shutdown_connections 2023-10-16T14:31:52.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.151+0000 7f94fb5ff700 1 -- 172.21.15.178:0/1015300945 wait complete. 2023-10-16T14:31:52.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.151+0000 7f94fb5ff700 1 Processor -- start 2023-10-16T14:31:52.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 -- start start 2023-10-16T14:31:52.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 0x7f94f4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 0x7f94f41a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41a08f0 con 0x7f94f41020d0 2023-10-16T14:31:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41a0a60 con 0x7f94f4100f00 2023-10-16T14:31:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94fb5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41a0c20 con 0x7f94f4102a90 2023-10-16T14:31:52.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94f8b9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94f8b9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52026/0 (socket says 172.21.15.178:52026) 2023-10-16T14:31:52.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94f8b9a700 1 -- 172.21.15.178:0/2395718774 learned_addr learned my addr 172.21.15.178:0/2395718774 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:52.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94f9b9c700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 0x7f94f41a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.152+0000 7f94f939b700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 0x7f94f4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 -- 172.21.15.178:0/2395718774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 msgr2=0x7f94f41a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 0x7f94f41a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 -- 172.21.15.178:0/2395718774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 msgr2=0x7f94f4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 0x7f94f4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 -- 172.21.15.178:0/2395718774 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40095f0 con 0x7f94f41020d0 2023-10-16T14:31:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94f8b9a700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b620 tx=0x7f94e4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:52.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e401d070 con 0x7f94f41020d0 2023-10-16T14:31:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94e4022470 con 0x7f94f41020d0 2023-10-16T14:31:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f41a0ea0 con 0x7f94f41020d0 2023-10-16T14:31:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4017650 con 0x7f94f41020d0 2023-10-16T14:31:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.153+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f41a1210 con 0x7f94f41020d0 2023-10-16T14:31:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.154+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f410c830 con 0x7f94f41020d0 2023-10-16T14:31:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.157+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f94e40225e0 con 0x7f94f41020d0 2023-10-16T14:31:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.159+0000 7f94ea7fc700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 0x7f94e00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.159+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f94e40ceab0 con 0x7f94f41020d0 2023-10-16T14:31:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.159+0000 7f94f939b700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 0x7f94e00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.160+0000 7f94f939b700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 0x7f94e00a39e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94f419a280 tx=0x7f94f0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:52.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.161+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f94e4097fb0 con 0x7f94f41020d0 2023-10-16T14:31:52.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.369+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f41a1400 con 0x7f94f41020d0 2023-10-16T14:31:52.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.371+0000 7f94ea7fc700 1 -- 172.21.15.178:0/2395718774 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f94e409b5d0 con 0x7f94f41020d0 2023-10-16T14:31:52.374 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:52.374 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:52.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.373+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 msgr2=0x7f94e00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.373+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 0x7f94e00a39e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94f419a280 tx=0x7f94f0009260 comp rx=0 tx=0).stop 2023-10-16T14:31:52.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.373+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 msgr2=0x7f94f41992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.373+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b620 tx=0x7f94e4005740 comp rx=0 tx=0).stop 2023-10-16T14:31:52.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 shutdown_connections 2023-10-16T14:31:52.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f94e00a1520 0x7f94e00a39e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4102a90 0x7f94f41a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f94f4100f00 0x7f94f4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 --2- 172.21.15.178:0/2395718774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94f41020d0 0x7f94f41992f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 >> 172.21.15.178:0/2395718774 conn(0x7f94f40fc4c0 msgr2=0x7f94f41079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:52.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 shutdown_connections 2023-10-16T14:31:52.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.374+0000 7f94fb5ff700 1 -- 172.21.15.178:0/2395718774 wait complete. 2023-10-16T14:31:52.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:31:52.396 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:31:52.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 -- 172.21.15.178:0/2252486799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c100d10 msgr2=0x7f9c4c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2252486799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c100d10 0x7f9c4c101130 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9c480099c0 tx=0x7f9c48009cd0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 -- 172.21.15.178:0/2252486799 shutdown_connections 2023-10-16T14:31:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2252486799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2252486799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c100d10 0x7f9c4c101130 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2252486799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c101f00 0x7f9c4c102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 -- 172.21.15.178:0/2252486799 >> 172.21.15.178:0/2252486799 conn(0x7f9c4c0fc280 msgr2=0x7f9c4c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 -- 172.21.15.178:0/2252486799 shutdown_connections 2023-10-16T14:31:52.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.595+0000 7f9c5239d700 1 -- 172.21.15.178:0/2252486799 wait complete. 2023-10-16T14:31:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.596+0000 7f9c5239d700 1 Processor -- start 2023-10-16T14:31:52.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 -- start start 2023-10-16T14:31:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 0x7f9c4c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c101f00 0x7f9c4c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c073f80 con 0x7f9c4c1028a0 2023-10-16T14:31:52.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0740f0 con 0x7f9c4c101f00 2023-10-16T14:31:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5239d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c1a0940 con 0x7f9c4c100d10 2023-10-16T14:31:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c51b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c5139b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 0x7f9c4c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c51b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52042/0 (socket says 172.21.15.178:52042) 2023-10-16T14:31:52.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c51b9c700 1 -- 172.21.15.178:0/2500474591 learned_addr learned my addr 172.21.15.178:0/2500474591 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.597+0000 7f9c50b9a700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c101f00 0x7f9c4c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 -- 172.21.15.178:0/2500474591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c101f00 msgr2=0x7f9c4c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c101f00 0x7f9c4c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 -- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 msgr2=0x7f9c4c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 -- 172.21.15.178:0/2500474591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c480095f0 con 0x7f9c4c100d10 2023-10-16T14:31:52.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c51b9c700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5139b700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 0x7f9c4c074c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c480098e0 tx=0x7f9c4800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4801d070 con 0x7f9c4c100d10 2023-10-16T14:31:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c48005050 con 0x7f9c4c100d10 2023-10-16T14:31:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c1a0bc0 con 0x7f9c4c100d10 2023-10-16T14:31:52.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48004a70 con 0x7f9c4c100d10 2023-10-16T14:31:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.598+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c1a0fe0 con 0x7f9c4c100d10 2023-10-16T14:31:52.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.599+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c107b50 con 0x7f9c4c100d10 2023-10-16T14:31:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.601+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9c48005450 con 0x7f9c4c100d10 2023-10-16T14:31:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.602+0000 7f9c427fc700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 0x7f9c380a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:52.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.603+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9c480cdc40 con 0x7f9c4c100d10 2023-10-16T14:31:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.603+0000 7f9c50b9a700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 0x7f9c380a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.603+0000 7f9c50b9a700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 0x7f9c380a39e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009c60 tx=0x7f9c3c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:52.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.607+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9c480970e0 con 0x7f9c4c100d10 2023-10-16T14:31:52.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:52 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2395718774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:52.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.810+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c4c02cf70 con 0x7f9c4c100d10 2023-10-16T14:31:52.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.811+0000 7f9c427fc700 1 -- 172.21.15.178:0/2500474591 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c4809a700 con 0x7f9c4c100d10 2023-10-16T14:31:52.814 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:31:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 msgr2=0x7f9c380a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 0x7f9c380a39e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009c60 tx=0x7f9c3c009260 comp rx=0 tx=0).stop 2023-10-16T14:31:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 msgr2=0x7f9c4c074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:52.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 0x7f9c4c074c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9c480098e0 tx=0x7f9c4800f690 comp rx=0 tx=0).stop 2023-10-16T14:31:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 shutdown_connections 2023-10-16T14:31:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9c380a1520 0x7f9c380a39e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c4c1028a0 0x7f9c4c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9c4c100d10 0x7f9c4c074c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 --2- 172.21.15.178:0/2500474591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9c4c101f00 0x7f9c4c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:52.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 >> 172.21.15.178:0/2500474591 conn(0x7f9c4c0fc280 msgr2=0x7f9c4c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:52.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 shutdown_connections 2023-10-16T14:31:52.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:52.814+0000 7f9c5239d700 1 -- 172.21.15.178:0/2500474591 wait complete. 2023-10-16T14:31:52.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:52 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2395718774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:52.835 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:31:52.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:52 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2395718774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:53.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 -- 172.21.15.178:0/4132542794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 msgr2=0x7f2bec1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/4132542794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec1012a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009990 tx=0x7f2be8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 -- 172.21.15.178:0/4132542794 shutdown_connections 2023-10-16T14:31:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/4132542794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/4132542794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec1012a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/4132542794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.031+0000 7f2bf3bff700 1 -- 172.21.15.178:0/4132542794 >> 172.21.15.178:0/4132542794 conn(0x7f2bec0fc400 msgr2=0x7f2bec0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.032+0000 7f2bf3bff700 1 -- 172.21.15.178:0/4132542794 shutdown_connections 2023-10-16T14:31:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.032+0000 7f2bf3bff700 1 -- 172.21.15.178:0/4132542794 wait complete. 2023-10-16T14:31:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.032+0000 7f2bf3bff700 1 Processor -- start 2023-10-16T14:31:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 -- start start 2023-10-16T14:31:53.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a09b0 con 0x7f2bec102060 2023-10-16T14:31:53.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a0af0 con 0x7f2bec102a20 2023-10-16T14:31:53.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf3bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1a0c80 con 0x7f2bec100e80 2023-10-16T14:31:53.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf119a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:58628/0 (socket says 172.21.15.178:58628) 2023-10-16T14:31:53.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf199b700 1 -- 172.21.15.178:0/2399977184 learned_addr learned my addr 172.21.15.178:0/2399977184 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:53.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.033+0000 7f2bf219c700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 -- 172.21.15.178:0/2399977184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 msgr2=0x7f2bec198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 -- 172.21.15.178:0/2399977184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 msgr2=0x7f2bec1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 -- 172.21.15.178:0/2399977184 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80095f0 con 0x7f2bec102060 2023-10-16T14:31:53.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf199b700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:53.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf219c700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:53.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2bf119a700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec199370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9c0 tx=0x7f2bdc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.034+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc00cbd0 con 0x7f2bec102060 2023-10-16T14:31:53.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.035+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bdc00eeb0 con 0x7f2bec102060 2023-10-16T14:31:53.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.035+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003c70 con 0x7f2bec102060 2023-10-16T14:31:53.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.035+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec1a0f60 con 0x7f2bec102060 2023-10-16T14:31:53.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.035+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec1a13e0 con 0x7f2bec102060 2023-10-16T14:31:53.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.036+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec10c740 con 0x7f2bec102060 2023-10-16T14:31:53.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.038+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2bdc003dd0 con 0x7f2bec102060 2023-10-16T14:31:53.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.040+0000 7f2be2ffd700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 0x7f2bd80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.040+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2bdc014070 con 0x7f2bec102060 2023-10-16T14:31:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.040+0000 7f2bf199b700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 0x7f2bd80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.041+0000 7f2bf199b700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 0x7f2bd80a39c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2be800b2e0 tx=0x7f2be8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.045+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2bdc0965c0 con 0x7f2bec102060 2023-10-16T14:31:53.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.248+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bec1a1630 con 0x7f2bec102060 2023-10-16T14:31:53.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.249+0000 7f2be2ffd700 1 -- 172.21.15.178:0/2399977184 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bdc099be0 con 0x7f2bec102060 2023-10-16T14:31:53.252 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:31:53.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 msgr2=0x7f2bd80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 0x7f2bd80a39c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2be800b2e0 tx=0x7f2be8005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 msgr2=0x7f2bec199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec199370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc00e9c0 tx=0x7f2bdc00ecd0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 shutdown_connections 2023-10-16T14:31:53.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2bd80a1500 0x7f2bd80a39c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2bec102a20 0x7f2bec1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2bec100e80 0x7f2bec198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 --2- 172.21.15.178:0/2399977184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2bec102060 0x7f2bec199370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.251+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 >> 172.21.15.178:0/2399977184 conn(0x7f2bec0fc400 msgr2=0x7f2bec0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:53.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.252+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 shutdown_connections 2023-10-16T14:31:53.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.252+0000 7f2bf3bff700 1 -- 172.21.15.178:0/2399977184 wait complete. 2023-10-16T14:31:53.273 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:31:53.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 -- 172.21.15.178:0/648743454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 msgr2=0x7facd81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 --2- 172.21.15.178:0/648743454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd81012c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7facc8009990 tx=0x7facc8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 -- 172.21.15.178:0/648743454 shutdown_connections 2023-10-16T14:31:53.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 --2- 172.21.15.178:0/648743454 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 --2- 172.21.15.178:0/648743454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd81012c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 --2- 172.21.15.178:0/648743454 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 0x7facd8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 -- 172.21.15.178:0/648743454 >> 172.21.15.178:0/648743454 conn(0x7facd80fc420 msgr2=0x7facd80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:53.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 -- 172.21.15.178:0/648743454 shutdown_connections 2023-10-16T14:31:53.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.473+0000 7facde5ff700 1 -- 172.21.15.178:0/648743454 wait complete. 2023-10-16T14:31:53.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.474+0000 7facde5ff700 1 Processor -- start 2023-10-16T14:31:53.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.474+0000 7facde5ff700 1 -- start start 2023-10-16T14:31:53.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 0x7facd81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81a09e0 con 0x7facd8100ea0 2023-10-16T14:31:53.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81a0b20 con 0x7facd8102080 2023-10-16T14:31:53.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facde5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facd81a0cb0 con 0x7facd8102a40 2023-10-16T14:31:53.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facd7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facdcb9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facdcb9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:58650/0 (socket says 172.21.15.178:58650) 2023-10-16T14:31:53.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facd7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52092/0 (socket says 172.21.15.178:52092) 2023-10-16T14:31:53.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facdcb9c700 1 -- 172.21.15.178:0/3824970786 learned_addr learned my addr 172.21.15.178:0/3824970786 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:53.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facd77fe700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 0x7facd81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.475+0000 7facdcb9c700 1 -- 172.21.15.178:0/3824970786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 msgr2=0x7facd81993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facdcb9c700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 0x7facd81993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facdcb9c700 1 -- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 msgr2=0x7facd8198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facdcb9c700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facdcb9c700 1 -- 172.21.15.178:0/3824970786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80095f0 con 0x7facd8102a40 2023-10-16T14:31:53.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facd7fff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:53.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facdcb9c700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7facd000ef30 tx=0x7facd000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0018070 con 0x7facd8102a40 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd000f040 con 0x7facd8102a40 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd81a0f30 con 0x7facd8102a40 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd000cd00 con 0x7facd8102a40 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.476+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd81a1350 con 0x7facd8102a40 2023-10-16T14:31:53.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.477+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facd8193550 con 0x7facd8102a40 2023-10-16T14:31:53.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.479+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7facd0003a00 con 0x7facd8102a40 2023-10-16T14:31:53.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.480+0000 7facd57fa700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 0x7facc00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.480+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7facd00ccbd0 con 0x7facd8102a40 2023-10-16T14:31:53.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.481+0000 7facd7fff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 0x7facc00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.481+0000 7facd7fff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 0x7facc00a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facc800b2e0 tx=0x7facc8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.483+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7facd00cce60 con 0x7facd8102a40 2023-10-16T14:31:53.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:53 smithi178 ceph-mon[98495]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:53.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:53 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2500474591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:53.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:53 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2399977184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:53.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.686+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7facd802d090 con 0x7facd8102a40 2023-10-16T14:31:53.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.687+0000 7facd57fa700 1 -- 172.21.15.178:0/3824970786 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7facd0096070 con 0x7facd8102a40 2023-10-16T14:31:53.690 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:31:53.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 msgr2=0x7facc00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 0x7facc00a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7facc800b2e0 tx=0x7facc8005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 msgr2=0x7facd81a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7facd000ef30 tx=0x7facd000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 shutdown_connections 2023-10-16T14:31:53.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7facc00a1480 0x7facc00a3940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7facd8102a40 0x7facd81a0410 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facd8100ea0 0x7facd8198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 --2- 172.21.15.178:0/3824970786 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7facd8102080 0x7facd81993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 >> 172.21.15.178:0/3824970786 conn(0x7facd80fc420 msgr2=0x7facd8107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:53.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 shutdown_connections 2023-10-16T14:31:53.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.690+0000 7facde5ff700 1 -- 172.21.15.178:0/3824970786 wait complete. 2023-10-16T14:31:53.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:31:53.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:53 smithi186 ceph-mon[134892]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:53.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:53 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2500474591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:53.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:53 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2399977184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:53.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 -- 172.21.15.178:0/316348302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 msgr2=0x7f28a8073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/316348302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a8073340 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2898009990 tx=0x7f2898009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 -- 172.21.15.178:0/316348302 shutdown_connections 2023-10-16T14:31:53.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/316348302 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a810d4a0 0x7f28a810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/316348302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a8073340 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/316348302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a8073910 0x7f28a8073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 -- 172.21.15.178:0/316348302 >> 172.21.15.178:0/316348302 conn(0x7f28a80fc400 msgr2=0x7f28a80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:53.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 -- 172.21.15.178:0/316348302 shutdown_connections 2023-10-16T14:31:53.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.910+0000 7f28ae1ff700 1 -- 172.21.15.178:0/316348302 wait complete. 2023-10-16T14:31:53.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 Processor -- start 2023-10-16T14:31:53.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 -- start start 2023-10-16T14:31:53.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a8073910 0x7f28a819d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81a4db0 con 0x7f28a810d4a0 2023-10-16T14:31:53.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81a4ef0 con 0x7f28a8073910 2023-10-16T14:31:53.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.911+0000 7f28ae1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a81a5080 con 0x7f28a8074ef0 2023-10-16T14:31:53.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52104/0 (socket says 172.21.15.178:52104) 2023-10-16T14:31:53.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a77fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a8073910 0x7f28a819d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 -- 172.21.15.178:0/3942119597 learned_addr learned my addr 172.21.15.178:0/3942119597 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:53.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a6ffd700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 -- 172.21.15.178:0/3942119597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 msgr2=0x7f28a819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 -- 172.21.15.178:0/3942119597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a8073910 msgr2=0x7f28a819d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a8073910 0x7f28a819d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 -- 172.21.15.178:0/3942119597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980095f0 con 0x7f28a810d4a0 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.912+0000 7f28a7fff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28a000ef30 tx=0x7f28a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.913+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a0018070 con 0x7f28a810d4a0 2023-10-16T14:31:53.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.913+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28a000f040 con 0x7f28a810d4a0 2023-10-16T14:31:53.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.913+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a81a5300 con 0x7f28a810d4a0 2023-10-16T14:31:53.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.913+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28a000cd00 con 0x7f28a810d4a0 2023-10-16T14:31:53.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.913+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a81a57c0 con 0x7f28a810d4a0 2023-10-16T14:31:53.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.914+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8102ff0 con 0x7f28a810d4a0 2023-10-16T14:31:53.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.916+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f28a0003a00 con 0x7f28a810d4a0 2023-10-16T14:31:53.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.917+0000 7f28a4ff9700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 0x7f28900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:53.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.917+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f28a00cd3f0 con 0x7f28a810d4a0 2023-10-16T14:31:53.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.917+0000 7f28a77fe700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 0x7f28900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:53.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.918+0000 7f28a77fe700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 0x7f28900a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f289800b2e0 tx=0x7f2898005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:53.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:53.919+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f28a00968f0 con 0x7f28a810d4a0 2023-10-16T14:31:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:53 smithi187 ceph-mon[135297]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:31:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:53 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2500474591' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:53 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2399977184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.126+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f28a8112ea0 con 0x7f28a810d4a0 2023-10-16T14:31:54.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.126+0000 7f28a4ff9700 1 -- 172.21.15.178:0/3942119597 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f28a0099f10 con 0x7f28a810d4a0 2023-10-16T14:31:54.129 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:31:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 msgr2=0x7f28900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 0x7f28900a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f289800b2e0 tx=0x7f2898005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 msgr2=0x7f28a81a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f28a000ef30 tx=0x7f28a000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 shutdown_connections 2023-10-16T14:31:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28900a1480 0x7f28900a3940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28a810d4a0 0x7f28a81a47e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28a8073910 0x7f28a819d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 --2- 172.21.15.178:0/3942119597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28a8074ef0 0x7f28a819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 >> 172.21.15.178:0/3942119597 conn(0x7f28a80fc400 msgr2=0x7f28a80fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 shutdown_connections 2023-10-16T14:31:54.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.129+0000 7f28ae1ff700 1 -- 172.21.15.178:0/3942119597 wait complete. 2023-10-16T14:31:54.150 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T14:31:54.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 -- 172.21.15.178:0/2140273387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102080 msgr2=0x7f2804102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 --2- 172.21.15.178:0/2140273387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102080 0x7f2804102500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f27f80098f0 tx=0x7f27f8009c00 comp rx=0 tx=0).stop 2023-10-16T14:31:54.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 -- 172.21.15.178:0/2140273387 shutdown_connections 2023-10-16T14:31:54.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 --2- 172.21.15.178:0/2140273387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102a40 0x7f2804109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 --2- 172.21.15.178:0/2140273387 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f28041012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 --2- 172.21.15.178:0/2140273387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102080 0x7f2804102500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 -- 172.21.15.178:0/2140273387 >> 172.21.15.178:0/2140273387 conn(0x7f28040fc420 msgr2=0x7f28040fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:54.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 -- 172.21.15.178:0/2140273387 shutdown_connections 2023-10-16T14:31:54.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.346+0000 7f280a1ff700 1 -- 172.21.15.178:0/2140273387 wait complete. 2023-10-16T14:31:54.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.347+0000 7f280a1ff700 1 Processor -- start 2023-10-16T14:31:54.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 -- start start 2023-10-16T14:31:54.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102a40 0x7f28040737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2804073f80 con 0x7f2804102080 2023-10-16T14:31:54.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040740f0 con 0x7f2804100ea0 2023-10-16T14:31:54.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f280a1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28041a0c90 con 0x7f2804102a40 2023-10-16T14:31:54.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f28037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f28037fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:51186/0 (socket says 172.21.15.178:51186) 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f2802ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52112/0 (socket says 172.21.15.178:52112) 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.348+0000 7f28037fe700 1 -- 172.21.15.178:0/1890543223 learned_addr learned my addr 172.21.15.178:0/1890543223 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2803fff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102a40 0x7f28040737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 -- 172.21.15.178:0/1890543223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102a40 msgr2=0x7f28040737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102a40 0x7f28040737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 -- 172.21.15.178:0/1890543223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 msgr2=0x7f2804074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 -- 172.21.15.178:0/1890543223 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009d00 con 0x7f2804102080 2023-10-16T14:31:54.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f28037fe700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:31:54.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2802ffd700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:54.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f801c070 con 0x7f2804102080 2023-10-16T14:31:54.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f8009520 con 0x7f2804102080 2023-10-16T14:31:54.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f800f7d0 con 0x7f2804102080 2023-10-16T14:31:54.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003ac0 con 0x7f2804102080 2023-10-16T14:31:54.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.349+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28041a11a0 con 0x7f2804102080 2023-10-16T14:31:54.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.350+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804108510 con 0x7f2804102080 2023-10-16T14:31:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.352+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f27f8020030 con 0x7f2804102080 2023-10-16T14:31:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.353+0000 7f2800ff9700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 0x7f27ec0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.354+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27f80cdf00 con 0x7f2804102080 2023-10-16T14:31:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.354+0000 7f28037fe700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 0x7f27ec0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.354+0000 7f28037fe700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 0x7f27ec0a3990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f27f4005950 tx=0x7f27f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:54.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.357+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27f80973a0 con 0x7f2804102080 2023-10-16T14:31:54.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.574+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f280410c760 con 0x7f2804102080 2023-10-16T14:31:54.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.574+0000 7f2800ff9700 1 -- 172.21.15.178:0/1890543223 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+602 (secure 0 0 0) 0x7f27f809a9c0 con 0x7f2804102080 2023-10-16T14:31:54.578 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:54.578 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2364174490 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:54.578 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2438067538 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:54.578 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:54.578 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:54.579 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3729267496 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:54.579 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3093526104 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:54.579 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/1552969994 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:54.579 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:54.579 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3372273467 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:54.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 msgr2=0x7f27ec0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 0x7f27ec0a3990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f27f4005950 tx=0x7f27f4009260 comp rx=0 tx=0).stop 2023-10-16T14:31:54.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 msgr2=0x7f28040732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27f8000c00 tx=0x7f27f800b790 comp rx=0 tx=0).stop 2023-10-16T14:31:54.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 shutdown_connections 2023-10-16T14:31:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27ec0a14d0 0x7f27ec0a3990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2804102a40 0x7f28040737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2804100ea0 0x7f2804074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 --2- 172.21.15.178:0/1890543223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2804102080 0x7f28040732a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 >> 172.21.15.178:0/1890543223 conn(0x7f28040fc420 msgr2=0x7f2804105650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:54.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 shutdown_connections 2023-10-16T14:31:54.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.577+0000 7f280a1ff700 1 -- 172.21.15.178:0/1890543223 wait complete. 2023-10-16T14:31:54.586 INFO:teuthology.orchestra.run.smithi178.stderr:listed 10 entries 2023-10-16T14:31:54.598 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:31:54.599 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:31:54.643 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:31:54.742 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:54 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3824970786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.743 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:54 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3942119597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:54 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3824970786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:54 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3942119597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.877+0000 7f8384dff700 1 -- 172.21.15.178:0/3293921809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 msgr2=0x7f83801012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.877+0000 7f8384dff700 1 --2- 172.21.15.178:0/3293921809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f83801012c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8370009990 tx=0x7f8370009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 -- 172.21.15.178:0/3293921809 shutdown_connections 2023-10-16T14:31:54.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 --2- 172.21.15.178:0/3293921809 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102a40 0x7f8380109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 --2- 172.21.15.178:0/3293921809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f83801012c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 --2- 172.21.15.178:0/3293921809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102080 0x7f8380102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 -- 172.21.15.178:0/3293921809 >> 172.21.15.178:0/3293921809 conn(0x7f83800fc420 msgr2=0x7f83800fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:54.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 -- 172.21.15.178:0/3293921809 shutdown_connections 2023-10-16T14:31:54.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.878+0000 7f8384dff700 1 -- 172.21.15.178:0/3293921809 wait complete. 2023-10-16T14:31:54.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 Processor -- start 2023-10-16T14:31:54.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 -- start start 2023-10-16T14:31:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102080 0x7f83800732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 0x7f83800737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8380073f80 con 0x7f8380100ea0 2023-10-16T14:31:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800740f0 con 0x7f8380102080 2023-10-16T14:31:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f8384dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83801a0bd0 con 0x7f8380102a40 2023-10-16T14:31:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f837659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f837659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52132/0 (socket says 172.21.15.178:52132) 2023-10-16T14:31:54.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f837659c700 1 -- 172.21.15.178:0/469522555 learned_addr learned my addr 172.21.15.178:0/469522555 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f837ed9d700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 0x7f83800737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.879+0000 7f837e59c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102080 0x7f83800732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 -- 172.21.15.178:0/469522555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 msgr2=0x7f83800737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 0x7f83800737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 -- 172.21.15.178:0/469522555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102080 msgr2=0x7f83800732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:54.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102080 0x7f83800732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:54.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 -- 172.21.15.178:0/469522555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83700095f0 con 0x7f8380100ea0 2023-10-16T14:31:54.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837ed9d700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 0x7f83800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:54.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f837659c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8370004710 tx=0x7f8370009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:54.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837001d070 con 0x7f8380100ea0 2023-10-16T14:31:54.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8370021410 con 0x7f8380100ea0 2023-10-16T14:31:54.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83801a0e50 con 0x7f8380100ea0 2023-10-16T14:31:54.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370017650 con 0x7f8380100ea0 2023-10-16T14:31:54.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.880+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83801a1210 con 0x7f8380100ea0 2023-10-16T14:31:54.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.881+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838010c7d0 con 0x7f8380100ea0 2023-10-16T14:31:54.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.883+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8370021580 con 0x7f8380100ea0 2023-10-16T14:31:54.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.885+0000 7f8377fff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 0x7f836c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:54.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.886+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83700ce940 con 0x7f8380100ea0 2023-10-16T14:31:54.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.886+0000 7f837e59c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 0x7f836c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:54.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.886+0000 7f837e59c700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 0x7f836c0a3a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8380100bc0 tx=0x7f8368009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:54.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:54.888+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8370097de0 con 0x7f8380100ea0 2023-10-16T14:31:54.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:54 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3824970786' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:54.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:54 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3942119597' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:55.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.092+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8380107da0 con 0x7f8380100ea0 2023-10-16T14:31:55.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.093+0000 7f8377fff700 1 -- 172.21.15.178:0/469522555 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f837009b400 con 0x7f8380100ea0 2023-10-16T14:31:55.097 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:31:55.097 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:31:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 msgr2=0x7f836c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 0x7f836c0a3a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8380100bc0 tx=0x7f8368009260 comp rx=0 tx=0).stop 2023-10-16T14:31:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 msgr2=0x7f8380074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8370004710 tx=0x7f8370009580 comp rx=0 tx=0).stop 2023-10-16T14:31:55.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 shutdown_connections 2023-10-16T14:31:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f836c0a15a0 0x7f836c0a3a60 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8380100bc0 tx=0x7f8368009260 comp rx=0 tx=0).stop 2023-10-16T14:31:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8380102a40 0x7f83800737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8380100ea0 0x7f8380074c60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 --2- 172.21.15.178:0/469522555 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8380102080 0x7f83800732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 >> 172.21.15.178:0/469522555 conn(0x7f83800fc420 msgr2=0x7f83800fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 shutdown_connections 2023-10-16T14:31:55.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.096+0000 7f8384dff700 1 -- 172.21.15.178:0/469522555 wait complete. 2023-10-16T14:31:55.117 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:31:55.118 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:31:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 -- 172.21.15.178:0/795544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10102080 msgr2=0x7fdd10102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 --2- 172.21.15.178:0/795544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10102080 0x7fdd10102500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009990 tx=0x7fdd04009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 -- 172.21.15.178:0/795544796 shutdown_connections 2023-10-16T14:31:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 --2- 172.21.15.178:0/795544796 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102a40 0x7fdd10109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 --2- 172.21.15.178:0/795544796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10100ea0 0x7fdd101012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 --2- 172.21.15.178:0/795544796 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10102080 0x7fdd10102500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 -- 172.21.15.178:0/795544796 >> 172.21.15.178:0/795544796 conn(0x7fdd100fc420 msgr2=0x7fdd100fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 -- 172.21.15.178:0/795544796 shutdown_connections 2023-10-16T14:31:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.318+0000 7fdd155ff700 1 -- 172.21.15.178:0/795544796 wait complete. 2023-10-16T14:31:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.319+0000 7fdd155ff700 1 Processor -- start 2023-10-16T14:31:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.319+0000 7fdd155ff700 1 -- start start 2023-10-16T14:31:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 0x7fdd10196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 0x7fdd101971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 0x7fdd1019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019e820 con 0x7fdd10102080 2023-10-16T14:31:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019e990 con 0x7fdd10100ea0 2023-10-16T14:31:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd155ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019eb50 con 0x7fdd10102a40 2023-10-16T14:31:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 0x7fdd101971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 0x7fdd10196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 0x7fdd101971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52152/0 (socket says 172.21.15.178:52152) 2023-10-16T14:31:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0e7fc700 1 -- 172.21.15.178:0/791412260 learned_addr learned my addr 172.21.15.178:0/791412260 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0f7fe700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 0x7fdd1019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 -- 172.21.15.178:0/791412260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 msgr2=0x7fdd1019e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 0x7fdd1019e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 -- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 msgr2=0x7fdd101971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 0x7fdd101971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0effd700 1 -- 172.21.15.178:0/791412260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040095f0 con 0x7fdd10100ea0 2023-10-16T14:31:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.320+0000 7fdd0f7fe700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 0x7fdd1019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdd0effd700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 0x7fdd10196cb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000e9c0 tx=0x7fdd0000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:55.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0000cbd0 con 0x7fdd10100ea0 2023-10-16T14:31:55.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1019ee30 con 0x7fdd10100ea0 2023-10-16T14:31:55.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd0000eeb0 con 0x7fdd10100ea0 2023-10-16T14:31:55.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00003c70 con 0x7fdd10100ea0 2023-10-16T14:31:55.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.321+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1019f260 con 0x7fdd10100ea0 2023-10-16T14:31:55.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.322+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1010c760 con 0x7fdd10100ea0 2023-10-16T14:31:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.324+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdd00004870 con 0x7fdd10100ea0 2023-10-16T14:31:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.325+0000 7fdcf7fff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 0x7fdcf80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.325+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdd00014070 con 0x7fdd10100ea0 2023-10-16T14:31:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.325+0000 7fdd0e7fc700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 0x7fdcf80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.326+0000 7fdd0e7fc700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 0x7fdcf80a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd101981b0 tx=0x7fdd04005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:55.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.328+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdd00096950 con 0x7fdd10100ea0 2023-10-16T14:31:55.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.531+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd10107ee0 con 0x7fdd10100ea0 2023-10-16T14:31:55.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.532+0000 7fdcf7fff700 1 -- 172.21.15.178:0/791412260 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd00099f70 con 0x7fdd10100ea0 2023-10-16T14:31:55.535 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:31:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 msgr2=0x7fdcf80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 0x7fdcf80a39c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdd101981b0 tx=0x7fdd04005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 msgr2=0x7fdd10196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 0x7fdd10196cb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000e9c0 tx=0x7fdd0000ecd0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 shutdown_connections 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdcf80a1500 0x7fdcf80a39c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd10102a40 0x7fdd1019e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdd10100ea0 0x7fdd10196cb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 --2- 172.21.15.178:0/791412260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd10102080 0x7fdd101971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 >> 172.21.15.178:0/791412260 conn(0x7fdd100fc420 msgr2=0x7fdd100fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:55.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 shutdown_connections 2023-10-16T14:31:55.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.535+0000 7fdd155ff700 1 -- 172.21.15.178:0/791412260 wait complete. 2023-10-16T14:31:55.556 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:31:55.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 -- 172.21.15.178:0/2494787025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 msgr2=0x7f3e341024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 --2- 172.21.15.178:0/2494787025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e341024e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3e280098f0 tx=0x7f3e28009c00 comp rx=0 tx=0).stop 2023-10-16T14:31:55.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 -- 172.21.15.178:0/2494787025 shutdown_connections 2023-10-16T14:31:55.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 --2- 172.21.15.178:0/2494787025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e34109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 --2- 172.21.15.178:0/2494787025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e341012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 --2- 172.21.15.178:0/2494787025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e341024e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 -- 172.21.15.178:0/2494787025 >> 172.21.15.178:0/2494787025 conn(0x7f3e340fc400 msgr2=0x7f3e340fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:55.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 -- 172.21.15.178:0/2494787025 shutdown_connections 2023-10-16T14:31:55.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.754+0000 7f3e39bff700 1 -- 172.21.15.178:0/2494787025 wait complete. 2023-10-16T14:31:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 Processor -- start 2023-10-16T14:31:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 -- start start 2023-10-16T14:31:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e34196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e34197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3419e870 con 0x7f3e34102a20 2023-10-16T14:31:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3419e9e0 con 0x7f3e34100e80 2023-10-16T14:31:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.755+0000 7f3e39bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3419eba0 con 0x7f3e34102060 2023-10-16T14:31:55.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52166/0 (socket says 172.21.15.178:52166) 2023-10-16T14:31:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 -- 172.21.15.178:0/3315448820 learned_addr learned my addr 172.21.15.178:0/3315448820 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e32ffd700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e34197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e337fe700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e34196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 -- 172.21.15.178:0/3315448820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 msgr2=0x7f3e34197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e34197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 -- 172.21.15.178:0/3315448820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 msgr2=0x7f3e34196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e34196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 -- 172.21.15.178:0/3315448820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c009d00 con 0x7f3e34102a20 2023-10-16T14:31:55.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e337fe700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e34196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:31:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.756+0000 7f3e33fff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e3c0 tx=0x7f3e2c00e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.757+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c0092d0 con 0x7f3e34102a20 2023-10-16T14:31:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.757+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2c00f040 con 0x7f3e34102a20 2023-10-16T14:31:55.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.757+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e28009520 con 0x7f3e34102a20 2023-10-16T14:31:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.757+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c007cc0 con 0x7f3e34102a20 2023-10-16T14:31:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.757+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3419f1b0 con 0x7f3e34102a20 2023-10-16T14:31:55.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.758+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e34108590 con 0x7f3e34102a20 2023-10-16T14:31:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.760+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3e2c004ad0 con 0x7f3e34102a20 2023-10-16T14:31:55.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.761+0000 7f3e30ff9700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 0x7f3e1c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.761+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3e2c0ccf30 con 0x7f3e34102a20 2023-10-16T14:31:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.761+0000 7f3e337fe700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 0x7f3e1c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.762+0000 7f3e337fe700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 0x7f3e1c0a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e34198200 tx=0x7f3e24009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:55.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.764+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3e2c0fa100 con 0x7f3e34102a20 2023-10-16T14:31:55.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:55 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1890543223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:55.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:55 smithi178 ceph-mon[98495]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:55.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:55 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/469522555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:55.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:55 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1890543223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:55.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:55 smithi186 ceph-mon[134892]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:55.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:55 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/469522555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:55.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:55 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1890543223' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:55.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:55 smithi187 ceph-mon[135297]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:55.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:55 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/469522555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:31:55.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.963+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e3410c8d0 con 0x7f3e34102a20 2023-10-16T14:31:55.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.964+0000 7f3e30ff9700 1 -- 172.21.15.178:0/3315448820 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e2c0963d0 con 0x7f3e34102a20 2023-10-16T14:31:55.967 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:31:55.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.966+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 msgr2=0x7f3e1c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.966+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 0x7f3e1c0a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3e34198200 tx=0x7f3e24009260 comp rx=0 tx=0).stop 2023-10-16T14:31:55.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.966+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 msgr2=0x7f3e3419e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:55.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.966+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e3c0 tx=0x7f3e2c00e6d0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.966+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 shutdown_connections 2023-10-16T14:31:55.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3e1c0a1500 0x7f3e1c0a39c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e34102a20 0x7f3e3419e2a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3e34100e80 0x7f3e34196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 --2- 172.21.15.178:0/3315448820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e34102060 0x7f3e34197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:55.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 >> 172.21.15.178:0/3315448820 conn(0x7f3e340fc400 msgr2=0x7f3e341056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:55.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 shutdown_connections 2023-10-16T14:31:55.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:55.967+0000 7f3e39bff700 1 -- 172.21.15.178:0/3315448820 wait complete. 2023-10-16T14:31:55.987 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:31:56.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 -- 172.21.15.178:0/1152082575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102a40 msgr2=0x7f3728109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 --2- 172.21.15.178:0/1152082575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102a40 0x7f3728109360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f372000b0a0 tx=0x7f372000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 -- 172.21.15.178:0/1152082575 shutdown_connections 2023-10-16T14:31:56.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 --2- 172.21.15.178:0/1152082575 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102a40 0x7f3728109360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 --2- 172.21.15.178:0/1152082575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f37281012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 --2- 172.21.15.178:0/1152082575 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102080 0x7f3728102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 -- 172.21.15.178:0/1152082575 >> 172.21.15.178:0/1152082575 conn(0x7f37280fc420 msgr2=0x7f37280fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:56.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 -- 172.21.15.178:0/1152082575 shutdown_connections 2023-10-16T14:31:56.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.185+0000 7f372cdff700 1 -- 172.21.15.178:0/1152082575 wait complete. 2023-10-16T14:31:56.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.186+0000 7f372cdff700 1 Processor -- start 2023-10-16T14:31:56.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 -- start start 2023-10-16T14:31:56.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102080 0x7f37280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102a40 0x7f37280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3728073f80 con 0x7f3728102080 2023-10-16T14:31:56.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280740f0 con 0x7f3728102a40 2023-10-16T14:31:56.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372cdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37281a0d80 con 0x7f3728100ea0 2023-10-16T14:31:56.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f3725d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102080 0x7f37280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:58778/0 (socket says 172.21.15.178:58778) 2023-10-16T14:31:56.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.187+0000 7f372659c700 1 -- 172.21.15.178:0/1550709349 learned_addr learned my addr 172.21.15.178:0/1550709349 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:56.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f3726d9d700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102a40 0x7f37280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 -- 172.21.15.178:0/1550709349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102a40 msgr2=0x7f37280737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102a40 0x7f37280737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 -- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102080 msgr2=0x7f37280732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102080 0x7f37280732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 -- 172.21.15.178:0/1550709349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720009d00 con 0x7f3728100ea0 2023-10-16T14:31:56.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372659c700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f371800e970 tx=0x7f371800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:56.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718009790 con 0x7f3728100ea0 2023-10-16T14:31:56.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f371800c7e0 con 0x7f3728100ea0 2023-10-16T14:31:56.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37281a1060 con 0x7f3728100ea0 2023-10-16T14:31:56.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718003c70 con 0x7f3728100ea0 2023-10-16T14:31:56.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.188+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37281a1540 con 0x7f3728100ea0 2023-10-16T14:31:56.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.189+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3728107e50 con 0x7f3728100ea0 2023-10-16T14:31:56.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.191+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f371800cc50 con 0x7f3728100ea0 2023-10-16T14:31:56.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.193+0000 7f37177fe700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 0x7f37100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.193+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3718014070 con 0x7f3728100ea0 2023-10-16T14:31:56.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.193+0000 7f3725d9b700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 0x7f37100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.193+0000 7f3725d9b700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 0x7f37100a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3728100bc0 tx=0x7f371c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:56.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.194+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3718096660 con 0x7f3728100ea0 2023-10-16T14:31:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.397+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f372802cf70 con 0x7f3728100ea0 2023-10-16T14:31:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.398+0000 7f37177fe700 1 -- 172.21.15.178:0/1550709349 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3718003dd0 con 0x7f3728100ea0 2023-10-16T14:31:56.401 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:31:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 msgr2=0x7f37100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 0x7f37100a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3728100bc0 tx=0x7f371c009260 comp rx=0 tx=0).stop 2023-10-16T14:31:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 msgr2=0x7f3728074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f371800e970 tx=0x7f371800ec80 comp rx=0 tx=0).stop 2023-10-16T14:31:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 shutdown_connections 2023-10-16T14:31:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37100a1480 0x7f37100a3940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3728102a40 0x7f37280737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3728100ea0 0x7f3728074c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 --2- 172.21.15.178:0/1550709349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3728102080 0x7f37280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.400+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 >> 172.21.15.178:0/1550709349 conn(0x7f37280fc420 msgr2=0x7f37280fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.401+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 shutdown_connections 2023-10-16T14:31:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.401+0000 7f372cdff700 1 -- 172.21.15.178:0/1550709349 wait complete. 2023-10-16T14:31:56.421 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:31:56.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.620+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/455417298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102070 msgr2=0x7f2f241024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.620+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/455417298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102070 0x7f2f241024f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009990 tx=0x7f2f18009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/455417298 shutdown_connections 2023-10-16T14:31:56.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/455417298 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102a30 0x7f2f24109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/455417298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 0x7f2f241012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/455417298 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102070 0x7f2f241024f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/455417298 >> 172.21.15.178:0/455417298 conn(0x7f2f240fc430 msgr2=0x7f2f240fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:56.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/455417298 shutdown_connections 2023-10-16T14:31:56.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.621+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/455417298 wait complete. 2023-10-16T14:31:56.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 Processor -- start 2023-10-16T14:31:56.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 -- start start 2023-10-16T14:31:56.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 0x7f2f24198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102070 0x7f2f24199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241a0940 con 0x7f2f24102a30 2023-10-16T14:31:56.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241a0a80 con 0x7f2f24102070 2023-10-16T14:31:56.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f2a7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f241a0c10 con 0x7f2f24100e90 2023-10-16T14:31:56.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f28d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f28d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52202/0 (socket says 172.21.15.178:52202) 2023-10-16T14:31:56.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f28d9c700 1 -- 172.21.15.178:0/838094689 learned_addr learned my addr 172.21.15.178:0/838094689 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:56.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f237fe700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102070 0x7f2f24199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.622+0000 7f2f23fff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 0x7f2f24198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 -- 172.21.15.178:0/838094689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 msgr2=0x7f2f24198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 0x7f2f24198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 -- 172.21.15.178:0/838094689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102070 msgr2=0x7f2f24199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102070 0x7f2f24199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 -- 172.21.15.178:0/838094689 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180095f0 con 0x7f2f24102a30 2023-10-16T14:31:56.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f28d9c700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00ef30 tx=0x7f2f1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:56.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c018070 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f1c00f040 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f241a0e90 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00cd00 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.623+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f241a12c0 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.624+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2410c750 con 0x7f2f24102a30 2023-10-16T14:31:56.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.626+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2f1c003a00 con 0x7f2f24102a30 2023-10-16T14:31:56.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.627+0000 7f2f217fa700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 0x7f2f0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:56.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.628+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2f1c0cca60 con 0x7f2f24102a30 2023-10-16T14:31:56.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.628+0000 7f2f23fff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 0x7f2f0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:56.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.628+0000 7f2f23fff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 0x7f2f0c0a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2f24101be0 tx=0x7f2f14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:56.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.631+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2f1c095f00 con 0x7f2f24102a30 2023-10-16T14:31:56.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/791412260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3315448820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1550709349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/791412260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3315448820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1550709349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.833+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2f241a14b0 con 0x7f2f24102a30 2023-10-16T14:31:56.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.834+0000 7f2f217fa700 1 -- 172.21.15.178:0/838094689 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2f1c099520 con 0x7f2f24102a30 2023-10-16T14:31:56.836 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:31:56.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 msgr2=0x7f2f0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 0x7f2f0c0a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2f24101be0 tx=0x7f2f14009260 comp rx=0 tx=0).stop 2023-10-16T14:31:56.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 msgr2=0x7f2f241a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:56.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00ef30 tx=0x7f2f1c00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 shutdown_connections 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f0c0a1480 0x7f2f0c0a3940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f24102a30 0x7f2f241a0370 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f24100e90 0x7f2f24198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 --2- 172.21.15.178:0/838094689 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2f24102070 0x7f2f24199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 >> 172.21.15.178:0/838094689 conn(0x7f2f240fc430 msgr2=0x7f2f240fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:56.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 shutdown_connections 2023-10-16T14:31:56.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:56.836+0000 7f2f2a7ff700 1 -- 172.21.15.178:0/838094689 wait complete. 2023-10-16T14:31:56.857 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T14:31:56.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/791412260' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3315448820' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:31:56.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1550709349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:31:57.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 -- 172.21.15.178:0/4055233670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4101a60 msgr2=0x7f38c4103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:57.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 --2- 172.21.15.178:0/4055233670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4101a60 0x7f38c4103e50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009990 tx=0x7f38b4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 -- 172.21.15.178:0/4055233670 shutdown_connections 2023-10-16T14:31:57.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 --2- 172.21.15.178:0/4055233670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4106cc0 0x7f38c41090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 --2- 172.21.15.178:0/4055233670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4101a60 0x7f38c4103e50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 --2- 172.21.15.178:0/4055233670 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4104390 0x7f38c4106780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 -- 172.21.15.178:0/4055233670 >> 172.21.15.178:0/4055233670 conn(0x7f38c40fb3b0 msgr2=0x7f38c40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:57.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 -- 172.21.15.178:0/4055233670 shutdown_connections 2023-10-16T14:31:57.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.054+0000 7f38c93ff700 1 -- 172.21.15.178:0/4055233670 wait complete. 2023-10-16T14:31:57.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 Processor -- start 2023-10-16T14:31:57.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 -- start start 2023-10-16T14:31:57.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4101a60 0x7f38c4196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:57.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:57.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4106cc0 0x7f38c419e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:57.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419e700 con 0x7f38c4104390 2023-10-16T14:31:57.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419e840 con 0x7f38c4106cc0 2023-10-16T14:31:57.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.055+0000 7f38c93ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c419e9d0 con 0x7f38c4101a60 2023-10-16T14:31:57.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:57.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52228/0 (socket says 172.21.15.178:52228) 2023-10-16T14:31:57.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 -- 172.21.15.178:0/516349201 learned_addr learned my addr 172.21.15.178:0/516349201 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:31:57.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c2ffd700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4101a60 0x7f38c4196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:57.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c37fe700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4106cc0 0x7f38c419e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:57.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 -- 172.21.15.178:0/516349201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4101a60 msgr2=0x7f38c4196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:57.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4101a60 0x7f38c4196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 -- 172.21.15.178:0/516349201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4106cc0 msgr2=0x7f38c419e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:57.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4106cc0 0x7f38c419e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 -- 172.21.15.178:0/516349201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009c40 con 0x7f38c4104390 2023-10-16T14:31:57.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38c27fc700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e9a0 tx=0x7f38b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:57.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.056+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8009790 con 0x7f38c4104390 2023-10-16T14:31:57.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.057+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38b800ce80 con 0x7f38c4104390 2023-10-16T14:31:57.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.057+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b40095f0 con 0x7f38c4104390 2023-10-16T14:31:57.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.057+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8018680 con 0x7f38c4104390 2023-10-16T14:31:57.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.057+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c419f080 con 0x7f38c4104390 2023-10-16T14:31:57.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.058+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c40fcf30 con 0x7f38c4104390 2023-10-16T14:31:57.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.059+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f38b8010430 con 0x7f38c4104390 2023-10-16T14:31:57.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.062+0000 7f38abfff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 0x7f38ac0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:31:57.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.062+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f38b8014070 con 0x7f38c4104390 2023-10-16T14:31:57.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.062+0000 7f38c2ffd700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 0x7f38ac0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:31:57.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.063+0000 7f38c2ffd700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 0x7f38ac0a3a10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b2e0 tx=0x7f38b4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:31:57.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.066+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f38b8052100 con 0x7f38c4104390 2023-10-16T14:31:57.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.288+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38c410c750 con 0x7f38c4104390 2023-10-16T14:31:57.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.288+0000 7f38abfff700 1 -- 172.21.15.178:0/516349201 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+602 (secure 0 0 0) 0x7f38b8099f40 con 0x7f38c4104390 2023-10-16T14:31:57.292 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:57.292 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2364174490 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:57.292 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2438067538 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:57.292 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3729267496 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3093526104 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/1552969994 2023-10-17T14:25:06.501803+0000 2023-10-16T14:31:57.293 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:57.294 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3372273467 2023-10-17T14:24:39.404169+0000 2023-10-16T14:31:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 msgr2=0x7f38ac0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 0x7f38ac0a3a10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b2e0 tx=0x7f38b4005fb0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 msgr2=0x7f38c41970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:31:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f38b800e9a0 tx=0x7f38b800ecb0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 shutdown_connections 2023-10-16T14:31:57.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38ac0a1550 0x7f38ac0a3a10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38c4106cc0 0x7f38c419e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38c4101a60 0x7f38c4196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 --2- 172.21.15.178:0/516349201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c4104390 0x7f38c41970c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:31:57.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 >> 172.21.15.178:0/516349201 conn(0x7f38c40fb3b0 msgr2=0x7f38c4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:31:57.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 shutdown_connections 2023-10-16T14:31:57.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:31:57.291+0000 7f38c93ff700 1 -- 172.21.15.178:0/516349201 wait complete. 2023-10-16T14:31:57.297 INFO:teuthology.orchestra.run.smithi178.stderr:listed 10 entries 2023-10-16T14:31:57.312 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi178.front.sepia.ceph.com... 2023-10-16T14:31:57.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:31:57.313 DEBUG:teuthology.orchestra.run.smithi178:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-10-16T14:31:57.379 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi178.stderr:ceph-fuse[152070]: fuse finished with error 0 and tester_r 0 2023-10-16T14:31:57.384 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T14:31:57.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:57 smithi178 ceph-mon[98495]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:57.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:57 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/838094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:57.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:57 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/516349201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:57.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:57 smithi186 ceph-mon[134892]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:57.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:57 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/838094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:57.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:57 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/516349201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:57.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:57 smithi187 ceph-mon[135297]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-10-16T14:31:57.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:57 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/838094689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:31:57.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:57 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/516349201' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:31:59.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:31:59 smithi187 ceph-mon[135297]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:00.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:31:59 smithi178 ceph-mon[98495]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:31:59 smithi186 ceph-mon[134892]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:01 smithi178 ceph-mon[98495]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:01 smithi186 ceph-mon[134892]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:01.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:01 smithi187 ceph-mon[135297]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:03.485 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:32:03.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:32:03.486 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-10-16T14:32:03.553 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:32:03.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 -- 172.21.15.178:0/3528581485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78102090 msgr2=0x7f3b78102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:03.757 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/3528581485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78102090 0x7f3b78102510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009990 tx=0x7f3b60009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 -- 172.21.15.178:0/3528581485 shutdown_connections 2023-10-16T14:32:03.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/3528581485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102a50 0x7f3b78109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/3528581485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78100eb0 0x7f3b781012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/3528581485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78102090 0x7f3b78102510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 -- 172.21.15.178:0/3528581485 >> 172.21.15.178:0/3528581485 conn(0x7f3b780fc430 msgr2=0x7f3b780fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:03.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 -- 172.21.15.178:0/3528581485 shutdown_connections 2023-10-16T14:32:03.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.754+0000 7f3b7dfff700 1 -- 172.21.15.178:0/3528581485 wait complete. 2023-10-16T14:32:03.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.755+0000 7f3b7dfff700 1 Processor -- start 2023-10-16T14:32:03.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.755+0000 7f3b7dfff700 1 -- start start 2023-10-16T14:32:03.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:03.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 0x7f3b781992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78102a50 0x7f3b781a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b781a08f0 con 0x7f3b78100eb0 2023-10-16T14:32:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b781a0a30 con 0x7f3b78102090 2023-10-16T14:32:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b7dfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b781a0bc0 con 0x7f3b78102a50 2023-10-16T14:32:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33408/0 (socket says 172.21.15.178:33408) 2023-10-16T14:32:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 -- 172.21.15.178:0/2762478511 learned_addr learned my addr 172.21.15.178:0/2762478511 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b77fff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78102a50 0x7f3b781a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b777fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 0x7f3b781992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 -- 172.21.15.178:0/2762478511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78102a50 msgr2=0x7f3b781a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78102a50 0x7f3b781a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 -- 172.21.15.178:0/2762478511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 msgr2=0x7f3b781992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 0x7f3b781992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b6f7fe700 1 -- 172.21.15.178:0/2762478511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600095f0 con 0x7f3b78100eb0 2023-10-16T14:32:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.756+0000 7f3b777fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 0x7f3b781992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b6f7fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800e970 tx=0x7f3b6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68009790 con 0x7f3b78100eb0 2023-10-16T14:32:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b6800eec0 con 0x7f3b78100eb0 2023-10-16T14:32:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b781a0ea0 con 0x7f3b78100eb0 2023-10-16T14:32:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003cf0 con 0x7f3b78100eb0 2023-10-16T14:32:03.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.757+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b781a12d0 con 0x7f3b78100eb0 2023-10-16T14:32:03.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.758+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7810c770 con 0x7f3b78100eb0 2023-10-16T14:32:03.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.760+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3b6800ccd0 con 0x7f3b78100eb0 2023-10-16T14:32:03.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.761+0000 7f3b757fa700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 0x7f3b640a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:03.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.761+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b68014070 con 0x7f3b78100eb0 2023-10-16T14:32:03.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.762+0000 7f3b777fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 0x7f3b640a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:03.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.762+0000 7f3b777fe700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 0x7f3b640a3a60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b60000c00 tx=0x7f3b60005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:03.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.764+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3b68096680 con 0x7f3b78100eb0 2023-10-16T14:32:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:03 smithi187 ceph-mon[135297]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:03.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.966+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b7810c970 con 0x7f3b78100eb0 2023-10-16T14:32:03.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.968+0000 7f3b757fa700 1 -- 172.21.15.178:0/2762478511 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f3b68099ca0 con 0x7f3b78100eb0 2023-10-16T14:32:03.971 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:03.971 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:32:03.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.970+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 msgr2=0x7f3b640a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:03.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.970+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 0x7f3b640a3a60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b60000c00 tx=0x7f3b60005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 msgr2=0x7f3b78198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:03.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800e970 tx=0x7f3b6800ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:03.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 shutdown_connections 2023-10-16T14:32:03.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3b640a15a0 0x7f3b640a3a60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b78102a50 0x7f3b781a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b78100eb0 0x7f3b78198d70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 --2- 172.21.15.178:0/2762478511 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b78102090 0x7f3b781992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:03.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 >> 172.21.15.178:0/2762478511 conn(0x7f3b780fc430 msgr2=0x7f3b780fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:03.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 shutdown_connections 2023-10-16T14:32:03.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:03.971+0000 7f3b7dfff700 1 -- 172.21.15.178:0/2762478511 wait complete. 2023-10-16T14:32:03.990 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:03 smithi178 ceph-mon[98495]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:03.993 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:32:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:03 smithi186 ceph-mon[134892]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:04.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 -- 172.21.15.178:0/2292561295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 msgr2=0x7f2418102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 --2- 172.21.15.178:0/2292561295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f2418102500 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2408009990 tx=0x7f2408009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 -- 172.21.15.178:0/2292561295 shutdown_connections 2023-10-16T14:32:04.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 --2- 172.21.15.178:0/2292561295 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 0x7f2418109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 --2- 172.21.15.178:0/2292561295 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f24181012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 --2- 172.21.15.178:0/2292561295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f2418102500 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 -- 172.21.15.178:0/2292561295 >> 172.21.15.178:0/2292561295 conn(0x7f24180fc420 msgr2=0x7f24180fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:04.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 -- 172.21.15.178:0/2292561295 shutdown_connections 2023-10-16T14:32:04.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.188+0000 7f241f3ff700 1 -- 172.21.15.178:0/2292561295 wait complete. 2023-10-16T14:32:04.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 Processor -- start 2023-10-16T14:32:04.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 -- start start 2023-10-16T14:32:04.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f2418074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 0x7f24180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418073f80 con 0x7f2418100ea0 2023-10-16T14:32:04.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24180740f0 con 0x7f2418102a40 2023-10-16T14:32:04.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.190+0000 7f241f3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24181a0ba0 con 0x7f2418102080 2023-10-16T14:32:04.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241d19b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f2418074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57212/0 (socket says 172.21.15.178:57212) 2023-10-16T14:32:04.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241d19b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f2418074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33418/0 (socket says 172.21.15.178:33418) 2023-10-16T14:32:04.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 -- 172.21.15.178:0/3063827135 learned_addr learned my addr 172.21.15.178:0/3063827135 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:04.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241d99c700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 0x7f24180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 -- 172.21.15.178:0/3063827135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 msgr2=0x7f24180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 0x7f24180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 -- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 msgr2=0x7f2418074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f2418074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 -- 172.21.15.178:0/3063827135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080095f0 con 0x7f2418102080 2023-10-16T14:32:04.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.191+0000 7f241c99a700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2408004710 tx=0x7f2408009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:04.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.192+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240801d070 con 0x7f2418102080 2023-10-16T14:32:04.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.192+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2408021410 con 0x7f2418102080 2023-10-16T14:32:04.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.192+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24181a0e20 con 0x7f2418102080 2023-10-16T14:32:04.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.192+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408017650 con 0x7f2418102080 2023-10-16T14:32:04.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.192+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24181a1190 con 0x7f2418102080 2023-10-16T14:32:04.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.193+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2418107db0 con 0x7f2418102080 2023-10-16T14:32:04.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.195+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2408021580 con 0x7f2418102080 2023-10-16T14:32:04.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.196+0000 7f240e7fc700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 0x7f24040a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.196+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f24080ceac0 con 0x7f2418102080 2023-10-16T14:32:04.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.196+0000 7f241d19b700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 0x7f24040a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.197+0000 7f241d19b700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 0x7f24040a39e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2418101bf0 tx=0x7f2414009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:04.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.198+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2408097f60 con 0x7f2418102080 2023-10-16T14:32:04.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.399+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f241810c7d0 con 0x7f2418102080 2023-10-16T14:32:04.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.401+0000 7f240e7fc700 1 -- 172.21.15.178:0/3063827135 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24020 (secure 0 0 0) 0x7f240809b580 con 0x7f2418102080 2023-10-16T14:32:04.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:04.404 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":87,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:31:41.570379+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:32:04.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.403+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 msgr2=0x7f24040a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.403+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 0x7f24040a39e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2418101bf0 tx=0x7f2414009260 comp rx=0 tx=0).stop 2023-10-16T14:32:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.403+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 msgr2=0x7f24180732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.403+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2408004710 tx=0x7f2408009580 comp rx=0 tx=0).stop 2023-10-16T14:32:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 shutdown_connections 2023-10-16T14:32:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24040a1520 0x7f24040a39e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2418102a40 0x7f24180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2418100ea0 0x7f2418074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 --2- 172.21.15.178:0/3063827135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2418102080 0x7f24180732a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 >> 172.21.15.178:0/3063827135 conn(0x7f24180fc420 msgr2=0x7f24180fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 shutdown_connections 2023-10-16T14:32:04.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.404+0000 7f241f3ff700 1 -- 172.21.15.178:0/3063827135 wait complete. 2023-10-16T14:32:04.426 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T14:32:04.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3272481562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98100e80 msgr2=0x7f9d981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3272481562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98100e80 0x7f9d981012a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9d940098f0 tx=0x7f9d94009c00 comp rx=0 tx=0).stop 2023-10-16T14:32:04.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3272481562 shutdown_connections 2023-10-16T14:32:04.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3272481562 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d98109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3272481562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98100e80 0x7f9d981012a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3272481562 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98102060 0x7f9d981024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3272481562 >> 172.21.15.178:0/3272481562 conn(0x7f9d980fc400 msgr2=0x7f9d980fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:04.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3272481562 shutdown_connections 2023-10-16T14:32:04.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.624+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3272481562 wait complete. 2023-10-16T14:32:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.625+0000 7f9d9ffff700 1 Processor -- start 2023-10-16T14:32:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.625+0000 7f9d9ffff700 1 -- start start 2023-10-16T14:32:04.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98100e80 0x7f9d98198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98102060 0x7f9d981993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d981a09f0 con 0x7f9d98102a20 2023-10-16T14:32:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d981a0b30 con 0x7f9d98100e80 2023-10-16T14:32:04.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9ffff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d981a0cc0 con 0x7f9d98102060 2023-10-16T14:32:04.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33426/0 (socket says 172.21.15.178:33426) 2023-10-16T14:32:04.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 -- 172.21.15.178:0/3791381603 learned_addr learned my addr 172.21.15.178:0/3791381603 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9dd9b700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98100e80 0x7f9d98198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9d59a700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98102060 0x7f9d981993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 -- 172.21.15.178:0/3791381603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98102060 msgr2=0x7f9d981993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98102060 0x7f9d981993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 -- 172.21.15.178:0/3791381603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98100e80 msgr2=0x7f9d98198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98100e80 0x7f9d98198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.626+0000 7f9d9e59c700 1 -- 172.21.15.178:0/3791381603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880095f0 con 0x7f9d98102a20 2023-10-16T14:32:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d9e59c700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000ef30 tx=0x7f9d9000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90018070 con 0x7f9d98102a20 2023-10-16T14:32:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d9000f040 con 0x7f9d98102a20 2023-10-16T14:32:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d94009520 con 0x7f9d98102a20 2023-10-16T14:32:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000cd00 con 0x7f9d98102a20 2023-10-16T14:32:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.627+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d981a1210 con 0x7f9d98102a20 2023-10-16T14:32:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.628+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9810c740 con 0x7f9d98102a20 2023-10-16T14:32:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.630+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d90003a00 con 0x7f9d98102a20 2023-10-16T14:32:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.631+0000 7f9d8effd700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 0x7f9d840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.631+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9d900cca90 con 0x7f9d98102a20 2023-10-16T14:32:04.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.632+0000 7f9d9dd9b700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 0x7f9d840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:04.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.632+0000 7f9d9dd9b700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 0x7f9d840a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009fd0 tx=0x7f9d94019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:04.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.635+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9d90095f30 con 0x7f9d98102a20 2023-10-16T14:32:04.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2762478511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:04.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3063827135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:04.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:04.885+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d981a1400 con 0x7f9d98102a20 2023-10-16T14:32:04.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2762478511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:04.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3063827135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2762478511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3063827135' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:05.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.657+0000 7f9d8effd700 1 -- 172.21.15.178:0/3791381603 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9d90099550 con 0x7f9d98102a20 2023-10-16T14:32:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 msgr2=0x7f9d840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:05.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 0x7f9d840a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009fd0 tx=0x7f9d94019040 comp rx=0 tx=0).stop 2023-10-16T14:32:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 msgr2=0x7f9d981a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000ef30 tx=0x7f9d9000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 shutdown_connections 2023-10-16T14:32:05.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d840a1480 0x7f9d840a3940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d98102a20 0x7f9d981a0420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d98100e80 0x7f9d98198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 --2- 172.21.15.178:0/3791381603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d98102060 0x7f9d981993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 >> 172.21.15.178:0/3791381603 conn(0x7f9d980fc400 msgr2=0x7f9d980fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:05.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 shutdown_connections 2023-10-16T14:32:05.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.660+0000 7f9d9ffff700 1 -- 172.21.15.178:0/3791381603 wait complete. 2023-10-16T14:32:05.682 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-10-16T14:32:05.683 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-10-16T14:32:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 -- 172.21.15.178:0/1193454662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768073910 msgr2=0x7f8768073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 --2- 172.21.15.178:0/1193454662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768073910 0x7f8768073d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f875c009990 tx=0x7f875c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 -- 172.21.15.178:0/1193454662 shutdown_connections 2023-10-16T14:32:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 --2- 172.21.15.178:0/1193454662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768108db0 0x7f876810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 --2- 172.21.15.178:0/1193454662 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 0x7f8768073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 --2- 172.21.15.178:0/1193454662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768073910 0x7f8768073d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 -- 172.21.15.178:0/1193454662 >> 172.21.15.178:0/1193454662 conn(0x7f87680fc420 msgr2=0x7f87680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 -- 172.21.15.178:0/1193454662 shutdown_connections 2023-10-16T14:32:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.883+0000 7f876d9ff700 1 -- 172.21.15.178:0/1193454662 wait complete. 2023-10-16T14:32:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 Processor -- start 2023-10-16T14:32:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 -- start start 2023-10-16T14:32:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 0x7f8768198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 0x7f8768199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 0x7f87681a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87681a09d0 con 0x7f8768108db0 2023-10-16T14:32:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87681a0b10 con 0x7f8768074ef0 2023-10-16T14:32:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.884+0000 7f876d9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87681a0ca0 con 0x7f8768073910 2023-10-16T14:32:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 0x7f87681a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f8766ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 0x7f8768198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f8766ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 0x7f8768198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57234/0 (socket says 172.21.15.178:57234) 2023-10-16T14:32:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f8766ffd700 1 -- 172.21.15.178:0/4020180764 learned_addr learned my addr 172.21.15.178:0/4020180764 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87667fc700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 0x7f8768199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 -- 172.21.15.178:0/4020180764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 msgr2=0x7f8768198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:05.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 0x7f8768198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 -- 172.21.15.178:0/4020180764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 msgr2=0x7f8768199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:05.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 0x7f8768199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:05.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 -- 172.21.15.178:0/4020180764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c0095f0 con 0x7f8768108db0 2023-10-16T14:32:05.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f87677fe700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 0x7f87681a0400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f876000efc0 tx=0x7f876000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:05.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760018070 con 0x7f8768108db0 2023-10-16T14:32:05.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f876000f040 con 0x7f8768108db0 2023-10-16T14:32:05.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87681a0f20 con 0x7f8768108db0 2023-10-16T14:32:05.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000cd40 con 0x7f8768108db0 2023-10-16T14:32:05.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.885+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87681a13a0 con 0x7f8768108db0 2023-10-16T14:32:05.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.886+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8768107460 con 0x7f8768108db0 2023-10-16T14:32:05.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.888+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8760003a00 con 0x7f8768108db0 2023-10-16T14:32:05.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.890+0000 7f876c9fd700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 0x7f87500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:05.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.890+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f87600cca50 con 0x7f8768108db0 2023-10-16T14:32:05.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.890+0000 7f8766ffd700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 0x7f87500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:05.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.891+0000 7f8766ffd700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 0x7f87500a39c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8758009d20 tx=0x7f8758009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:05.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:05.894+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8760095ef0 con 0x7f8768108db0 2023-10-16T14:32:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:05 smithi187 ceph-mon[135297]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:05.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-16T14:32:06.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:05 smithi178 ceph-mon[98495]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-16T14:32:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:05 smithi186 ceph-mon[134892]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-10-16T14:32:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-10-16T14:32:06.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.140+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f876802d020 con 0x7f8768108db0 2023-10-16T14:32:06.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.663+0000 7f876c9fd700 1 -- 172.21.15.178:0/4020180764 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f876001b320 con 0x7f8768108db0 2023-10-16T14:32:06.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 msgr2=0x7f87500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:06.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 0x7f87500a39c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8758009d20 tx=0x7f8758009260 comp rx=0 tx=0).stop 2023-10-16T14:32:06.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 msgr2=0x7f87681a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:06.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 0x7f87681a0400 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f876000efc0 tx=0x7f876000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 shutdown_connections 2023-10-16T14:32:06.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87500a1500 0x7f87500a39c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8768108db0 0x7f87681a0400 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8768073910 0x7f8768198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 --2- 172.21.15.178:0/4020180764 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8768074ef0 0x7f8768199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 >> 172.21.15.178:0/4020180764 conn(0x7f87680fc420 msgr2=0x7f87680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:06.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 shutdown_connections 2023-10-16T14:32:06.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.667+0000 7f876d9ff700 1 -- 172.21.15.178:0/4020180764 wait complete. 2023-10-16T14:32:06.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-10-16T14:32:06.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 -- 172.21.15.178:0/1640232439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102080 msgr2=0x7f91d4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:06.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/1640232439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102080 0x7f91d4102500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009990 tx=0x7f91c4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 -- 172.21.15.178:0/1640232439 shutdown_connections 2023-10-16T14:32:06.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/1640232439 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4102a40 0x7f91d4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/1640232439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4100ea0 0x7f91d41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/1640232439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102080 0x7f91d4102500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 -- 172.21.15.178:0/1640232439 >> 172.21.15.178:0/1640232439 conn(0x7f91d40fc420 msgr2=0x7f91d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:06.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 -- 172.21.15.178:0/1640232439 shutdown_connections 2023-10-16T14:32:06.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.896+0000 7f91dc7ff700 1 -- 172.21.15.178:0/1640232439 wait complete. 2023-10-16T14:32:06.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 Processor -- start 2023-10-16T14:32:06.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 -- start start 2023-10-16T14:32:06.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4100ea0 0x7f91d4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:06.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:06.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:06.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d419e820 con 0x7f91d4102a40 2023-10-16T14:32:06.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d419e990 con 0x7f91d4100ea0 2023-10-16T14:32:06.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.897+0000 7f91dc7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d419eb50 con 0x7f91d4102080 2023-10-16T14:32:06.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:06.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91d9d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:06.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33460/0 (socket says 172.21.15.178:33460) 2023-10-16T14:32:06.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91d9d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57258/0 (socket says 172.21.15.178:57258) 2023-10-16T14:32:06.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 -- 172.21.15.178:0/92880706 learned_addr learned my addr 172.21.15.178:0/92880706 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:06.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91da59b700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4100ea0 0x7f91d4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:06.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 -- 172.21.15.178:0/92880706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 msgr2=0x7f91d41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:06.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 -- 172.21.15.178:0/92880706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4100ea0 msgr2=0x7f91d4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:06.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4100ea0 0x7f91d4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:06.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 -- 172.21.15.178:0/92880706 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40095f0 con 0x7f91d4102a40 2023-10-16T14:32:06.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91d9d9a700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:06.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.898+0000 7f91dad9c700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00ef30 tx=0x7f91cc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:06.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.899+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc018070 con 0x7f91d4102a40 2023-10-16T14:32:06.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.899+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc00f040 con 0x7f91d4102a40 2023-10-16T14:32:06.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.899+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00cd00 con 0x7f91d4102a40 2023-10-16T14:32:06.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.899+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40730e0 con 0x7f91d4102a40 2023-10-16T14:32:06.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.899+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d4073570 con 0x7f91d4102a40 2023-10-16T14:32:06.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.900+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d410c760 con 0x7f91d4102a40 2023-10-16T14:32:06.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.902+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f91cc003a00 con 0x7f91d4102a40 2023-10-16T14:32:06.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.903+0000 7f91cb7fe700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 0x7f91c00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:06.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.903+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f91cc0cd4b0 con 0x7f91d4102a40 2023-10-16T14:32:06.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.903+0000 7f91da59b700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 0x7f91c00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:06.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.904+0000 7f91da59b700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 0x7f91c00a39c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91d4197b80 tx=0x7f91d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:06.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:06.905+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91cc096830 con 0x7f91d4102a40 2023-10-16T14:32:06.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-16T14:32:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-16T14:32:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]: dispatch 2023-10-16T14:32:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-16T14:32:06.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-10-16T14:32:06.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-16T14:32:06.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-16T14:32:06.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-10-16T14:32:06.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-16T14:32:06.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-16T14:32:06.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-10-16T14:32:06.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:06 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-16T14:32:07.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-16T14:32:07.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:07.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-16T14:32:07.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]: dispatch 2023-10-16T14:32:07.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-16T14:32:07.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-10-16T14:32:07.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-16T14:32:07.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-16T14:32:07.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-10-16T14:32:07.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-16T14:32:07.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-16T14:32:07.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-10-16T14:32:07.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:06 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-16T14:32:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3791381603' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-10-16T14:32:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:07.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-10-16T14:32:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]: dispatch 2023-10-16T14:32:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2023-10-16T14:32:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-10-16T14:32:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2023-10-16T14:32:07.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2023-10-16T14:32:07.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2023-10-16T14:32:07.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-10-16T14:32:07.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-10-16T14:32:07.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2023-10-16T14:32:07.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:06 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2023-10-16T14:32:07.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.154+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f91d4073790 con 0x7f91d4102a40 2023-10-16T14:32:07.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.683+0000 7f91cb7fe700 1 -- 172.21.15.178:0/92880706 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f91cc099e50 con 0x7f91d4102a40 2023-10-16T14:32:07.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 msgr2=0x7f91c00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:07.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 0x7f91c00a39c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91d4197b80 tx=0x7f91d0009260 comp rx=0 tx=0).stop 2023-10-16T14:32:07.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 msgr2=0x7f91d419e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:07.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00ef30 tx=0x7f91cc00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 shutdown_connections 2023-10-16T14:32:07.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91c00a1500 0x7f91c00a39c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91d4102a40 0x7f91d419e250 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91d4100ea0 0x7f91d4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 --2- 172.21.15.178:0/92880706 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91d4102080 0x7f91d41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.686+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 >> 172.21.15.178:0/92880706 conn(0x7f91d40fc420 msgr2=0x7f91d40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:07.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.687+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 shutdown_connections 2023-10-16T14:32:07.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.687+0000 7f91dc7ff700 1 -- 172.21.15.178:0/92880706 wait complete. 2023-10-16T14:32:07.710 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-10-16T14:32:07.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.907+0000 7fb7acfff700 1 -- 172.21.15.178:0/1496634636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 msgr2=0x7fb7a8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:07.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.907+0000 7fb7acfff700 1 --2- 172.21.15.178:0/1496634636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8102500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb790009990 tx=0x7fb790009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.907+0000 7fb7acfff700 1 -- 172.21.15.178:0/1496634636 shutdown_connections 2023-10-16T14:32:07.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.907+0000 7fb7acfff700 1 --2- 172.21.15.178:0/1496634636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 0x7fb7a8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.908+0000 7fb7acfff700 1 --2- 172.21.15.178:0/1496634636 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.908+0000 7fb7acfff700 1 --2- 172.21.15.178:0/1496634636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8102500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.908+0000 7fb7acfff700 1 -- 172.21.15.178:0/1496634636 >> 172.21.15.178:0/1496634636 conn(0x7fb7a80fc420 msgr2=0x7fb7a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:07.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.908+0000 7fb7acfff700 1 -- 172.21.15.178:0/1496634636 shutdown_connections 2023-10-16T14:32:07.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.908+0000 7fb7acfff700 1 -- 172.21.15.178:0/1496634636 wait complete. 2023-10-16T14:32:07.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 Processor -- start 2023-10-16T14:32:07.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 -- start start 2023-10-16T14:32:07.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:07.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:07.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 0x7fb7a81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:07.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a81a09d0 con 0x7fb7a8102a40 2023-10-16T14:32:07.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a81a0b10 con 0x7fb7a8100ea0 2023-10-16T14:32:07.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7acfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a81a0ca0 con 0x7fb7a8102080 2023-10-16T14:32:07.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb79e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:07.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7a6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 0x7fb7a81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:07.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb79e59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:33620/0 (socket says 172.21.15.178:33620) 2023-10-16T14:32:07.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb79e59c700 1 -- 172.21.15.178:0/3312661838 learned_addr learned my addr 172.21.15.178:0/3312661838 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:07.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.909+0000 7fb7a659c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:07.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 -- 172.21.15.178:0/3312661838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 msgr2=0x7fb7a8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:07.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 -- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 msgr2=0x7fb7a81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:07.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 0x7fb7a81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:07.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 -- 172.21.15.178:0/3312661838 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900095f0 con 0x7fb7a8100ea0 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb7a659c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79e59c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d980 tx=0x7fb79800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004020 con 0x7fb7a8100ea0 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79800f690 con 0x7fb7a8100ea0 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a81a0f80 con 0x7fb7a8100ea0 2023-10-16T14:32:07.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.910+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798003a60 con 0x7fb7a8100ea0 2023-10-16T14:32:07.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.911+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a81a13a0 con 0x7fb7a8100ea0 2023-10-16T14:32:07.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.911+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a810c760 con 0x7fb7a8100ea0 2023-10-16T14:32:07.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.914+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb79800fb00 con 0x7fb7a8100ea0 2023-10-16T14:32:07.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.915+0000 7fb79ffff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 0x7fb7940a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:07.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.915+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7980ccf80 con 0x7fb7a8100ea0 2023-10-16T14:32:07.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.915+0000 7fb7a659c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 0x7fb7940a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:07.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.916+0000 7fb7a659c700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 0x7fb7940a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b2e0 tx=0x7fb790005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:07.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:07.917+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb798096270 con 0x7fb7a8100ea0 2023-10-16T14:32:07.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-16T14:32:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-16T14:32:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-16T14:32:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]': finished 2023-10-16T14:32:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-16T14:32:07.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-10-16T14:32:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-16T14:32:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-16T14:32:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-10-16T14:32:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-16T14:32:07.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-16T14:32:07.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-10-16T14:32:07.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-16T14:32:07.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-16T14:32:07.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:07.927 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: osdmap e88: 12 total, 12 up, 12 in 2023-10-16T14:32:07.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:07.928 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:07 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-16T14:32:08.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-16T14:32:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-16T14:32:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-16T14:32:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]': finished 2023-10-16T14:32:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-16T14:32:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-10-16T14:32:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-16T14:32:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-16T14:32:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-10-16T14:32:08.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-16T14:32:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-16T14:32:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-10-16T14:32:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-16T14:32:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-16T14:32:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:08.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: osdmap e88: 12 total, 12 up, 12 in 2023-10-16T14:32:08.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:08.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:07 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-16T14:32:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2023-10-16T14:32:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-10-16T14:32:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4020180764' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-10-16T14:32:08.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [5, 6]}]': finished 2023-10-16T14:32:08.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2023-10-16T14:32:08.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-10-16T14:32:08.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2023-10-16T14:32:08.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2023-10-16T14:32:08.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2023-10-16T14:32:08.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-10-16T14:32:08.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-10-16T14:32:08.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2023-10-16T14:32:08.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2023-10-16T14:32:08.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-10-16T14:32:08.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-10-16T14:32:08.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: osdmap e88: 12 total, 12 up, 12 in 2023-10-16T14:32:08.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:08.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:07 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-10-16T14:32:08.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.169+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fb7a81a15f0 con 0x7fb7a8100ea0 2023-10-16T14:32:08.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.690+0000 7fb79ffff700 1 -- 172.21.15.178:0/3312661838 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb798099890 con 0x7fb7a8100ea0 2023-10-16T14:32:08.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 msgr2=0x7fb7940a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:08.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 0x7fb7940a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb79000b2e0 tx=0x7fb790005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 msgr2=0x7fb7a8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:08.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d980 tx=0x7fb79800dc90 comp rx=0 tx=0).stop 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 shutdown_connections 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7940a1480 0x7fb7940a3940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7a8102a40 0x7fb7a81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb7a8100ea0 0x7fb7a8198e50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 --2- 172.21.15.178:0/3312661838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7a8102080 0x7fb7a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 >> 172.21.15.178:0/3312661838 conn(0x7fb7a80fc420 msgr2=0x7fb7a80fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:08.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 shutdown_connections 2023-10-16T14:32:08.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.693+0000 7fb7acfff700 1 -- 172.21.15.178:0/3312661838 wait complete. 2023-10-16T14:32:08.721 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-10-16T14:32:08.722 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-10-16T14:32:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1339715385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 msgr2=0x7fed24109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1339715385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed24109340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00b0a0 tx=0x7fed1c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1339715385 shutdown_connections 2023-10-16T14:32:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1339715385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed24109340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1339715385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed241012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1339715385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 0x7fed241024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1339715385 >> 172.21.15.178:0/1339715385 conn(0x7fed240fc400 msgr2=0x7fed240fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:08.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1339715385 shutdown_connections 2023-10-16T14:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.920+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1339715385 wait complete. 2023-10-16T14:32:08.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.921+0000 7fed2b1ff700 1 Processor -- start 2023-10-16T14:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.921+0000 7fed2b1ff700 1 -- start start 2023-10-16T14:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.921+0000 7fed2b1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed24074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:08.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.921+0000 7fed2b1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 0x7fed240732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2b1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:08.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2b1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24073f80 con 0x7fed24102a20 2023-10-16T14:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2b1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed240740f0 con 0x7fed24102060 2023-10-16T14:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2b1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed241a0d80 con 0x7fed24100e80 2023-10-16T14:32:08.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed28f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed24074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57700/0 (socket says 172.21.15.178:57700) 2023-10-16T14:32:08.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 -- 172.21.15.178:0/1811038954 learned_addr learned my addr 172.21.15.178:0/1811038954 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed23fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 0x7fed240732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 -- 172.21.15.178:0/1811038954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 msgr2=0x7fed24074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:08.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed24074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 -- 172.21.15.178:0/1811038954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 msgr2=0x7fed240732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 0x7fed240732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:08.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 -- 172.21.15.178:0/1811038954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009d00 con 0x7fed24102a20 2023-10-16T14:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed28f9b700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed24074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.922+0000 7fed2979c700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009fd0 tx=0x7fed1c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.923+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00e040 con 0x7fed24102a20 2023-10-16T14:32:08.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.923+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed1c003e10 con 0x7fed24102a20 2023-10-16T14:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.923+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed241a1000 con 0x7fed24102a20 2023-10-16T14:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.923+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c01b6a0 con 0x7fed24102a20 2023-10-16T14:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.923+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed241a1420 con 0x7fed24102a20 2023-10-16T14:32:08.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.924+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2410c740 con 0x7fed24102a20 2023-10-16T14:32:08.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.926+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fed1c01d470 con 0x7fed24102a20 2023-10-16T14:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.927+0000 7fed21ffb700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 0x7fed140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.927+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fed1c0ce190 con 0x7fed24102a20 2023-10-16T14:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.927+0000 7fed28f9b700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 0x7fed140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.928+0000 7fed28f9b700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 0x7fed140a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed24101bd0 tx=0x7fed18009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:08.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:08.931+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fed1c097510 con 0x7fed24102a20 2023-10-16T14:32:09.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: osdmap e89: 12 total, 12 up, 12 in 2023-10-16T14:32:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-16T14:32:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-16T14:32:09.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3312661838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:08 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: osdmap e89: 12 total, 12 up, 12 in 2023-10-16T14:32:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-16T14:32:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:09.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-16T14:32:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3312661838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:08 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.170+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fed240612f0 con 0x7fed140a1480 2023-10-16T14:32:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: osdmap e89: 12 total, 12 up, 12 in 2023-10-16T14:32:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/92880706' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-10-16T14:32:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-10-16T14:32:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-10-16T14:32:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3312661838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:08 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-10-16T14:32:09.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:32:09.178+0000 7f05ee1bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T14:32:09.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:32:09.178+0000 7f05ee1bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T14:32:09.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:32:09.178+0000 7f05ee1bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T14:32:09.553 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:32:09.178+0000 7f05ee1bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T14:32:09.554 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:32:09.178+0000 7f05ee1bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-16T14:32:09.741 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.739+0000 7fed21ffb700 1 -- 172.21.15.178:0/1811038954 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fed240612f0 con 0x7fed140a1480 2023-10-16T14:32:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 msgr2=0x7fed140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 0x7fed140a3940 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fed24101bd0 tx=0x7fed18009260 comp rx=0 tx=0).stop 2023-10-16T14:32:09.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 msgr2=0x7fed240737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009fd0 tx=0x7fed1c00bce0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 shutdown_connections 2023-10-16T14:32:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fed140a1480 0x7fed140a3940 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed24102a20 0x7fed240737e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed24100e80 0x7fed24074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 --2- 172.21.15.178:0/1811038954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fed24102060 0x7fed240732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 >> 172.21.15.178:0/1811038954 conn(0x7fed240fc400 msgr2=0x7fed240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 shutdown_connections 2023-10-16T14:32:09.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.741+0000 7fed2b1ff700 1 -- 172.21.15.178:0/1811038954 wait complete. 2023-10-16T14:32:09.764 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-10-16T14:32:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 -- 172.21.15.178:0/3662427920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102a20 msgr2=0x7f32f4109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 --2- 172.21.15.178:0/3662427920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102a20 0x7f32f4109340 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00b0a0 tx=0x7f32ec00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 -- 172.21.15.178:0/3662427920 shutdown_connections 2023-10-16T14:32:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 --2- 172.21.15.178:0/3662427920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102a20 0x7f32f4109340 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 --2- 172.21.15.178:0/3662427920 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4100e80 0x7f32f41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 --2- 172.21.15.178:0/3662427920 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4102060 0x7f32f41024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 -- 172.21.15.178:0/3662427920 >> 172.21.15.178:0/3662427920 conn(0x7f32f40fc400 msgr2=0x7f32f40fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 -- 172.21.15.178:0/3662427920 shutdown_connections 2023-10-16T14:32:09.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.963+0000 7f32fafff700 1 -- 172.21.15.178:0/3662427920 wait complete. 2023-10-16T14:32:09.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.964+0000 7f32fafff700 1 Processor -- start 2023-10-16T14:32:09.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.964+0000 7f32fafff700 1 -- start start 2023-10-16T14:32:09.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.964+0000 7f32fafff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4100e80 0x7f32f4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:09.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32fafff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:09.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32fafff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 0x7f32f41a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:09.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32fafff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41a0aa0 con 0x7f32f4102060 2023-10-16T14:32:09.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32fafff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41a0be0 con 0x7f32f4102a20 2023-10-16T14:32:09.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32fafff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41a0d70 con 0x7f32f4100e80 2023-10-16T14:32:09.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:09.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57724/0 (socket says 172.21.15.178:57724) 2023-10-16T14:32:09.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 -- 172.21.15.178:0/3141430996 learned_addr learned my addr 172.21.15.178:0/3141430996 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:09.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f8d9b700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4100e80 0x7f32f4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:09.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f959c700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 0x7f32f41a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:09.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 -- 172.21.15.178:0/3141430996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4100e80 msgr2=0x7f32f4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:09.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4100e80 0x7f32f4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 -- 172.21.15.178:0/3141430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 msgr2=0x7f32f41a04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:09.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 0x7f32f41a04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:09.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 -- 172.21.15.178:0/3141430996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e8009520 con 0x7f32f4102060 2023-10-16T14:32:09.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f959c700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 0x7f32f41a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:09.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.965+0000 7f32f3fff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e970 tx=0x7f32e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:09.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.966+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0009790 con 0x7f32f4102060 2023-10-16T14:32:09.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.966+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e000c7e0 con 0x7f32f4102060 2023-10-16T14:32:09.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.966+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec009d00 con 0x7f32f4102060 2023-10-16T14:32:09.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.966+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0003c70 con 0x7f32f4102060 2023-10-16T14:32:09.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.966+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f41a1340 con 0x7f32f4102060 2023-10-16T14:32:09.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.967+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f410c740 con 0x7f32f4102060 2023-10-16T14:32:09.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.969+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f32e000cc50 con 0x7f32f4102060 2023-10-16T14:32:09.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.970+0000 7f32f1ffb700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 0x7f32e40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:09.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.970+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32e0014070 con 0x7f32f4102060 2023-10-16T14:32:09.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.970+0000 7f32f8d9b700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 0x7f32e40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:09.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.971+0000 7f32f8d9b700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 0x7f32e40a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32f4101bd0 tx=0x7f32e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:09.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:09.972+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f32e0096640 con 0x7f32f4102060 2023-10-16T14:32:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-10-16T14:32:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-16T14:32:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:32:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-16T14:32:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-16T14:32:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:32:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-16T14:32:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-16T14:32:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:09 smithi178 ceph-mon[98495]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-16T14:32:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-10-16T14:32:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-16T14:32:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:32:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-16T14:32:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-16T14:32:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:32:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-16T14:32:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-16T14:32:10.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:09 smithi186 ceph-mon[134892]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-16T14:32:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: pgmap v290: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 1 op/s 2023-10-16T14:32:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-10-16T14:32:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:32:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-10-16T14:32:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-10-16T14:32:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:32:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-10-16T14:32:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2023-10-16T14:32:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:09 smithi187 ceph-mon[135297]: daemon mds.h is now active in filesystem cephfs as rank 2 2023-10-16T14:32:10.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.208+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f32f4108a10 con 0x7f32e40a1480 2023-10-16T14:32:10.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.213+0000 7f32f1ffb700 1 -- 172.21.15.178:0/3141430996 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f32f4108a10 con 0x7f32e40a1480 2023-10-16T14:32:10.216 INFO:teuthology.orchestra.run.smithi178.stdout:/volumes/_nogroup/sv_0/0578edd7-9b00-4d9e-b647-f4b885805273 2023-10-16T14:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 msgr2=0x7f32e40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 0x7f32e40a3940 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32f4101bd0 tx=0x7f32e8009260 comp rx=0 tx=0).stop 2023-10-16T14:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 msgr2=0x7f32f4199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e970 tx=0x7f32e000ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 shutdown_connections 2023-10-16T14:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32e40a1480 0x7f32e40a3940 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f32f4102a20 0x7f32f41a04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32f4100e80 0x7f32f4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 --2- 172.21.15.178:0/3141430996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32f4102060 0x7f32f4199460 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.215+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 >> 172.21.15.178:0/3141430996 conn(0x7f32f40fc400 msgr2=0x7f32f40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.216+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 shutdown_connections 2023-10-16T14:32:10.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.216+0000 7f32fafff700 1 -- 172.21.15.178:0/3141430996 wait complete. 2023-10-16T14:32:10.237 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-10-16T14:32:10.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 -- 172.21.15.178:0/847995395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8102a40 msgr2=0x7f37d8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 --2- 172.21.15.178:0/847995395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8102a40 0x7f37d8109360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f37d000b0a0 tx=0x7f37d000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 -- 172.21.15.178:0/847995395 shutdown_connections 2023-10-16T14:32:10.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 --2- 172.21.15.178:0/847995395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8102a40 0x7f37d8109360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 --2- 172.21.15.178:0/847995395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8100ea0 0x7f37d81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 --2- 172.21.15.178:0/847995395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 0x7f37d8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.441 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 -- 172.21.15.178:0/847995395 >> 172.21.15.178:0/847995395 conn(0x7f37d80fc420 msgr2=0x7f37d80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:10.441 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 -- 172.21.15.178:0/847995395 shutdown_connections 2023-10-16T14:32:10.441 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.436+0000 7f37dddff700 1 -- 172.21.15.178:0/847995395 wait complete. 2023-10-16T14:32:10.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.437+0000 7f37dddff700 1 Processor -- start 2023-10-16T14:32:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.437+0000 7f37dddff700 1 -- start start 2023-10-16T14:32:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 0x7f37d8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 0x7f37d81a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81a0ab0 con 0x7f37d8100ea0 2023-10-16T14:32:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81a0bf0 con 0x7f37d8102a40 2023-10-16T14:32:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37dddff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d81a0d80 con 0x7f37d8102080 2023-10-16T14:32:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57742/0 (socket says 172.21.15.178:57742) 2023-10-16T14:32:10.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 -- 172.21.15.178:0/2416450685 learned_addr learned my addr 172.21.15.178:0/2416450685 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:10.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d6ffd700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 0x7f37d8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d7fff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 0x7f37d81a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 -- 172.21.15.178:0/2416450685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 msgr2=0x7f37d8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 0x7f37d8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 -- 172.21.15.178:0/2416450685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 msgr2=0x7f37d81a04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 0x7f37d81a04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d77fe700 1 -- 172.21.15.178:0/2416450685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0009d00 con 0x7f37d8100ea0 2023-10-16T14:32:10.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.438+0000 7f37d7fff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 0x7f37d81a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:32:10.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37d77fe700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37c800e970 tx=0x7f37c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:10.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8009790 con 0x7f37d8100ea0 2023-10-16T14:32:10.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37c800eec0 con 0x7f37d8100ea0 2023-10-16T14:32:10.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8003cf0 con 0x7f37d8100ea0 2023-10-16T14:32:10.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d81a1060 con 0x7f37d8100ea0 2023-10-16T14:32:10.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.439+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d81a1520 con 0x7f37d8100ea0 2023-10-16T14:32:10.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.440+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d810c760 con 0x7f37d8100ea0 2023-10-16T14:32:10.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.442+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f37c800ccd0 con 0x7f37d8100ea0 2023-10-16T14:32:10.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.444+0000 7f37d4ff9700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 0x7f37c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.444+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f37c8014070 con 0x7f37d8100ea0 2023-10-16T14:32:10.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.444+0000 7f37d6ffd700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 0x7f37c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.445+0000 7f37d6ffd700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 0x7f37c00a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37d8100bc0 tx=0x7f37cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:10.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.448+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f37c80965d0 con 0x7f37d8100ea0 2023-10-16T14:32:10.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.680+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f37d8108a10 con 0x7f37c00a1480 2023-10-16T14:32:10.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.706+0000 7f37d4ff9700 1 -- 172.21.15.178:0/2416450685 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f37d8108a10 con 0x7f37c00a1480 2023-10-16T14:32:10.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 msgr2=0x7f37c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 0x7f37c00a3940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f37d8100bc0 tx=0x7f37cc009260 comp rx=0 tx=0).stop 2023-10-16T14:32:10.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 msgr2=0x7f37d8198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f37c800e970 tx=0x7f37c800ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:10.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 shutdown_connections 2023-10-16T14:32:10.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37c00a1480 0x7f37c00a3940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37d8102a40 0x7f37d81a04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d8100ea0 0x7f37d8198f30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 --2- 172.21.15.178:0/2416450685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37d8102080 0x7f37d8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 >> 172.21.15.178:0/2416450685 conn(0x7f37d80fc420 msgr2=0x7f37d80fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:10.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 shutdown_connections 2023-10-16T14:32:10.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.708+0000 7f37dddff700 1 -- 172.21.15.178:0/2416450685 wait complete. 2023-10-16T14:32:10.730 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-10-16T14:32:10.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 -- 172.21.15.178:0/1270286274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24108db0 msgr2=0x7f7d2410b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 --2- 172.21.15.178:0/1270286274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24108db0 0x7f7d2410b240 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00b0a0 tx=0x7f7d1c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 -- 172.21.15.178:0/1270286274 shutdown_connections 2023-10-16T14:32:10.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 --2- 172.21.15.178:0/1270286274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24108db0 0x7f7d2410b240 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 --2- 172.21.15.178:0/1270286274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24074ef0 0x7f7d24073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 --2- 172.21.15.178:0/1270286274 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 -- 172.21.15.178:0/1270286274 >> 172.21.15.178:0/1270286274 conn(0x7f7d240fc420 msgr2=0x7f7d240fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:10.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 -- 172.21.15.178:0/1270286274 shutdown_connections 2023-10-16T14:32:10.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.928+0000 7f7d297ff700 1 -- 172.21.15.178:0/1270286274 wait complete. 2023-10-16T14:32:10.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.929+0000 7f7d297ff700 1 Processor -- start 2023-10-16T14:32:10.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.929+0000 7f7d297ff700 1 -- start start 2023-10-16T14:32:10.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.929+0000 7f7d297ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d297ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d297ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 0x7f7d241a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d297ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d241a0a80 con 0x7f7d24074ef0 2023-10-16T14:32:10.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d297ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d241a0bc0 con 0x7f7d24073910 2023-10-16T14:32:10.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d297ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d241a0d50 con 0x7f7d24108db0 2023-10-16T14:32:10.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d22ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57752/0 (socket says 172.21.15.178:57752) 2023-10-16T14:32:10.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 -- 172.21.15.178:0/2268732744 learned_addr learned my addr 172.21.15.178:0/2268732744 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:10.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d237fe700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 0x7f7d241a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 -- 172.21.15.178:0/2268732744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 msgr2=0x7f7d241a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 0x7f7d241a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 -- 172.21.15.178:0/2268732744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 msgr2=0x7f7d24198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:10.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:10.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 -- 172.21.15.178:0/2268732744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009d00 con 0x7f7d24074ef0 2023-10-16T14:32:10.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d22ffd700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:10.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d237fe700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 0x7f7d241a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:10.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.930+0000 7f7d227fc700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e970 tx=0x7f7d1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:10.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.931+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18009790 con 0x7f7d24074ef0 2023-10-16T14:32:10.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.931+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d1800eec0 con 0x7f7d24074ef0 2023-10-16T14:32:10.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.931+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d241a0fd0 con 0x7f7d24074ef0 2023-10-16T14:32:10.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.931+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d18003cf0 con 0x7f7d24074ef0 2023-10-16T14:32:10.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.931+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d241a14b0 con 0x7f7d24074ef0 2023-10-16T14:32:10.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.932+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24107460 con 0x7f7d24074ef0 2023-10-16T14:32:10.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.934+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 16) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7d1800ccd0 con 0x7f7d24074ef0 2023-10-16T14:32:10.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.935+0000 7f7d0bfff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 0x7f7d0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:10.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.935+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7d18014070 con 0x7f7d24074ef0 2023-10-16T14:32:10.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.936+0000 7f7d22ffd700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 0x7f7d0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:10.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.936+0000 7f7d22ffd700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 0x7f7d0c0a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d14011490 tx=0x7f7d14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:10.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:10.939+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7d180965b0 con 0x7f7d24074ef0 2023-10-16T14:32:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:10 smithi178 ceph-mon[98495]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:10 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:active 2023-10-16T14:32:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:10 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-16T14:32:11.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:10 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:10 smithi186 ceph-mon[134892]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:10 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:active 2023-10-16T14:32:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:10 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-16T14:32:11.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:10 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:10 smithi187 ceph-mon[135297]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:10 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] up:active 2023-10-16T14:32:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:10 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2023-10-16T14:32:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:10 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:11.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.176+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d2402d2a0 con 0x7f7d0c0a1480 2023-10-16T14:32:11.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.180+0000 7f7d0bfff700 1 -- 172.21.15.178:0/2268732744 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f7d2402d2a0 con 0x7f7d0c0a1480 2023-10-16T14:32:11.183 INFO:teuthology.orchestra.run.smithi178.stdout:/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408 2023-10-16T14:32:11.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 msgr2=0x7f7d0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 0x7f7d0c0a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7d14011490 tx=0x7f7d14009260 comp rx=0 tx=0).stop 2023-10-16T14:32:11.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 msgr2=0x7f7d24199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d1800e970 tx=0x7f7d1800ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:11.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 shutdown_connections 2023-10-16T14:32:11.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7d0c0a1480 0x7f7d0c0a3940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d24108db0 0x7f7d241a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7d24073910 0x7f7d24198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 --2- 172.21.15.178:0/2268732744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d24074ef0 0x7f7d24199440 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 >> 172.21.15.178:0/2268732744 conn(0x7f7d240fc420 msgr2=0x7f7d240fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:11.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 shutdown_connections 2023-10-16T14:32:11.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.183+0000 7f7d297ff700 1 -- 172.21.15.178:0/2268732744 wait complete. 2023-10-16T14:32:11.207 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 -- 172.21.15.178:0/519432340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90073910 msgr2=0x7f7c90073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 --2- 172.21.15.178:0/519432340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90073910 0x7f7c90073d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c780098f0 tx=0x7f7c78009c00 comp rx=0 tx=0).stop 2023-10-16T14:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 -- 172.21.15.178:0/519432340 shutdown_connections 2023-10-16T14:32:11.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 --2- 172.21.15.178:0/519432340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90108dc0 0x7f7c9010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 --2- 172.21.15.178:0/519432340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90074ef0 0x7f7c90073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 --2- 172.21.15.178:0/519432340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90073910 0x7f7c90073d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 -- 172.21.15.178:0/519432340 >> 172.21.15.178:0/519432340 conn(0x7f7c900fc430 msgr2=0x7f7c900fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 -- 172.21.15.178:0/519432340 shutdown_connections 2023-10-16T14:32:11.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.404+0000 7f7c953ff700 1 -- 172.21.15.178:0/519432340 wait complete. 2023-10-16T14:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 Processor -- start 2023-10-16T14:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 -- start start 2023-10-16T14:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90073910 0x7f7c90198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90108dc0 0x7f7c901a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a0960 con 0x7f7c90108dc0 2023-10-16T14:32:11.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a0aa0 con 0x7f7c90073910 2023-10-16T14:32:11.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.405+0000 7f7c953ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c901a0c30 con 0x7f7c90074ef0 2023-10-16T14:32:11.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42544/0 (socket says 172.21.15.178:42544) 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 -- 172.21.15.178:0/2245721445 learned_addr learned my addr 172.21.15.178:0/2245721445 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90108dc0 0x7f7c901a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c86ffd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90073910 0x7f7c90198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 -- 172.21.15.178:0/2245721445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90073910 msgr2=0x7f7c90198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90073910 0x7f7c90198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 -- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90108dc0 msgr2=0x7f7c901a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90108dc0 0x7f7c901a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 -- 172.21.15.178:0/2245721445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c78009520 con 0x7f7c90074ef0 2023-10-16T14:32:11.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8effd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009fd0 tx=0x7f7c7800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:11.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.406+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7801c070 con 0x7f7c90074ef0 2023-10-16T14:32:11.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.407+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c7800f7d0 con 0x7f7c90074ef0 2023-10-16T14:32:11.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.407+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c901a0eb0 con 0x7f7c90074ef0 2023-10-16T14:32:11.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.407+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78003a20 con 0x7f7c90074ef0 2023-10-16T14:32:11.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.407+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c901a1220 con 0x7f7c90074ef0 2023-10-16T14:32:11.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.408+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90107470 con 0x7f7c90074ef0 2023-10-16T14:32:11.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.410+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f7c78020030 con 0x7f7c90074ef0 2023-10-16T14:32:11.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.411+0000 7f7c8cff9700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 0x7f7c7c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.411+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7c780ce370 con 0x7f7c90074ef0 2023-10-16T14:32:11.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.411+0000 7f7c86ffd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 0x7f7c7c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.412+0000 7f7c86ffd700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 0x7f7c7c0a3a80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005fd0 tx=0x7f7c80009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:11.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.413+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7c78097590 con 0x7f7c90074ef0 2023-10-16T14:32:11.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.660+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9002d2e0 con 0x7f7c90074ef0 2023-10-16T14:32:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.661+0000 7f7c8cff9700 1 -- 172.21.15.178:0/2245721445 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f7c78095e60 con 0x7f7c90074ef0 2023-10-16T14:32:11.664 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:11.664 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:09.697134+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:32:11.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 msgr2=0x7f7c7c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 0x7f7c7c0a3a80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005fd0 tx=0x7f7c80009310 comp rx=0 tx=0).stop 2023-10-16T14:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 msgr2=0x7f7c90199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009fd0 tx=0x7f7c7800b790 comp rx=0 tx=0).stop 2023-10-16T14:32:11.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 shutdown_connections 2023-10-16T14:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7c7c0a15c0 0x7f7c7c0a3a80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c90108dc0 0x7f7c901a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7c90073910 0x7f7c90198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 --2- 172.21.15.178:0/2245721445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c90074ef0 0x7f7c90199320 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 >> 172.21.15.178:0/2245721445 conn(0x7f7c900fc430 msgr2=0x7f7c900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 shutdown_connections 2023-10-16T14:32:11.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.664+0000 7f7c953ff700 1 -- 172.21.15.178:0/2245721445 wait complete. 2023-10-16T14:32:11.670 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 25 2023-10-16T14:32:11.685 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:32:11.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/2447687785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102a50 msgr2=0x7f1d04109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/2447687785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102a50 0x7f1d04109370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00b0a0 tx=0x7f1cfc00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/2447687785 shutdown_connections 2023-10-16T14:32:11.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/2447687785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102a50 0x7f1d04109370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/2447687785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04100eb0 0x7f1d041012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/2447687785 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04102090 0x7f1d04102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/2447687785 >> 172.21.15.178:0/2447687785 conn(0x7f1d040fc430 msgr2=0x7f1d040fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:11.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/2447687785 shutdown_connections 2023-10-16T14:32:11.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.881+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/2447687785 wait complete. 2023-10-16T14:32:11.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.882+0000 7f1d0b9ff700 1 Processor -- start 2023-10-16T14:32:11.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 -- start start 2023-10-16T14:32:11.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 0x7f1d04196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04102a50 0x7f1d0419e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419e7a0 con 0x7f1d04102090 2023-10-16T14:32:11.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419e910 con 0x7f1d04100eb0 2023-10-16T14:32:11.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0b9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0419ead0 con 0x7f1d04102a50 2023-10-16T14:32:11.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d08f9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d0979b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 0x7f1d04196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d08f9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57800/0 (socket says 172.21.15.178:57800) 2023-10-16T14:32:11.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d08f9a700 1 -- 172.21.15.178:0/3009745591 learned_addr learned my addr 172.21.15.178:0/3009745591 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:11.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.883+0000 7f1d09f9c700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04102a50 0x7f1d0419e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 -- 172.21.15.178:0/3009745591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04102a50 msgr2=0x7f1d0419e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04102a50 0x7f1d0419e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 -- 172.21.15.178:0/3009745591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 msgr2=0x7f1d04196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:11.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 0x7f1d04196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:11.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 -- 172.21.15.178:0/3009745591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc009d00 con 0x7f1d04102090 2023-10-16T14:32:11.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d0979b700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 0x7f1d04196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:11.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d08f9a700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e970 tx=0x7f1cf400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:11.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4009790 con 0x7f1d04102090 2023-10-16T14:32:11.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cf400c7e0 con 0x7f1d04102090 2023-10-16T14:32:11.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0419edb0 con 0x7f1d04102090 2023-10-16T14:32:11.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4003c70 con 0x7f1d04102090 2023-10-16T14:32:11.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.884+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0419f280 con 0x7f1d04102090 2023-10-16T14:32:11.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.885+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d04107dc0 con 0x7f1d04102090 2023-10-16T14:32:11.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.887+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f1cf400cc50 con 0x7f1d04102090 2023-10-16T14:32:11.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.889+0000 7f1cfa7fc700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 0x7f1cf00a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:11.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.890+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1cf4014070 con 0x7f1d04102090 2023-10-16T14:32:11.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.890+0000 7f1d0979b700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 0x7f1cf00a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:11.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.890+0000 7f1d0979b700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 0x7f1cf00a3b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d04198130 tx=0x7f1d00009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:11.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:11.894+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1cf40964c0 con 0x7f1d04102090 2023-10-16T14:32:12.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:11 smithi178 ceph-mon[98495]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:11 smithi178 ceph-mon[98495]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-16T14:32:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:11 smithi178 ceph-mon[98495]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:11 smithi178 ceph-mon[98495]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-16T14:32:12.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:11 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2245721445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:32:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-mon[134892]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-mon[134892]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-16T14:32:12.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-mon[134892]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-mon[134892]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-16T14:32:12.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2245721445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:32:12.077 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: 2023-10-16T14:32:11.848+0000 7ff450db5700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-10-16T14:32:12.077 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:32:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: 2023-10-16T14:32:11.848+0000 7ff450db5700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-10-16T14:32:12.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.098+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d0410c9d0 con 0x7f1d04102090 2023-10-16T14:32:12.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.100+0000 7f1cfa7fc700 1 -- 172.21.15.178:0/3009745591 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24546 (secure 0 0 0) 0x7f1cf4003dd0 con 0x7f1d04102090 2023-10-16T14:32:12.103 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:12.103 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":89,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:32:07.664080+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:32:12.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.102+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 msgr2=0x7f1cf00a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.102+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 0x7f1cf00a3b00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d04198130 tx=0x7f1d00009260 comp rx=0 tx=0).stop 2023-10-16T14:32:12.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.102+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 msgr2=0x7f1d041971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.102+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e970 tx=0x7f1cf400ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:12.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 shutdown_connections 2023-10-16T14:32:12.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1cf00a1640 0x7f1cf00a3b00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d04102a50 0x7f1d0419e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1d04100eb0 0x7f1d04196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 --2- 172.21.15.178:0/3009745591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d04102090 0x7f1d041971a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 >> 172.21.15.178:0/3009745591 conn(0x7f1d040fc430 msgr2=0x7f1d040fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:12.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 shutdown_connections 2023-10-16T14:32:12.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.103+0000 7f1d0b9ff700 1 -- 172.21.15.178:0/3009745591 wait complete. 2023-10-16T14:32:12.125 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T14:32:12.126 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:32:12.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:11 smithi187 ceph-mon[135297]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:11 smithi187 ceph-mon[135297]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.9 KiB/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-10-16T14:32:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:11 smithi187 ceph-mon[135297]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:11 smithi187 ceph-mon[135297]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-10-16T14:32:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:11 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2245721445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:32:12.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 -- 172.21.15.178:0/1434357480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018102090 msgr2=0x7f1018102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 --2- 172.21.15.178:0/1434357480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018102090 0x7f1018102510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1008009990 tx=0x7f1008009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 -- 172.21.15.178:0/1434357480 shutdown_connections 2023-10-16T14:32:12.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 --2- 172.21.15.178:0/1434357480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102a50 0x7f1018109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 --2- 172.21.15.178:0/1434357480 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018100eb0 0x7f10181012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 --2- 172.21.15.178:0/1434357480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018102090 0x7f1018102510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 -- 172.21.15.178:0/1434357480 >> 172.21.15.178:0/1434357480 conn(0x7f10180fc430 msgr2=0x7f10180fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:12.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 -- 172.21.15.178:0/1434357480 shutdown_connections 2023-10-16T14:32:12.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.322+0000 7f10201ff700 1 -- 172.21.15.178:0/1434357480 wait complete. 2023-10-16T14:32:12.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.323+0000 7f10201ff700 1 Processor -- start 2023-10-16T14:32:12.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.323+0000 7f10201ff700 1 -- start start 2023-10-16T14:32:12.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018100eb0 0x7f1018074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 0x7f10180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1018073f80 con 0x7f1018102090 2023-10-16T14:32:12.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10180740f0 con 0x7f1018102a50 2023-10-16T14:32:12.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f10201ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a0be0 con 0x7f1018100eb0 2023-10-16T14:32:12.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f101d79a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f101d79a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57816/0 (socket says 172.21.15.178:57816) 2023-10-16T14:32:12.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f101d79a700 1 -- 172.21.15.178:0/611225385 learned_addr learned my addr 172.21.15.178:0/611225385 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:12.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f101e79c700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 0x7f10180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.324+0000 7f101df9b700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018100eb0 0x7f1018074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 -- 172.21.15.178:0/611225385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018100eb0 msgr2=0x7f1018074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018100eb0 0x7f1018074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 -- 172.21.15.178:0/611225385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 msgr2=0x7f10180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 0x7f10180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 -- 172.21.15.178:0/611225385 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014009520 con 0x7f1018102090 2023-10-16T14:32:12.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101e79c700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 0x7f10180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:12.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f101d79a700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1008005720 tx=0x7f1008009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:12.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100801d070 con 0x7f1018102090 2023-10-16T14:32:12.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10080095f0 con 0x7f1018102090 2023-10-16T14:32:12.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1008021410 con 0x7f1018102090 2023-10-16T14:32:12.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008017650 con 0x7f1018102090 2023-10-16T14:32:12.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.325+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10181a10f0 con 0x7f1018102090 2023-10-16T14:32:12.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.326+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101810c770 con 0x7f1018102090 2023-10-16T14:32:12.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.328+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f1008021580 con 0x7f1018102090 2023-10-16T14:32:12.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.329+0000 7f100effd700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 0x7f10040a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.329+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10080ce620 con 0x7f1018102090 2023-10-16T14:32:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.330+0000 7f101df9b700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 0x7f10040a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.330+0000 7f101df9b700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 0x7f10040a3a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1018101c00 tx=0x7f1014009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.333+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1008097840 con 0x7f1018102090 2023-10-16T14:32:12.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.543+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10180612f0 con 0x7f10040a15c0 2023-10-16T14:32:12.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.558+0000 7f100effd700 1 -- 172.21.15.178:0/611225385 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423489 (secure 0 0 0) 0x7f10180612f0 con 0x7f10040a15c0 2023-10-16T14:32:12.567 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:12.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 msgr2=0x7f10040a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 0x7f10040a3a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1018101c00 tx=0x7f1014009260 comp rx=0 tx=0).stop 2023-10-16T14:32:12.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 msgr2=0x7f10180732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1008005720 tx=0x7f1008009580 comp rx=0 tx=0).stop 2023-10-16T14:32:12.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 shutdown_connections 2023-10-16T14:32:12.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10040a15c0 0x7f10040a3a80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1018102a50 0x7f10180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1018100eb0 0x7f1018074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 --2- 172.21.15.178:0/611225385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1018102090 0x7f10180732a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 >> 172.21.15.178:0/611225385 conn(0x7f10180fc430 msgr2=0x7f10180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:12.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 shutdown_connections 2023-10-16T14:32:12.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.564+0000 7f10201ff700 1 -- 172.21.15.178:0/611225385 wait complete. 2023-10-16T14:32:12.575 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:32:12.586 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2023-10-16T14:32:10.590581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597538,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":143,"num_read_kb":124,"num_write":280,"num_write_kb":2980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":226,"ondisk_log_size":226,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337556,"kb_used_data":13312,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604268,"statfs":{"total":1151940427776,"available":1151594770432,"internally_reserved":0,"allocated":13631488,"data_stored":7498399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2530,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":50,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.080325+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:31:41.080325+0000","last_peered":"2023-10-16T14:31:41.080325+0000","last_clean":"2023-10-16T14:31:41.080325+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:31:41.080325+0000","last_undegraded":"2023-10-16T14:31:41.080325+0000","last_fullsized":"2023-10-16T14:31:41.080325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.081312+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:31:41.081312+0000","last_peered":"2023-10-16T14:31:41.081312+0000","last_clean":"2023-10-16T14:31:41.081312+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:31:41.081312+0000","last_undegraded":"2023-10-16T14:31:41.081312+0000","last_fullsized":"2023-10-16T14:31:41.081312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079120+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:31:42.079120+0000","last_peered":"2023-10-16T14:31:42.079120+0000","last_clean":"2023-10-16T14:31:42.079120+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:31:42.079120+0000","last_undegraded":"2023-10-16T14:31:42.079120+0000","last_fullsized":"2023-10-16T14:31:42.079120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079634+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:31:42.079634+0000","last_peered":"2023-10-16T14:31:42.079634+0000","last_clean":"2023-10-16T14:31:42.079634+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:31:42.079634+0000","last_undegraded":"2023-10-16T14:31:42.079634+0000","last_fullsized":"2023-10-16T14:31:42.079634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.068798+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:31:41.068798+0000","last_peered":"2023-10-16T14:31:41.068798+0000","last_clean":"2023-10-16T14:31:41.068798+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:31:41.068798+0000","last_undegraded":"2023-10-16T14:31:41.068798+0000","last_fullsized":"2023-10-16T14:31:41.068798+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.076160+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:31:41.076160+0000","last_peered":"2023-10-16T14:31:41.076160+0000","last_clean":"2023-10-16T14:31:41.076160+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:31:41.076160+0000","last_undegraded":"2023-10-16T14:31:41.076160+0000","last_fullsized":"2023-10-16T14:31:41.076160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.075163+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:31:41.075163+0000","last_peered":"2023-10-16T14:31:41.075163+0000","last_clean":"2023-10-16T14:31:41.075163+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:31:41.075163+0000","last_undegraded":"2023-10-16T14:31:41.075163+0000","last_fullsized":"2023-10-16T14:31:41.075163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.087161+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:31:41.087161+0000","last_peered":"2023-10-16T14:31:41.087161+0000","last_clean":"2023-10-16T14:31:41.087161+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:31:41.087161+0000","last_undegraded":"2023-10-16T14:31:41.087161+0000","last_fullsized":"2023-10-16T14:31:41.087161+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.087583+0000","last_change":"2023-10-16T14:31:36.044595+0000","last_active":"2023-10-16T14:31:41.087583+0000","last_peered":"2023-10-16T14:31:41.087583+0000","last_clean":"2023-10-16T14:31:41.087583+0000","last_became_active":"2023-10-16T14:31:36.044074+0000","last_became_peered":"2023-10-16T14:31:36.044074+0000","last_unstale":"2023-10-16T14:31:41.087583+0000","last_undegraded":"2023-10-16T14:31:41.087583+0000","last_fullsized":"2023-10-16T14:31:41.087583+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:53.379173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.080817+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:31:41.080817+0000","last_peered":"2023-10-16T14:31:41.080817+0000","last_clean":"2023-10-16T14:31:41.080817+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:31:41.080817+0000","last_undegraded":"2023-10-16T14:31:41.080817+0000","last_fullsized":"2023-10-16T14:31:41.080817+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708008+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:32:09.708008+0000","last_peered":"2023-10-16T14:32:09.708008+0000","last_clean":"2023-10-16T14:32:09.708008+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:32:09.708008+0000","last_undegraded":"2023-10-16T14:32:09.708008+0000","last_fullsized":"2023-10-16T14:32:09.708008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":19,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079094+0000","last_change":"2023-10-16T14:32:06.668761+0000","last_active":"2023-10-16T14:32:06.668761+0000","last_peered":"2023-10-16T14:31:42.079094+0000","last_clean":"2023-10-16T14:31:42.079094+0000","last_became_active":"2023-10-16T14:31:36.055367+0000","last_became_peered":"2023-10-16T14:31:36.055367+0000","last_unstale":"2023-10-16T14:31:42.079094+0000","last_undegraded":"2023-10-16T14:31:42.079094+0000","last_fullsized":"2023-10-16T14:31:42.079094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,8],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069800+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:31:41.069800+0000","last_peered":"2023-10-16T14:31:41.069800+0000","last_clean":"2023-10-16T14:31:41.069800+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:31:41.069800+0000","last_undegraded":"2023-10-16T14:31:41.069800+0000","last_fullsized":"2023-10-16T14:31:41.069800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:43.830854+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:31:43.830854+0000","last_peered":"2023-10-16T14:31:43.830854+0000","last_clean":"2023-10-16T14:31:43.830854+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:31:43.830854+0000","last_undegraded":"2023-10-16T14:31:43.830854+0000","last_fullsized":"2023-10-16T14:31:43.830854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.084228+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:31:41.084228+0000","last_peered":"2023-10-16T14:31:41.084228+0000","last_clean":"2023-10-16T14:31:41.084228+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:31:41.084228+0000","last_undegraded":"2023-10-16T14:31:41.084228+0000","last_fullsized":"2023-10-16T14:31:41.084228+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689194+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:32:06.689194+0000","last_peered":"2023-10-16T14:32:06.689194+0000","last_clean":"2023-10-16T14:32:06.689194+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:32:06.689194+0000","last_undegraded":"2023-10-16T14:32:06.689194+0000","last_fullsized":"2023-10-16T14:32:06.689194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.572079+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:31:42.572079+0000","last_peered":"2023-10-16T14:31:42.572079+0000","last_clean":"2023-10-16T14:31:42.572079+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:31:42.572079+0000","last_undegraded":"2023-10-16T14:31:42.572079+0000","last_fullsized":"2023-10-16T14:31:42.572079+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.080912+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:31:41.080912+0000","last_peered":"2023-10-16T14:31:41.080912+0000","last_clean":"2023-10-16T14:31:41.080912+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:31:41.080912+0000","last_undegraded":"2023-10-16T14:31:41.080912+0000","last_fullsized":"2023-10-16T14:31:41.080912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.572182+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:31:42.572182+0000","last_peered":"2023-10-16T14:31:42.572182+0000","last_clean":"2023-10-16T14:31:42.572182+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:31:42.572182+0000","last_undegraded":"2023-10-16T14:31:42.572182+0000","last_fullsized":"2023-10-16T14:31:42.572182+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689673+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:32:06.689673+0000","last_peered":"2023-10-16T14:32:06.689673+0000","last_clean":"2023-10-16T14:32:06.689673+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:32:06.689673+0000","last_undegraded":"2023-10-16T14:32:06.689673+0000","last_fullsized":"2023-10-16T14:32:06.689673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690399+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:06.690399+0000","last_peered":"2023-10-16T14:32:06.690399+0000","last_clean":"2023-10-16T14:32:06.690399+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:06.690399+0000","last_undegraded":"2023-10-16T14:32:06.690399+0000","last_fullsized":"2023-10-16T14:32:06.690399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069049+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:31:41.069049+0000","last_peered":"2023-10-16T14:31:41.069049+0000","last_clean":"2023-10-16T14:31:41.069049+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:31:41.069049+0000","last_undegraded":"2023-10-16T14:31:41.069049+0000","last_fullsized":"2023-10-16T14:31:41.069049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.572829+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:31:42.572829+0000","last_peered":"2023-10-16T14:31:42.572829+0000","last_clean":"2023-10-16T14:31:42.572829+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:31:42.572829+0000","last_undegraded":"2023-10-16T14:31:42.572829+0000","last_fullsized":"2023-10-16T14:31:42.572829+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.709111+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:32:09.709111+0000","last_peered":"2023-10-16T14:32:09.709111+0000","last_clean":"2023-10-16T14:32:09.709111+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:32:09.709111+0000","last_undegraded":"2023-10-16T14:32:09.709111+0000","last_fullsized":"2023-10-16T14:32:09.709111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:43.705376+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:31:43.705376+0000","last_peered":"2023-10-16T14:31:43.705376+0000","last_clean":"2023-10-16T14:31:43.705376+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:31:43.705376+0000","last_undegraded":"2023-10-16T14:31:43.705376+0000","last_fullsized":"2023-10-16T14:31:43.705376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682374+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:07.682374+0000","last_peered":"2023-10-16T14:32:07.682374+0000","last_clean":"2023-10-16T14:32:07.682374+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:07.682374+0000","last_undegraded":"2023-10-16T14:32:07.682374+0000","last_fullsized":"2023-10-16T14:32:07.682374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.086947+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:31:41.086947+0000","last_peered":"2023-10-16T14:31:41.086947+0000","last_clean":"2023-10-16T14:31:41.086947+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:31:41.086947+0000","last_undegraded":"2023-10-16T14:31:41.086947+0000","last_fullsized":"2023-10-16T14:31:41.086947+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079402+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:31:42.079402+0000","last_peered":"2023-10-16T14:31:42.079402+0000","last_clean":"2023-10-16T14:31:42.079402+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:31:42.079402+0000","last_undegraded":"2023-10-16T14:31:42.079402+0000","last_fullsized":"2023-10-16T14:31:42.079402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:43.705157+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:31:43.705157+0000","last_peered":"2023-10-16T14:31:43.705157+0000","last_clean":"2023-10-16T14:31:43.705157+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:31:43.705157+0000","last_undegraded":"2023-10-16T14:31:43.705157+0000","last_fullsized":"2023-10-16T14:31:43.705157+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674404+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:06.674404+0000","last_peered":"2023-10-16T14:32:06.674404+0000","last_clean":"2023-10-16T14:32:06.674404+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:06.674404+0000","last_undegraded":"2023-10-16T14:32:06.674404+0000","last_fullsized":"2023-10-16T14:32:06.674404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079202+0000","last_change":"2023-10-16T14:32:06.668507+0000","last_active":"2023-10-16T14:32:06.668507+0000","last_peered":"2023-10-16T14:31:42.079202+0000","last_clean":"2023-10-16T14:31:42.079202+0000","last_became_active":"2023-10-16T14:31:37.039674+0000","last_became_peered":"2023-10-16T14:31:37.039674+0000","last_unstale":"2023-10-16T14:31:42.079202+0000","last_undegraded":"2023-10-16T14:31:42.079202+0000","last_fullsized":"2023-10-16T14:31:42.079202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:43.705216+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:31:43.705216+0000","last_peered":"2023-10-16T14:31:43.705216+0000","last_clean":"2023-10-16T14:31:43.705216+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:31:43.705216+0000","last_undegraded":"2023-10-16T14:31:43.705216+0000","last_fullsized":"2023-10-16T14:31:43.705216+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069799+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:31:41.069799+0000","last_peered":"2023-10-16T14:31:41.069799+0000","last_clean":"2023-10-16T14:31:41.069799+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:31:41.069799+0000","last_undegraded":"2023-10-16T14:31:41.069799+0000","last_fullsized":"2023-10-16T14:31:41.069799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675184+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:06.675184+0000","last_peered":"2023-10-16T14:32:06.675184+0000","last_clean":"2023-10-16T14:32:06.675184+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:06.675184+0000","last_undegraded":"2023-10-16T14:32:06.675184+0000","last_fullsized":"2023-10-16T14:32:06.675184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069653+0000","last_change":"2023-10-16T14:31:37.046094+0000","last_active":"2023-10-16T14:31:41.069653+0000","last_peered":"2023-10-16T14:31:41.069653+0000","last_clean":"2023-10-16T14:31:41.069653+0000","last_became_active":"2023-10-16T14:31:37.045731+0000","last_became_peered":"2023-10-16T14:31:37.045731+0000","last_unstale":"2023-10-16T14:31:41.069653+0000","last_undegraded":"2023-10-16T14:31:41.069653+0000","last_fullsized":"2023-10-16T14:31:41.069653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:21:31.639533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687172+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:07.687172+0000","last_peered":"2023-10-16T14:32:07.687172+0000","last_clean":"2023-10-16T14:32:07.687172+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:07.687172+0000","last_undegraded":"2023-10-16T14:32:07.687172+0000","last_fullsized":"2023-10-16T14:32:07.687172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069681+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:31:41.069681+0000","last_peered":"2023-10-16T14:31:41.069681+0000","last_clean":"2023-10-16T14:31:41.069681+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:31:41.069681+0000","last_undegraded":"2023-10-16T14:31:41.069681+0000","last_fullsized":"2023-10-16T14:31:41.069681+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.087066+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:31:41.087066+0000","last_peered":"2023-10-16T14:31:41.087066+0000","last_clean":"2023-10-16T14:31:41.087066+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:31:41.087066+0000","last_undegraded":"2023-10-16T14:31:41.087066+0000","last_fullsized":"2023-10-16T14:31:41.087066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.080425+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:31:41.080425+0000","last_peered":"2023-10-16T14:31:41.080425+0000","last_clean":"2023-10-16T14:31:41.080425+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:31:41.080425+0000","last_undegraded":"2023-10-16T14:31:41.080425+0000","last_fullsized":"2023-10-16T14:31:41.080425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.572694+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:31:42.572694+0000","last_peered":"2023-10-16T14:31:42.572694+0000","last_clean":"2023-10-16T14:31:42.572694+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:31:42.572694+0000","last_undegraded":"2023-10-16T14:31:42.572694+0000","last_fullsized":"2023-10-16T14:31:42.572694+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672544+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:06.672544+0000","last_peered":"2023-10-16T14:32:06.672544+0000","last_clean":"2023-10-16T14:32:06.672544+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:06.672544+0000","last_undegraded":"2023-10-16T14:32:06.672544+0000","last_fullsized":"2023-10-16T14:32:06.672544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673284+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:06.673284+0000","last_peered":"2023-10-16T14:32:06.673284+0000","last_clean":"2023-10-16T14:32:06.673284+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:06.673284+0000","last_undegraded":"2023-10-16T14:32:06.673284+0000","last_fullsized":"2023-10-16T14:32:06.673284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673588+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:06.673588+0000","last_peered":"2023-10-16T14:32:06.673588+0000","last_clean":"2023-10-16T14:32:06.673588+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:06.673588+0000","last_undegraded":"2023-10-16T14:32:06.673588+0000","last_fullsized":"2023-10-16T14:32:06.673588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:43.830491+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:31:43.830491+0000","last_peered":"2023-10-16T14:31:43.830491+0000","last_clean":"2023-10-16T14:31:43.830491+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:31:43.830491+0000","last_undegraded":"2023-10-16T14:31:43.830491+0000","last_fullsized":"2023-10-16T14:31:43.830491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.068724+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:31:41.068724+0000","last_peered":"2023-10-16T14:31:41.068724+0000","last_clean":"2023-10-16T14:31:41.068724+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:31:41.068724+0000","last_undegraded":"2023-10-16T14:31:41.068724+0000","last_fullsized":"2023-10-16T14:31:41.068724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.076318+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:31:41.076318+0000","last_peered":"2023-10-16T14:31:41.076318+0000","last_clean":"2023-10-16T14:31:41.076318+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:31:41.076318+0000","last_undegraded":"2023-10-16T14:31:41.076318+0000","last_fullsized":"2023-10-16T14:31:41.076318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.069778+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:31:41.069778+0000","last_peered":"2023-10-16T14:31:41.069778+0000","last_clean":"2023-10-16T14:31:41.069778+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:31:41.069778+0000","last_undegraded":"2023-10-16T14:31:41.069778+0000","last_fullsized":"2023-10-16T14:31:41.069778+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-10-16T14:31:41.068796+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:31:41.068796+0000","last_peered":"2023-10-16T14:31:41.068796+0000","last_clean":"2023-10-16T14:31:41.068796+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:31:41.068796+0000","last_undegraded":"2023-10-16T14:31:41.068796+0000","last_fullsized":"2023-10-16T14:31:41.068796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7170,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":99,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":19128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89,"ondisk_log_size":89,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73899999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.82199999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1067930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73199999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":11,"last update":"Mon Oct 16 14:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68700000000000006}]}]},{"osd":9,"up_from":66,"seq":283467841563,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":463836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.746}]},{"osd":5,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90600000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.88},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":482745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.751,"15min":0.751},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625}]},{"osd":4,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90500000000000003,"5min":1.087,"15min":1.087},"last":0.52700000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84299999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.878}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1568768,"data_stored":1063732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73999999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":462923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.32500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.62,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.502,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85999999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.93999999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72299999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.879}]},{"osd":4,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0349999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.65500000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.232,"15min":1.232},"last":0.997}]},{"osd":8,"last update":"Mon Oct 16 14:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.107},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.014}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.92900000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":1.081},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":1.0680000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.47499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95999999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.436,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.181,"15min":1.181},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.754}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":462740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.42699999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0389999999999999,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.58899999999999997,"15min":0.56100000000000005},"min":{"1min":0.437,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.026,"5min":1.141,"15min":1.141},"last":0.56000000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94799999999999995,"5min":1.202,"15min":1.202},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95799999999999996,"5min":1.101,"15min":1.101},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57599999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":1.127,"15min":1.127},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.80000000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.70499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1020000000000001,"5min":1.274,"15min":1.274},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.67900000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.95399999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.91700000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.877}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.43099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":1.022,"15min":1.022},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.754,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:31:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.876}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.496}]},{"osd":4,"last update":"Mon Oct 16 14:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.747},"min":{"1min":0.59699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":482709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"min":{"1min":0.317,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.37}]},{"osd":2,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.88,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.872},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.873}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1078668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":1.425,"15min":1.425},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.221,"15min":0.221},"max":{"1min":0.52000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84099999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.628}]},{"osd":7,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.196,"15min":1.196},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77400000000000002,"5min":0.878,"15min":0.878},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:32:12.598 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:32:12.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.822+0000 7f26f8bff700 1 -- 172.21.15.178:0/3068863826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102090 msgr2=0x7f26f4102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.822+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3068863826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102090 0x7f26f4102510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009990 tx=0x7f26dc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 -- 172.21.15.178:0/3068863826 shutdown_connections 2023-10-16T14:32:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3068863826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102a50 0x7f26f4109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3068863826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3068863826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102090 0x7f26f4102510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 -- 172.21.15.178:0/3068863826 >> 172.21.15.178:0/3068863826 conn(0x7f26f40fc430 msgr2=0x7f26f40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 -- 172.21.15.178:0/3068863826 shutdown_connections 2023-10-16T14:32:12.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.823+0000 7f26f8bff700 1 -- 172.21.15.178:0/3068863826 wait complete. 2023-10-16T14:32:12.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 Processor -- start 2023-10-16T14:32:12.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 -- start start 2023-10-16T14:32:12.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 0x7f26f4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 0x7f26f41a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a0900 con 0x7f26f4102a50 2023-10-16T14:32:12.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a0a70 con 0x7f26f4100eb0 2023-10-16T14:32:12.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f8bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f41a0c30 con 0x7f26f4102090 2023-10-16T14:32:12.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26f2d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 0x7f26f41a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.824+0000 7f26ea59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26f259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 0x7f26f4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48682/0 (socket says 172.21.15.178:48682) 2023-10-16T14:32:12.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 -- 172.21.15.178:0/3186394266 learned_addr learned my addr 172.21.15.178:0/3186394266 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 -- 172.21.15.178:0/3186394266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 msgr2=0x7f26f4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 0x7f26f4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 -- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 msgr2=0x7f26f41a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 0x7f26f41a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:12.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 -- 172.21.15.178:0/3186394266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc0095f0 con 0x7f26f4100eb0 2023-10-16T14:32:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26f2d9d700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 0x7f26f41a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26f259c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 0x7f26f4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ea59c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e970 tx=0x7f26e400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4009790 con 0x7f26f4100eb0 2023-10-16T14:32:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26e400c7e0 con 0x7f26f4100eb0 2023-10-16T14:32:12.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.825+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f41a0f10 con 0x7f26f4100eb0 2023-10-16T14:32:12.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.826+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4003c70 con 0x7f26f4100eb0 2023-10-16T14:32:12.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.826+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f41a1330 con 0x7f26f4100eb0 2023-10-16T14:32:12.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.827+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f410c770 con 0x7f26f4100eb0 2023-10-16T14:32:12.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.828+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f26e400cc50 con 0x7f26f4100eb0 2023-10-16T14:32:12.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.830+0000 7f26ebfff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 0x7f26e00a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:12.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.830+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f26e4014070 con 0x7f26f4100eb0 2023-10-16T14:32:12.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.830+0000 7f26f259c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 0x7f26e00a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:12.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.830+0000 7f26f259c700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 0x7f26e00a3a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00b2e0 tx=0x7f26dc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:12.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:12.832+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f26e4096510 con 0x7f26f4100eb0 2023-10-16T14:32:13.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.031+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26f41090d0 con 0x7f26e00a15c0 2023-10-16T14:32:13.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.043+0000 7f26ebfff700 1 -- 172.21.15.178:0/3186394266 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423578 (secure 0 0 0) 0x7f26f41090d0 con 0x7f26e00a15c0 2023-10-16T14:32:13.052 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:13.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 msgr2=0x7f26e00a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 0x7f26e00a3a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00b2e0 tx=0x7f26dc005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 msgr2=0x7f26f4198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e970 tx=0x7f26e400ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:13.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 shutdown_connections 2023-10-16T14:32:13.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f26e00a15c0 0x7f26e00a3a80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26f4102a50 0x7f26f41a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f26f4100eb0 0x7f26f4198dc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 --2- 172.21.15.178:0/3186394266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26f4102090 0x7f26f4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 >> 172.21.15.178:0/3186394266 conn(0x7f26f40fc430 msgr2=0x7f26f41079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:13.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 shutdown_connections 2023-10-16T14:32:13.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.049+0000 7f26f8bff700 1 -- 172.21.15.178:0/3186394266 wait complete. 2023-10-16T14:32:13.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:12 smithi178 ceph-mon[98495]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:13.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:12 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3009745591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:13.058 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:32:13.070 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-10-16T14:32:12.591278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597628,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":298,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337700,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604124,"statfs":{"total":1151940427776,"available":1151594622976,"internally_reserved":0,"allocated":13783040,"data_stored":7565557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2620,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":68,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079120+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:31:42.079120+0000","last_peered":"2023-10-16T14:31:42.079120+0000","last_clean":"2023-10-16T14:31:42.079120+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:31:42.079120+0000","last_undegraded":"2023-10-16T14:31:42.079120+0000","last_fullsized":"2023-10-16T14:31:42.079120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079634+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:31:42.079634+0000","last_peered":"2023-10-16T14:31:42.079634+0000","last_clean":"2023-10-16T14:31:42.079634+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:31:42.079634+0000","last_undegraded":"2023-10-16T14:31:42.079634+0000","last_fullsized":"2023-10-16T14:31:42.079634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708008+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:32:09.708008+0000","last_peered":"2023-10-16T14:32:09.708008+0000","last_clean":"2023-10-16T14:32:09.708008+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:32:09.708008+0000","last_undegraded":"2023-10-16T14:32:09.708008+0000","last_fullsized":"2023-10-16T14:32:09.708008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":19,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079094+0000","last_change":"2023-10-16T14:32:06.668761+0000","last_active":"2023-10-16T14:32:06.668761+0000","last_peered":"2023-10-16T14:31:42.079094+0000","last_clean":"2023-10-16T14:31:42.079094+0000","last_became_active":"2023-10-16T14:31:36.055367+0000","last_became_peered":"2023-10-16T14:31:36.055367+0000","last_unstale":"2023-10-16T14:31:42.079094+0000","last_undegraded":"2023-10-16T14:31:42.079094+0000","last_fullsized":"2023-10-16T14:31:42.079094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,8],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689194+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:32:06.689194+0000","last_peered":"2023-10-16T14:32:06.689194+0000","last_clean":"2023-10-16T14:32:06.689194+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:32:06.689194+0000","last_undegraded":"2023-10-16T14:32:06.689194+0000","last_fullsized":"2023-10-16T14:32:06.689194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671298+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:32:06.671298+0000","last_peered":"2023-10-16T14:32:06.671298+0000","last_clean":"2023-10-16T14:32:06.671298+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:32:06.671298+0000","last_undegraded":"2023-10-16T14:32:06.671298+0000","last_fullsized":"2023-10-16T14:32:06.671298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689673+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:32:06.689673+0000","last_peered":"2023-10-16T14:32:06.689673+0000","last_clean":"2023-10-16T14:32:06.689673+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:32:06.689673+0000","last_undegraded":"2023-10-16T14:32:06.689673+0000","last_fullsized":"2023-10-16T14:32:06.689673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690399+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:06.690399+0000","last_peered":"2023-10-16T14:32:06.690399+0000","last_clean":"2023-10-16T14:32:06.690399+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:06.690399+0000","last_undegraded":"2023-10-16T14:32:06.690399+0000","last_fullsized":"2023-10-16T14:32:06.690399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.709111+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:32:09.709111+0000","last_peered":"2023-10-16T14:32:09.709111+0000","last_clean":"2023-10-16T14:32:09.709111+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:32:09.709111+0000","last_undegraded":"2023-10-16T14:32:09.709111+0000","last_fullsized":"2023-10-16T14:32:09.709111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682374+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:07.682374+0000","last_peered":"2023-10-16T14:32:07.682374+0000","last_clean":"2023-10-16T14:32:07.682374+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:07.682374+0000","last_undegraded":"2023-10-16T14:32:07.682374+0000","last_fullsized":"2023-10-16T14:32:07.682374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079402+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:31:42.079402+0000","last_peered":"2023-10-16T14:31:42.079402+0000","last_clean":"2023-10-16T14:31:42.079402+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:31:42.079402+0000","last_undegraded":"2023-10-16T14:31:42.079402+0000","last_fullsized":"2023-10-16T14:31:42.079402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674404+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:06.674404+0000","last_peered":"2023-10-16T14:32:06.674404+0000","last_clean":"2023-10-16T14:32:06.674404+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:06.674404+0000","last_undegraded":"2023-10-16T14:32:06.674404+0000","last_fullsized":"2023-10-16T14:32:06.674404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079202+0000","last_change":"2023-10-16T14:32:06.668507+0000","last_active":"2023-10-16T14:32:06.668507+0000","last_peered":"2023-10-16T14:31:42.079202+0000","last_clean":"2023-10-16T14:31:42.079202+0000","last_became_active":"2023-10-16T14:31:37.039674+0000","last_became_peered":"2023-10-16T14:31:37.039674+0000","last_unstale":"2023-10-16T14:31:42.079202+0000","last_undegraded":"2023-10-16T14:31:42.079202+0000","last_fullsized":"2023-10-16T14:31:42.079202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708749+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:32:09.708749+0000","last_peered":"2023-10-16T14:32:09.708749+0000","last_clean":"2023-10-16T14:32:09.708749+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:32:09.708749+0000","last_undegraded":"2023-10-16T14:32:09.708749+0000","last_fullsized":"2023-10-16T14:32:09.708749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675184+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:06.675184+0000","last_peered":"2023-10-16T14:32:06.675184+0000","last_clean":"2023-10-16T14:32:06.675184+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:06.675184+0000","last_undegraded":"2023-10-16T14:32:06.675184+0000","last_fullsized":"2023-10-16T14:32:06.675184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687172+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:07.687172+0000","last_peered":"2023-10-16T14:32:07.687172+0000","last_clean":"2023-10-16T14:32:07.687172+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:07.687172+0000","last_undegraded":"2023-10-16T14:32:07.687172+0000","last_fullsized":"2023-10-16T14:32:07.687172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703933+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:08.703933+0000","last_peered":"2023-10-16T14:32:08.703933+0000","last_clean":"2023-10-16T14:32:08.703933+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:08.703933+0000","last_undegraded":"2023-10-16T14:32:08.703933+0000","last_fullsized":"2023-10-16T14:32:08.703933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672544+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:06.672544+0000","last_peered":"2023-10-16T14:32:06.672544+0000","last_clean":"2023-10-16T14:32:06.672544+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:06.672544+0000","last_undegraded":"2023-10-16T14:32:06.672544+0000","last_fullsized":"2023-10-16T14:32:06.672544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673284+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:06.673284+0000","last_peered":"2023-10-16T14:32:06.673284+0000","last_clean":"2023-10-16T14:32:06.673284+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:06.673284+0000","last_undegraded":"2023-10-16T14:32:06.673284+0000","last_fullsized":"2023-10-16T14:32:06.673284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673588+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:06.673588+0000","last_peered":"2023-10-16T14:32:06.673588+0000","last_clean":"2023-10-16T14:32:06.673588+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:06.673588+0000","last_undegraded":"2023-10-16T14:32:06.673588+0000","last_fullsized":"2023-10-16T14:32:06.673588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7260,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":117,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":20842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73899999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.82199999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1067930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73199999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":11,"last update":"Mon Oct 16 14:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68700000000000006}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.746}]},{"osd":5,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90600000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.88},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":482745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.751,"15min":0.751},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625}]},{"osd":4,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90500000000000003,"5min":1.087,"15min":1.087},"last":0.52700000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84299999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.878}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1568768,"data_stored":1063732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73999999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.32500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.62,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.502,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85999999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.495},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0349999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.65500000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.232,"15min":1.232},"last":0.81699999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.107},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.872},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.47499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95999999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.436,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.181,"15min":1.181},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.754}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.622},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94799999999999995,"5min":1.202,"15min":1.202},"last":0.88}]},{"osd":3,"last update":"Mon Oct 16 14:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72599999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95799999999999996,"5min":1.101,"15min":1.101},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":1.127,"15min":1.127},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.753}]},{"osd":8,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.70499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1020000000000001,"5min":1.274,"15min":1.274},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.56899999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Mon Oct 16 14:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.43099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":1.022,"15min":1.022},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.754,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:31:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.876}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.496}]},{"osd":4,"last update":"Mon Oct 16 14:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.747},"min":{"1min":0.59699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":482709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"min":{"1min":0.317,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.37}]},{"osd":2,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.88,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.872},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.873}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1078668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":1.425,"15min":1.425},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.221,"15min":0.221},"max":{"1min":0.52000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84099999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.628}]},{"osd":7,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.196,"15min":1.196},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77400000000000002,"5min":0.878,"15min":0.878},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:32:13.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:32:13.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:12 smithi186 ceph-mon[134892]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:13.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:12 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3009745591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:13.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:12 smithi187 ceph-mon[135297]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:32:13.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:12 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3009745591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:13.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/3255577519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 msgr2=0x7fbb541012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/3255577519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb541012c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbb44009990 tx=0x7fbb44009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/3255577519 shutdown_connections 2023-10-16T14:32:13.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/3255577519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102a40 0x7fbb54109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/3255577519 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb541012c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/3255577519 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102080 0x7fbb54102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/3255577519 >> 172.21.15.178:0/3255577519 conn(0x7fbb540fc420 msgr2=0x7fbb540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:13.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/3255577519 shutdown_connections 2023-10-16T14:32:13.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.306+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/3255577519 wait complete. 2023-10-16T14:32:13.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.307+0000 7fbb5a1ff700 1 Processor -- start 2023-10-16T14:32:13.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.307+0000 7fbb5a1ff700 1 -- start start 2023-10-16T14:32:13.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.307+0000 7fbb5a1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb54198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.307+0000 7fbb5a1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.307+0000 7fbb5a1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 0x7fbb541a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb5a1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a08e0 con 0x7fbb54102a40 2023-10-16T14:32:13.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb5a1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a0a20 con 0x7fbb54100ea0 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb5a1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541a0bb0 con 0x7fbb54102080 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42606/0 (socket says 172.21.15.178:42606) 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 -- 172.21.15.178:0/1679741164 learned_addr learned my addr 172.21.15.178:0/1679741164 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb53fff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 0x7fbb541a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb537fe700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb54198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 -- 172.21.15.178:0/1679741164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 msgr2=0x7fbb54198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb54198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 -- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 msgr2=0x7fbb541a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 0x7fbb541a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 -- 172.21.15.178:0/1679741164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb440095f0 con 0x7fbb54102080 2023-10-16T14:32:13.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb53fff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 0x7fbb541a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:13.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.308+0000 7fbb52ffd700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e970 tx=0x7fbb4800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:13.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.309+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48009790 con 0x7fbb54102080 2023-10-16T14:32:13.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.309+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb4800c7e0 con 0x7fbb54102080 2023-10-16T14:32:13.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.309+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541a0e90 con 0x7fbb54102080 2023-10-16T14:32:13.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.309+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48003c70 con 0x7fbb54102080 2023-10-16T14:32:13.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.309+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a1310 con 0x7fbb54102080 2023-10-16T14:32:13.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.310+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5410c760 con 0x7fbb54102080 2023-10-16T14:32:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.312+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fbb4800cc50 con 0x7fbb54102080 2023-10-16T14:32:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.313+0000 7fbb50ff9700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 0x7fbb3c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.313+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbb48014070 con 0x7fbb54102080 2023-10-16T14:32:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.313+0000 7fbb537fe700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 0x7fbb3c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.314+0000 7fbb537fe700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 0x7fbb3c0a3a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb4400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:13.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.315+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbb48096670 con 0x7fbb54102080 2023-10-16T14:32:13.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.512+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb54108a10 con 0x7fbb3c0a15c0 2023-10-16T14:32:13.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.525+0000 7fbb50ff9700 1 -- 172.21.15.178:0/1679741164 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423578 (secure 0 0 0) 0x7fbb54108a10 con 0x7fbb3c0a15c0 2023-10-16T14:32:13.533 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:13.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 msgr2=0x7fbb3c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 0x7fbb3c0a3a80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbb44000c00 tx=0x7fbb4400b260 comp rx=0 tx=0).stop 2023-10-16T14:32:13.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 msgr2=0x7fbb541992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800e970 tx=0x7fbb4800ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 shutdown_connections 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbb3c0a15c0 0x7fbb3c0a3a80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbb54102a40 0x7fbb541a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbb54100ea0 0x7fbb54198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 --2- 172.21.15.178:0/1679741164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb54102080 0x7fbb541992a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.530+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 >> 172.21.15.178:0/1679741164 conn(0x7fbb540fc420 msgr2=0x7fbb540fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:13.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.531+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 shutdown_connections 2023-10-16T14:32:13.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.531+0000 7fbb5a1ff700 1 -- 172.21.15.178:0/1679741164 wait complete. 2023-10-16T14:32:13.540 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:32:13.552 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-10-16T14:32:12.591278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597628,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":298,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337700,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604124,"statfs":{"total":1151940427776,"available":1151594622976,"internally_reserved":0,"allocated":13783040,"data_stored":7565557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2620,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":68,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079120+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:31:42.079120+0000","last_peered":"2023-10-16T14:31:42.079120+0000","last_clean":"2023-10-16T14:31:42.079120+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:31:42.079120+0000","last_undegraded":"2023-10-16T14:31:42.079120+0000","last_fullsized":"2023-10-16T14:31:42.079120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079634+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:31:42.079634+0000","last_peered":"2023-10-16T14:31:42.079634+0000","last_clean":"2023-10-16T14:31:42.079634+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:31:42.079634+0000","last_undegraded":"2023-10-16T14:31:42.079634+0000","last_fullsized":"2023-10-16T14:31:42.079634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708008+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:32:09.708008+0000","last_peered":"2023-10-16T14:32:09.708008+0000","last_clean":"2023-10-16T14:32:09.708008+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:32:09.708008+0000","last_undegraded":"2023-10-16T14:32:09.708008+0000","last_fullsized":"2023-10-16T14:32:09.708008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":19,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079094+0000","last_change":"2023-10-16T14:32:06.668761+0000","last_active":"2023-10-16T14:32:06.668761+0000","last_peered":"2023-10-16T14:31:42.079094+0000","last_clean":"2023-10-16T14:31:42.079094+0000","last_became_active":"2023-10-16T14:31:36.055367+0000","last_became_peered":"2023-10-16T14:31:36.055367+0000","last_unstale":"2023-10-16T14:31:42.079094+0000","last_undegraded":"2023-10-16T14:31:42.079094+0000","last_fullsized":"2023-10-16T14:31:42.079094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,8],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689194+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:32:06.689194+0000","last_peered":"2023-10-16T14:32:06.689194+0000","last_clean":"2023-10-16T14:32:06.689194+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:32:06.689194+0000","last_undegraded":"2023-10-16T14:32:06.689194+0000","last_fullsized":"2023-10-16T14:32:06.689194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671298+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:32:06.671298+0000","last_peered":"2023-10-16T14:32:06.671298+0000","last_clean":"2023-10-16T14:32:06.671298+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:32:06.671298+0000","last_undegraded":"2023-10-16T14:32:06.671298+0000","last_fullsized":"2023-10-16T14:32:06.671298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689673+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:32:06.689673+0000","last_peered":"2023-10-16T14:32:06.689673+0000","last_clean":"2023-10-16T14:32:06.689673+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:32:06.689673+0000","last_undegraded":"2023-10-16T14:32:06.689673+0000","last_fullsized":"2023-10-16T14:32:06.689673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690399+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:06.690399+0000","last_peered":"2023-10-16T14:32:06.690399+0000","last_clean":"2023-10-16T14:32:06.690399+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:06.690399+0000","last_undegraded":"2023-10-16T14:32:06.690399+0000","last_fullsized":"2023-10-16T14:32:06.690399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.709111+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:32:09.709111+0000","last_peered":"2023-10-16T14:32:09.709111+0000","last_clean":"2023-10-16T14:32:09.709111+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:32:09.709111+0000","last_undegraded":"2023-10-16T14:32:09.709111+0000","last_fullsized":"2023-10-16T14:32:09.709111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682374+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:07.682374+0000","last_peered":"2023-10-16T14:32:07.682374+0000","last_clean":"2023-10-16T14:32:07.682374+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:07.682374+0000","last_undegraded":"2023-10-16T14:32:07.682374+0000","last_fullsized":"2023-10-16T14:32:07.682374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079402+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:31:42.079402+0000","last_peered":"2023-10-16T14:31:42.079402+0000","last_clean":"2023-10-16T14:31:42.079402+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:31:42.079402+0000","last_undegraded":"2023-10-16T14:31:42.079402+0000","last_fullsized":"2023-10-16T14:31:42.079402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674404+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:06.674404+0000","last_peered":"2023-10-16T14:32:06.674404+0000","last_clean":"2023-10-16T14:32:06.674404+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:06.674404+0000","last_undegraded":"2023-10-16T14:32:06.674404+0000","last_fullsized":"2023-10-16T14:32:06.674404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079202+0000","last_change":"2023-10-16T14:32:06.668507+0000","last_active":"2023-10-16T14:32:06.668507+0000","last_peered":"2023-10-16T14:31:42.079202+0000","last_clean":"2023-10-16T14:31:42.079202+0000","last_became_active":"2023-10-16T14:31:37.039674+0000","last_became_peered":"2023-10-16T14:31:37.039674+0000","last_unstale":"2023-10-16T14:31:42.079202+0000","last_undegraded":"2023-10-16T14:31:42.079202+0000","last_fullsized":"2023-10-16T14:31:42.079202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708749+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:32:09.708749+0000","last_peered":"2023-10-16T14:32:09.708749+0000","last_clean":"2023-10-16T14:32:09.708749+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:32:09.708749+0000","last_undegraded":"2023-10-16T14:32:09.708749+0000","last_fullsized":"2023-10-16T14:32:09.708749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675184+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:06.675184+0000","last_peered":"2023-10-16T14:32:06.675184+0000","last_clean":"2023-10-16T14:32:06.675184+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:06.675184+0000","last_undegraded":"2023-10-16T14:32:06.675184+0000","last_fullsized":"2023-10-16T14:32:06.675184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687172+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:07.687172+0000","last_peered":"2023-10-16T14:32:07.687172+0000","last_clean":"2023-10-16T14:32:07.687172+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:07.687172+0000","last_undegraded":"2023-10-16T14:32:07.687172+0000","last_fullsized":"2023-10-16T14:32:07.687172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703933+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:08.703933+0000","last_peered":"2023-10-16T14:32:08.703933+0000","last_clean":"2023-10-16T14:32:08.703933+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:08.703933+0000","last_undegraded":"2023-10-16T14:32:08.703933+0000","last_fullsized":"2023-10-16T14:32:08.703933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672544+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:06.672544+0000","last_peered":"2023-10-16T14:32:06.672544+0000","last_clean":"2023-10-16T14:32:06.672544+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:06.672544+0000","last_undegraded":"2023-10-16T14:32:06.672544+0000","last_fullsized":"2023-10-16T14:32:06.672544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673284+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:06.673284+0000","last_peered":"2023-10-16T14:32:06.673284+0000","last_clean":"2023-10-16T14:32:06.673284+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:06.673284+0000","last_undegraded":"2023-10-16T14:32:06.673284+0000","last_fullsized":"2023-10-16T14:32:06.673284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673588+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:06.673588+0000","last_peered":"2023-10-16T14:32:06.673588+0000","last_clean":"2023-10-16T14:32:06.673588+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:06.673588+0000","last_undegraded":"2023-10-16T14:32:06.673588+0000","last_fullsized":"2023-10-16T14:32:06.673588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7260,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":117,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":20842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73899999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.82199999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1067930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73199999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":11,"last update":"Mon Oct 16 14:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68700000000000006}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.746}]},{"osd":5,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90600000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.88},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":482745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.751,"15min":0.751},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625}]},{"osd":4,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90500000000000003,"5min":1.087,"15min":1.087},"last":0.52700000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84299999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.878}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1568768,"data_stored":1063732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73999999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.32500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.62,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.502,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85999999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.495},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0349999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.65500000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.232,"15min":1.232},"last":0.81699999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.107},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.872},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.47499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95999999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.436,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.181,"15min":1.181},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.754}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.622},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94799999999999995,"5min":1.202,"15min":1.202},"last":0.88}]},{"osd":3,"last update":"Mon Oct 16 14:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72599999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95799999999999996,"5min":1.101,"15min":1.101},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":1.127,"15min":1.127},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.753}]},{"osd":8,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.70499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1020000000000001,"5min":1.274,"15min":1.274},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.56899999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Mon Oct 16 14:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.43099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":1.022,"15min":1.022},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.754,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:31:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.876}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.496}]},{"osd":4,"last update":"Mon Oct 16 14:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.747},"min":{"1min":0.59699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":482709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"min":{"1min":0.317,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.37}]},{"osd":2,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.88,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.872},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.873}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1078668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":1.425,"15min":1.425},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.221,"15min":0.221},"max":{"1min":0.52000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84099999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.628}]},{"osd":7,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.196,"15min":1.196},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77400000000000002,"5min":0.878,"15min":0.878},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:32:13.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-10-16T14:32:13.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "82'1", 'reported_seq': 19, 'reported_epoch': 87, 'state': 'peering', 'last_fresh': '2023-10-16T14:31:42.079094+0000', 'last_change': '2023-10-16T14:32:06.668761+0000', 'last_active': '2023-10-16T14:32:06.668761+0000', 'last_peered': '2023-10-16T14:31:42.079094+0000', 'last_clean': '2023-10-16T14:31:42.079094+0000', 'last_became_active': '2023-10-16T14:31:36.055367+0000', 'last_became_peered': '2023-10-16T14:31:36.055367+0000', 'last_unstale': '2023-10-16T14:31:42.079094+0000', 'last_undegraded': '2023-10-16T14:31:42.079094+0000', 'last_fullsized': '2023-10-16T14:31:42.079094+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 81, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T14:31:35.022987+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T14:31:35.022987+0000', 'last_clean_scrub_stamp': '2023-10-16T14:31:35.022987+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [10, 1, 6], 'acting': [10, 1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 6, 8], 'up_primary': 10, 'acting_primary': 10, 'purged_snaps': []} 2023-10-16T14:32:13.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-10-16T14:32:13.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 17, 'reported_epoch': 87, 'state': 'peering', 'last_fresh': '2023-10-16T14:31:42.079202+0000', 'last_change': '2023-10-16T14:32:06.668507+0000', 'last_active': '2023-10-16T14:32:06.668507+0000', 'last_peered': '2023-10-16T14:31:42.079202+0000', 'last_clean': '2023-10-16T14:31:42.079202+0000', 'last_became_active': '2023-10-16T14:31:37.039674+0000', 'last_became_peered': '2023-10-16T14:31:37.039674+0000', 'last_unstale': '2023-10-16T14:31:42.079202+0000', 'last_undegraded': '2023-10-16T14:31:42.079202+0000', 'last_fullsized': '2023-10-16T14:31:42.079202+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 81, 'last_epoch_clean': 82, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T14:31:36.027789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T14:31:36.027789+0000', 'last_clean_scrub_stamp': '2023-10-16T14:31:36.027789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [10, 9, 0], 'acting': [10, 9, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 4, 9], 'up_primary': 10, 'acting_primary': 10, 'purged_snaps': []} 2023-10-16T14:32:13.563 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-16T14:32:13.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 -- 172.21.15.178:0/2265745189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102a40 msgr2=0x7fe0a8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 --2- 172.21.15.178:0/2265745189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102a40 0x7fe0a8109360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000b0a0 tx=0x7fe0a000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 -- 172.21.15.178:0/2265745189 shutdown_connections 2023-10-16T14:32:13.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 --2- 172.21.15.178:0/2265745189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102a40 0x7fe0a8109360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 --2- 172.21.15.178:0/2265745189 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8100ea0 0x7fe0a81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 --2- 172.21.15.178:0/2265745189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8102080 0x7fe0a8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 -- 172.21.15.178:0/2265745189 >> 172.21.15.178:0/2265745189 conn(0x7fe0a80fc420 msgr2=0x7fe0a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:13.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 -- 172.21.15.178:0/2265745189 shutdown_connections 2023-10-16T14:32:13.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.788+0000 7fe0aefff700 1 -- 172.21.15.178:0/2265745189 wait complete. 2023-10-16T14:32:13.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.789+0000 7fe0aefff700 1 Processor -- start 2023-10-16T14:32:13.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.789+0000 7fe0aefff700 1 -- start start 2023-10-16T14:32:13.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8100ea0 0x7fe0a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 0x7fe0a81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a81a09d0 con 0x7fe0a8102080 2023-10-16T14:32:13.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a81a0b10 con 0x7fe0a8102a40 2023-10-16T14:32:13.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0aefff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a81a0ca0 con 0x7fe0a8100ea0 2023-10-16T14:32:13.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57878/0 (socket says 172.21.15.178:57878) 2023-10-16T14:32:13.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 -- 172.21.15.178:0/3060468101 learned_addr learned my addr 172.21.15.178:0/3060468101 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:13.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0acd9b700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8100ea0 0x7fe0a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0ad59c700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 0x7fe0a81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 -- 172.21.15.178:0/3060468101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8100ea0 msgr2=0x7fe0a8198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8100ea0 0x7fe0a8198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 -- 172.21.15.178:0/3060468101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 msgr2=0x7fe0a81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:13.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 0x7fe0a81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:13.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0a7fff700 1 -- 172.21.15.178:0/3060468101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a0009d00 con 0x7fe0a8102080 2023-10-16T14:32:13.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.790+0000 7fe0ad59c700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 0x7fe0a81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:13.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0a7fff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e970 tx=0x7fe09400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:13.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094009790 con 0x7fe0a8102080 2023-10-16T14:32:13.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe09400eec0 con 0x7fe0a8102080 2023-10-16T14:32:13.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a81a0f80 con 0x7fe0a8102080 2023-10-16T14:32:13.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094003cf0 con 0x7fe0a8102080 2023-10-16T14:32:13.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.791+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a81a1400 con 0x7fe0a8102080 2023-10-16T14:32:13.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.792+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a810c760 con 0x7fe0a8102080 2023-10-16T14:32:13.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.794+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fe09400ccd0 con 0x7fe0a8102080 2023-10-16T14:32:13.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.795+0000 7fe0a5ffb700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 0x7fe0980a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:13.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.795+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe094014070 con 0x7fe0a8102080 2023-10-16T14:32:13.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.796+0000 7fe0acd9b700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 0x7fe0980a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:13.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.796+0000 7fe0acd9b700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 0x7fe0980a3a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8101bf0 tx=0x7fe09c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:13.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:13.799+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe0940965e0 con 0x7fe0a8102080 2023-10-16T14:32:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:13 smithi178 ceph-mon[98495]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:13 smithi178 ceph-mon[98495]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:13 smithi186 ceph-mon[134892]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:13 smithi186 ceph-mon[134892]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:14.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.089+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe0a810c970 con 0x7fe0a8102080 2023-10-16T14:32:14.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.090+0000 7fe0a5ffb700 1 -- 172.21.15.178:0/3060468101 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2801 (secure 0 0 0) 0x7fe094099c00 con 0x7fe0a8102080 2023-10-16T14:32:14.093 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:14.093 INFO:teuthology.orchestra.run.smithi178.stdout:{"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":370,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":89,"num_osds":12,"num_up_osds":12,"osd_up_since":1697466648,"num_in_osds":12,"osd_in_since":1697466630,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":2}],"num_pgs":129,"num_pools":4,"num_objects":55,"data_bytes":597628,"bytes_used":345804800,"bytes_avail":1151594622976,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":72,"recovering_keys_per_sec":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"read_bytes_sec":2303,"write_bytes_sec":3199,"read_op_per_sec":2,"write_op_per_sec":8},"fsmap":{"epoch":25,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24515},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24520},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24532},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24548},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24550},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24593}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.178:8443/"}},"servicemap":{"epoch":22,"modified":"2023-10-16T14:32:12.591610+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-16T14:32:14.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.092+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 msgr2=0x7fe0980a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.092+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 0x7fe0980a3a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8101bf0 tx=0x7fe09c009260 comp rx=0 tx=0).stop 2023-10-16T14:32:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.092+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 msgr2=0x7fe0a8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.092+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe09400e970 tx=0x7fe09400ec80 comp rx=0 tx=0).stop 2023-10-16T14:32:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 shutdown_connections 2023-10-16T14:32:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0980a15c0 0x7fe0980a3a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0a8102a40 0x7fe0a81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0a8100ea0 0x7fe0a8198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 --2- 172.21.15.178:0/3060468101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0a8102080 0x7fe0a8199390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 >> 172.21.15.178:0/3060468101 conn(0x7fe0a80fc420 msgr2=0x7fe0a80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 shutdown_connections 2023-10-16T14:32:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.093+0000 7fe0aefff700 1 -- 172.21.15.178:0/3060468101 wait complete. 2023-10-16T14:32:14.114 INFO:tasks.cephfs.filesystem.ceph_manager:making progress, resetting timeout 2023-10-16T14:32:14.114 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:32:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:13 smithi187 ceph-mon[135297]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:13 smithi187 ceph-mon[135297]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 584 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 10 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:14.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.310+0000 7fa042fff700 1 -- 172.21.15.178:0/1102604277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c100ea0 msgr2=0x7fa03c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.310+0000 7fa042fff700 1 --2- 172.21.15.178:0/1102604277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c100ea0 0x7fa03c1012c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa030009990 tx=0x7fa030009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 -- 172.21.15.178:0/1102604277 shutdown_connections 2023-10-16T14:32:14.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 --2- 172.21.15.178:0/1102604277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 --2- 172.21.15.178:0/1102604277 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c100ea0 0x7fa03c1012c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 --2- 172.21.15.178:0/1102604277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c102080 0x7fa03c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 -- 172.21.15.178:0/1102604277 >> 172.21.15.178:0/1102604277 conn(0x7fa03c0fc420 msgr2=0x7fa03c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:14.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 -- 172.21.15.178:0/1102604277 shutdown_connections 2023-10-16T14:32:14.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.311+0000 7fa042fff700 1 -- 172.21.15.178:0/1102604277 wait complete. 2023-10-16T14:32:14.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 Processor -- start 2023-10-16T14:32:14.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 -- start start 2023-10-16T14:32:14.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 0x7fa03c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:14.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c102080 0x7fa03c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:14.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:14.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a0860 con 0x7fa03c102a40 2023-10-16T14:32:14.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a09d0 con 0x7fa03c102080 2023-10-16T14:32:14.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa042fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa03c1a0b90 con 0x7fa03c100ea0 2023-10-16T14:32:14.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa04159c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:14.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa040d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 0x7fa03c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:14.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa03bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c102080 0x7fa03c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:14.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa04159c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57900/0 (socket says 172.21.15.178:57900) 2023-10-16T14:32:14.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.312+0000 7fa04159c700 1 -- 172.21.15.178:0/329289731 learned_addr learned my addr 172.21.15.178:0/329289731 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:14.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 -- 172.21.15.178:0/329289731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c102080 msgr2=0x7fa03c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c102080 0x7fa03c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 -- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 msgr2=0x7fa03c1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 -- 172.21.15.178:0/329289731 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300095f0 con 0x7fa03c100ea0 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa04159c700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa040d9b700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 0x7fa03c198d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:14.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03001d070 con 0x7fa03c100ea0 2023-10-16T14:32:14.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03c1a0e10 con 0x7fa03c100ea0 2023-10-16T14:32:14.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa030022470 con 0x7fa03c100ea0 2023-10-16T14:32:14.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.313+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030017650 con 0x7fa03c100ea0 2023-10-16T14:32:14.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.314+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03c1a11d0 con 0x7fa03c100ea0 2023-10-16T14:32:14.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.314+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa03c10c710 con 0x7fa03c100ea0 2023-10-16T14:32:14.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.316+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fa0300225e0 con 0x7fa03c100ea0 2023-10-16T14:32:14.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.318+0000 7fa039ffb700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 0x7fa02c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:14.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.318+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0300cecc0 con 0x7fa03c100ea0 2023-10-16T14:32:14.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.318+0000 7fa03bfff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 0x7fa02c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:14.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.318+0000 7fa03bfff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 0x7fa02c0a3a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa03c100bc0 tx=0x7fa028009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:14.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.320+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa030097ee0 con 0x7fa03c100ea0 2023-10-16T14:32:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.518+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa03c108a10 con 0x7fa02c0a15c0 2023-10-16T14:32:14.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.531+0000 7fa039ffb700 1 -- 172.21.15.178:0/329289731 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423578 (secure 0 0 0) 0x7fa03c108a10 con 0x7fa02c0a15c0 2023-10-16T14:32:14.539 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.536+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 msgr2=0x7fa02c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.536+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 0x7fa02c0a3a80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa03c100bc0 tx=0x7fa028009260 comp rx=0 tx=0).stop 2023-10-16T14:32:14.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.536+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 msgr2=0x7fa03c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:14.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.536+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 0x7fa03c198d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa030000c00 tx=0x7fa030005720 comp rx=0 tx=0).stop 2023-10-16T14:32:14.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 shutdown_connections 2023-10-16T14:32:14.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa02c0a15c0 0x7fa02c0a3a80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa03c102a40 0x7fa03c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa03c100ea0 0x7fa03c198d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 --2- 172.21.15.178:0/329289731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa03c102080 0x7fa03c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:14.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 >> 172.21.15.178:0/329289731 conn(0x7fa03c0fc420 msgr2=0x7fa03c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:14.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 shutdown_connections 2023-10-16T14:32:14.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:14.537+0000 7fa042fff700 1 -- 172.21.15.178:0/329289731 wait complete. 2023-10-16T14:32:14.546 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:32:14.558 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2023-10-16T14:32:12.591278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597628,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":298,"num_write_kb":2982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337700,"kb_used_data":13460,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604124,"statfs":{"total":1151940427776,"available":1151594622976,"internally_reserved":0,"allocated":13783040,"data_stored":7565557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2620,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":68,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002287"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079120+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:31:42.079120+0000","last_peered":"2023-10-16T14:31:42.079120+0000","last_clean":"2023-10-16T14:31:42.079120+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:31:42.079120+0000","last_undegraded":"2023-10-16T14:31:42.079120+0000","last_fullsized":"2023-10-16T14:31:42.079120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079634+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:31:42.079634+0000","last_peered":"2023-10-16T14:31:42.079634+0000","last_clean":"2023-10-16T14:31:42.079634+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:31:42.079634+0000","last_undegraded":"2023-10-16T14:31:42.079634+0000","last_fullsized":"2023-10-16T14:31:42.079634+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708008+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:32:09.708008+0000","last_peered":"2023-10-16T14:32:09.708008+0000","last_clean":"2023-10-16T14:32:09.708008+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:32:09.708008+0000","last_undegraded":"2023-10-16T14:32:09.708008+0000","last_fullsized":"2023-10-16T14:32:09.708008+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":19,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079094+0000","last_change":"2023-10-16T14:32:06.668761+0000","last_active":"2023-10-16T14:32:06.668761+0000","last_peered":"2023-10-16T14:31:42.079094+0000","last_clean":"2023-10-16T14:31:42.079094+0000","last_became_active":"2023-10-16T14:31:36.055367+0000","last_became_peered":"2023-10-16T14:31:36.055367+0000","last_unstale":"2023-10-16T14:31:42.079094+0000","last_undegraded":"2023-10-16T14:31:42.079094+0000","last_fullsized":"2023-10-16T14:31:42.079094+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,6,8],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689194+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:32:06.689194+0000","last_peered":"2023-10-16T14:32:06.689194+0000","last_clean":"2023-10-16T14:32:06.689194+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:32:06.689194+0000","last_undegraded":"2023-10-16T14:32:06.689194+0000","last_fullsized":"2023-10-16T14:32:06.689194+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671298+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:32:06.671298+0000","last_peered":"2023-10-16T14:32:06.671298+0000","last_clean":"2023-10-16T14:32:06.671298+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:32:06.671298+0000","last_undegraded":"2023-10-16T14:32:06.671298+0000","last_fullsized":"2023-10-16T14:32:06.671298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'12","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689673+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:32:06.689673+0000","last_peered":"2023-10-16T14:32:06.689673+0000","last_clean":"2023-10-16T14:32:06.689673+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:32:06.689673+0000","last_undegraded":"2023-10-16T14:32:06.689673+0000","last_fullsized":"2023-10-16T14:32:06.689673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690399+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:06.690399+0000","last_peered":"2023-10-16T14:32:06.690399+0000","last_clean":"2023-10-16T14:32:06.690399+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:06.690399+0000","last_undegraded":"2023-10-16T14:32:06.690399+0000","last_fullsized":"2023-10-16T14:32:06.690399+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.709111+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:32:09.709111+0000","last_peered":"2023-10-16T14:32:09.709111+0000","last_clean":"2023-10-16T14:32:09.709111+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:32:09.709111+0000","last_undegraded":"2023-10-16T14:32:09.709111+0000","last_fullsized":"2023-10-16T14:32:09.709111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682374+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:07.682374+0000","last_peered":"2023-10-16T14:32:07.682374+0000","last_clean":"2023-10-16T14:32:07.682374+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:07.682374+0000","last_undegraded":"2023-10-16T14:32:07.682374+0000","last_fullsized":"2023-10-16T14:32:07.682374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-16T14:31:42.079402+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:31:42.079402+0000","last_peered":"2023-10-16T14:31:42.079402+0000","last_clean":"2023-10-16T14:31:42.079402+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:31:42.079402+0000","last_undegraded":"2023-10-16T14:31:42.079402+0000","last_fullsized":"2023-10-16T14:31:42.079402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674404+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:06.674404+0000","last_peered":"2023-10-16T14:32:06.674404+0000","last_clean":"2023-10-16T14:32:06.674404+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:06.674404+0000","last_undegraded":"2023-10-16T14:32:06.674404+0000","last_fullsized":"2023-10-16T14:32:06.674404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"peering","last_fresh":"2023-10-16T14:31:42.079202+0000","last_change":"2023-10-16T14:32:06.668507+0000","last_active":"2023-10-16T14:32:06.668507+0000","last_peered":"2023-10-16T14:31:42.079202+0000","last_clean":"2023-10-16T14:31:42.079202+0000","last_became_active":"2023-10-16T14:31:37.039674+0000","last_became_peered":"2023-10-16T14:31:37.039674+0000","last_unstale":"2023-10-16T14:31:42.079202+0000","last_undegraded":"2023-10-16T14:31:42.079202+0000","last_fullsized":"2023-10-16T14:31:42.079202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,4,9],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.708749+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:32:09.708749+0000","last_peered":"2023-10-16T14:32:09.708749+0000","last_clean":"2023-10-16T14:32:09.708749+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:32:09.708749+0000","last_undegraded":"2023-10-16T14:32:09.708749+0000","last_fullsized":"2023-10-16T14:32:09.708749+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675184+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:06.675184+0000","last_peered":"2023-10-16T14:32:06.675184+0000","last_clean":"2023-10-16T14:32:06.675184+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:06.675184+0000","last_undegraded":"2023-10-16T14:32:06.675184+0000","last_fullsized":"2023-10-16T14:32:06.675184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'1","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687172+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:07.687172+0000","last_peered":"2023-10-16T14:32:07.687172+0000","last_clean":"2023-10-16T14:32:07.687172+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:07.687172+0000","last_undegraded":"2023-10-16T14:32:07.687172+0000","last_fullsized":"2023-10-16T14:32:07.687172+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703933+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:08.703933+0000","last_peered":"2023-10-16T14:32:08.703933+0000","last_clean":"2023-10-16T14:32:08.703933+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:08.703933+0000","last_undegraded":"2023-10-16T14:32:08.703933+0000","last_fullsized":"2023-10-16T14:32:08.703933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672544+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:06.672544+0000","last_peered":"2023-10-16T14:32:06.672544+0000","last_clean":"2023-10-16T14:32:06.672544+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:06.672544+0000","last_undegraded":"2023-10-16T14:32:06.672544+0000","last_fullsized":"2023-10-16T14:32:06.672544+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673284+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:06.673284+0000","last_peered":"2023-10-16T14:32:06.673284+0000","last_clean":"2023-10-16T14:32:06.673284+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:06.673284+0000","last_undegraded":"2023-10-16T14:32:06.673284+0000","last_fullsized":"2023-10-16T14:32:06.673284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673588+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:06.673588+0000","last_peered":"2023-10-16T14:32:06.673588+0000","last_clean":"2023-10-16T14:32:06.673588+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:06.673588+0000","last_undegraded":"2023-10-16T14:32:06.673588+0000","last_fullsized":"2023-10-16T14:32:06.673588+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7260,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":117,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":221184,"data_stored":20842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449107,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47199999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.40200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73899999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.82199999999999995}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1067930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.432},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.45600000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.58799999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.73199999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.626}]},{"osd":11,"last update":"Mon Oct 16 14:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.68700000000000006}]}]},{"osd":9,"up_from":66,"seq":283467841564,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.76100000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.72899999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.746}]},{"osd":5,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.90600000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.88},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.93999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":482745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60599999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.69099999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.751,"15min":0.751},"last":0.56999999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.625}]},{"osd":4,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90500000000000003,"5min":1.087,"15min":1.087},"last":0.52700000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84299999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.878}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.85399999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332132,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28552,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716600,"statfs":{"total":95995035648,"available":95965798400,"internally_reserved":0,"allocated":1568768,"data_stored":1063732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56399999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.68700000000000006}]},{"osd":4,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73999999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.32500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.62,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.502,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85999999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.68500000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.435}]},{"osd":3,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.495},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.72599999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0349999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.65500000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.232,"15min":1.232},"last":0.81699999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.107},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.60899999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80500000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.872},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.93799999999999994}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":978944,"data_stored":476333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.53800000000000003,"15min":0.51000000000000001},"min":{"1min":0.47499999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.55700000000000005,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95999999999999996,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83599999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.436,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.181,"15min":1.181},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.66400000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53200000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.754}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.77100000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64300000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.76600000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.622},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94799999999999995,"5min":1.202,"15min":1.202},"last":0.88}]},{"osd":3,"last update":"Mon Oct 16 14:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.72599999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.73699999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95799999999999996,"5min":1.101,"15min":1.101},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53800000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":1.127,"15min":1.127},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.753}]},{"osd":8,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.70499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1020000000000001,"5min":1.274,"15min":1.274},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.56899999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.90000000000000002}]},{"osd":11,"last update":"Mon Oct 16 14:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":486836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.43099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":1.022,"15min":1.022},"last":0.58199999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.754,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.64400000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.34200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.78000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:31:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60799999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:31:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.876}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.82899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.496}]},{"osd":4,"last update":"Mon Oct 16 14:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.67000000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.747},"min":{"1min":0.59699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81999999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":995328,"data_stored":482709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"min":{"1min":0.317,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.37}]},{"osd":2,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.58299999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.88,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.872},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60299999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.503},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72199999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.68600000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.873}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1078668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":1.425,"15min":1.425},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.221,"15min":0.221},"max":{"1min":0.52000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84099999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.628}]},{"osd":7,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.196,"15min":1.196},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77400000000000002,"5min":0.878,"15min":0.878},"last":0.72499999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54700000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64400000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:32:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:14 smithi178 ceph-mon[98495]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:14 smithi178 ceph-mon[98495]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:14 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3060468101' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:32:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:14 smithi186 ceph-mon[134892]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:14 smithi186 ceph-mon[134892]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:14 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3060468101' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:32:15.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:14 smithi187 ceph-mon[135297]: from='client.24878 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:14 smithi187 ceph-mon[135297]: from='client.24871 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:15.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:14 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3060468101' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:32:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:15 smithi178 ceph-mon[98495]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:15 smithi178 ceph-mon[98495]: pgmap v293: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.6 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:15 smithi186 ceph-mon[134892]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:15 smithi186 ceph-mon[134892]: pgmap v293: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.6 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:16.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:15 smithi187 ceph-mon[135297]: from='client.24883 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:16.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:15 smithi187 ceph-mon[135297]: pgmap v293: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.6 KiB/s wr, 16 op/s; 72 B/s, 0 objects/s recovering 2023-10-16T14:32:17.568 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:32:17.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 -- 172.21.15.178:0/4149130027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890100ea0 msgr2=0x7f68901012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:17.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 --2- 172.21.15.178:0/4149130027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890100ea0 0x7f68901012c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f688c009990 tx=0x7f688c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 -- 172.21.15.178:0/4149130027 shutdown_connections 2023-10-16T14:32:17.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 --2- 172.21.15.178:0/4149130027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890102a40 0x7f6890109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 --2- 172.21.15.178:0/4149130027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890100ea0 0x7f68901012c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 --2- 172.21.15.178:0/4149130027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 0x7f6890102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 -- 172.21.15.178:0/4149130027 >> 172.21.15.178:0/4149130027 conn(0x7f68900fc420 msgr2=0x7f68900fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:17.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 -- 172.21.15.178:0/4149130027 shutdown_connections 2023-10-16T14:32:17.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.773+0000 7f68985ff700 1 -- 172.21.15.178:0/4149130027 wait complete. 2023-10-16T14:32:17.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.774+0000 7f68985ff700 1 Processor -- start 2023-10-16T14:32:17.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.774+0000 7f68985ff700 1 -- start start 2023-10-16T14:32:17.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 0x7f6890198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:17.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 0x7f68901992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:17.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:17.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a08f0 con 0x7f6890102a40 2023-10-16T14:32:17.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a0a60 con 0x7f6890100ea0 2023-10-16T14:32:17.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f68985ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901a0c20 con 0x7f6890102080 2023-10-16T14:32:17.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:17.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57926/0 (socket says 172.21.15.178:57926) 2023-10-16T14:32:17.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 -- 172.21.15.178:0/3893727270 learned_addr learned my addr 172.21.15.178:0/3893727270 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:17.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f689639b700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 0x7f6890198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:17.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6895b9a700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 0x7f68901992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:17.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 -- 172.21.15.178:0/3893727270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 msgr2=0x7f68901992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:17.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 0x7f68901992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 -- 172.21.15.178:0/3893727270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 msgr2=0x7f6890198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:17.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 0x7f6890198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:17.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f6896b9c700 1 -- 172.21.15.178:0/3893727270 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c0095f0 con 0x7f6890102a40 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.775+0000 7f689639b700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 0x7f6890198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f6896b9c700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f688800ef30 tx=0x7f688800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888018070 con 0x7f6890102a40 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f688800f040 con 0x7f6890102a40 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68901a0ea0 con 0x7f6890102a40 2023-10-16T14:32:17.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888003dc0 con 0x7f6890102a40 2023-10-16T14:32:17.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.776+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68901a1270 con 0x7f6890102a40 2023-10-16T14:32:17.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.777+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689010c760 con 0x7f6890102a40 2023-10-16T14:32:17.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.779+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f688800caa0 con 0x7f6890102a40 2023-10-16T14:32:17.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.780+0000 7f68877fe700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 0x7f687c0a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:17.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.781+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f68880cce70 con 0x7f6890102a40 2023-10-16T14:32:17.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.781+0000 7f689639b700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 0x7f687c0a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:17.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.781+0000 7f689639b700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 0x7f687c0a3a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688c000c00 tx=0x7f688c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:17.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.785+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6888096090 con 0x7f6890102a40 2023-10-16T14:32:17.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.981+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6890108a30 con 0x7f687c0a15c0 2023-10-16T14:32:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.994+0000 7f68877fe700 1 -- 172.21.15.178:0/3893727270 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423541 (secure 0 0 0) 0x7f6890108a30 con 0x7f687c0a15c0 2023-10-16T14:32:18.003 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 msgr2=0x7f687c0a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 0x7f687c0a3a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f688c000c00 tx=0x7f688c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 msgr2=0x7f68901a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f688800ef30 tx=0x7f688800c450 comp rx=0 tx=0).stop 2023-10-16T14:32:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 shutdown_connections 2023-10-16T14:32:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f687c0a15c0 0x7f687c0a3a80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6890102a40 0x7f68901a0320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6890100ea0 0x7f6890198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 --2- 172.21.15.178:0/3893727270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6890102080 0x7f68901992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 >> 172.21.15.178:0/3893727270 conn(0x7f68900fc420 msgr2=0x7f68900fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 shutdown_connections 2023-10-16T14:32:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:17.999+0000 7f68985ff700 1 -- 172.21.15.178:0/3893727270 wait complete. 2023-10-16T14:32:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:32:18.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:17 smithi178 ceph-mon[98495]: pgmap v294: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 58 B/s, 0 objects/s recovering 2023-10-16T14:32:18.020 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:17 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:18.022 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2023-10-16T14:32:16.592762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637790,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":176,"num_read_kb":184,"num_write":329,"num_write_kb":3028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":266,"ondisk_log_size":266,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337872,"kb_used_data":13664,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603952,"statfs":{"total":1151940427776,"available":1151594446848,"internally_reserved":0,"allocated":13991936,"data_stored":7730767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42782,"num_objects":40,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":50,"num_read_kb":75,"num_write":99,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.927420"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.668225+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:32:06.668225+0000","last_peered":"2023-10-16T14:32:06.668225+0000","last_clean":"2023-10-16T14:32:06.668225+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:32:06.668225+0000","last_undegraded":"2023-10-16T14:32:06.668225+0000","last_fullsized":"2023-10-16T14:32:06.668225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.669076+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:32:06.669076+0000","last_peered":"2023-10-16T14:32:06.669076+0000","last_clean":"2023-10-16T14:32:06.669076+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:32:06.669076+0000","last_undegraded":"2023-10-16T14:32:06.669076+0000","last_fullsized":"2023-10-16T14:32:06.669076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:13.716184+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:32:13.716184+0000","last_peered":"2023-10-16T14:32:13.716184+0000","last_clean":"2023-10-16T14:32:13.716184+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:32:13.716184+0000","last_undegraded":"2023-10-16T14:32:13.716184+0000","last_fullsized":"2023-10-16T14:32:13.716184+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702769+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:32:08.702769+0000","last_peered":"2023-10-16T14:32:08.702769+0000","last_clean":"2023-10-16T14:32:08.702769+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:32:08.702769+0000","last_undegraded":"2023-10-16T14:32:08.702769+0000","last_fullsized":"2023-10-16T14:32:08.702769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:10.710500+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:32:10.710500+0000","last_peered":"2023-10-16T14:32:10.710500+0000","last_clean":"2023-10-16T14:32:10.710500+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:32:10.710500+0000","last_undegraded":"2023-10-16T14:32:10.710500+0000","last_fullsized":"2023-10-16T14:32:10.710500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671298+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:32:06.671298+0000","last_peered":"2023-10-16T14:32:06.671298+0000","last_clean":"2023-10-16T14:32:06.671298+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:32:06.671298+0000","last_undegraded":"2023-10-16T14:32:06.671298+0000","last_fullsized":"2023-10-16T14:32:06.671298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'23","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:11.714956+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:32:11.714956+0000","last_peered":"2023-10-16T14:32:11.714956+0000","last_clean":"2023-10-16T14:32:11.714956+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:32:11.714956+0000","last_undegraded":"2023-10-16T14:32:11.714956+0000","last_fullsized":"2023-10-16T14:32:11.714956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43176,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":47,"num_write":24,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703074+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:08.703074+0000","last_peered":"2023-10-16T14:32:08.703074+0000","last_clean":"2023-10-16T14:32:08.703074+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:08.703074+0000","last_undegraded":"2023-10-16T14:32:08.703074+0000","last_fullsized":"2023-10-16T14:32:08.703074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:13.716286+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:32:13.716286+0000","last_peered":"2023-10-16T14:32:13.716286+0000","last_clean":"2023-10-16T14:32:13.716286+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:32:13.716286+0000","last_undegraded":"2023-10-16T14:32:13.716286+0000","last_fullsized":"2023-10-16T14:32:13.716286+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.706275+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:09.706275+0000","last_peered":"2023-10-16T14:32:09.706275+0000","last_clean":"2023-10-16T14:32:09.706275+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:09.706275+0000","last_undegraded":"2023-10-16T14:32:09.706275+0000","last_fullsized":"2023-10-16T14:32:09.706275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.669160+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:32:06.669160+0000","last_peered":"2023-10-16T14:32:06.669160+0000","last_clean":"2023-10-16T14:32:06.669160+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:32:06.669160+0000","last_undegraded":"2023-10-16T14:32:06.669160+0000","last_fullsized":"2023-10-16T14:32:06.669160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676695+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:07.676695+0000","last_peered":"2023-10-16T14:32:07.676695+0000","last_clean":"2023-10-16T14:32:07.676695+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:07.676695+0000","last_undegraded":"2023-10-16T14:32:07.676695+0000","last_fullsized":"2023-10-16T14:32:07.676695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.681861+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:32:07.681861+0000","last_peered":"2023-10-16T14:32:07.681861+0000","last_clean":"2023-10-16T14:32:07.681861+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:32:07.681861+0000","last_undegraded":"2023-10-16T14:32:07.681861+0000","last_fullsized":"2023-10-16T14:32:07.681861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:14.717353+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:32:14.717353+0000","last_peered":"2023-10-16T14:32:14.717353+0000","last_clean":"2023-10-16T14:32:14.717353+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:32:14.717353+0000","last_undegraded":"2023-10-16T14:32:14.717353+0000","last_fullsized":"2023-10-16T14:32:14.717353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.677326+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:07.677326+0000","last_peered":"2023-10-16T14:32:07.677326+0000","last_clean":"2023-10-16T14:32:07.677326+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:07.677326+0000","last_undegraded":"2023-10-16T14:32:07.677326+0000","last_fullsized":"2023-10-16T14:32:07.677326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703372+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:08.703372+0000","last_peered":"2023-10-16T14:32:08.703372+0000","last_clean":"2023-10-16T14:32:08.703372+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:08.703372+0000","last_undegraded":"2023-10-16T14:32:08.703372+0000","last_fullsized":"2023-10-16T14:32:08.703372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'5","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:10.911207+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:10.911207+0000","last_peered":"2023-10-16T14:32:10.911207+0000","last_clean":"2023-10-16T14:32:10.911207+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:10.911207+0000","last_undegraded":"2023-10-16T14:32:10.911207+0000","last_fullsized":"2023-10-16T14:32:10.911207+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920458+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:07.920458+0000","last_peered":"2023-10-16T14:32:07.920458+0000","last_clean":"2023-10-16T14:32:07.920458+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:07.920458+0000","last_undegraded":"2023-10-16T14:32:07.920458+0000","last_fullsized":"2023-10-16T14:32:07.920458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682730+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:07.682730+0000","last_peered":"2023-10-16T14:32:07.682730+0000","last_clean":"2023-10-16T14:32:07.682730+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:07.682730+0000","last_undegraded":"2023-10-16T14:32:07.682730+0000","last_fullsized":"2023-10-16T14:32:07.682730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.705275+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:08.705275+0000","last_peered":"2023-10-16T14:32:08.705275+0000","last_clean":"2023-10-16T14:32:08.705275+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:08.705275+0000","last_undegraded":"2023-10-16T14:32:08.705275+0000","last_fullsized":"2023-10-16T14:32:08.705275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47176,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":50,"num_read_kb":75,"num_write":146,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":141528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449108,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.42999999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47699999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.496},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.61799999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.72199999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.749},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.70099999999999996}]}]},{"osd":10,"up_from":71,"seq":304942678039,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1117460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34300000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.73399999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.442}]},{"osd":4,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.745},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.54900000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68100000000000005}]},{"osd":11,"last update":"Mon Oct 16 14:31:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71899999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841565,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56699999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.433,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.754,"15min":0.77900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83899999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":1.262,"15min":1.262},"last":0.74399999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.73999999999999999,"5min":1.54,"15min":1.54},"last":0.878},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84899999999999998,"5min":1.246,"15min":1.246},"last":0.73199999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":1.319,"15min":1.319},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71599999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.81699999999999995,"15min":0.84199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.83499999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.82099999999999995,"15min":0.84999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.91700000000000004,"15min":0.95499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.876,"15min":0.91500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.452,"15min":1.452},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.96699999999999997,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.90600000000000003,"5min":1.474,"15min":1.474},"last":0.93300000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":1.637,"15min":1.637},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81100000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.0569999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.016}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96699999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103202,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":486841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53000000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.39500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75900000000000001,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.48499999999999999,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.87,"15min":0.87},"last":0.501},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.498,"15min":0.505},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":0.751,"15min":0.751},"last":0.56699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.66500000000000004,"15min":0.67400000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93999999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68200000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.82499999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46300000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99099999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79200000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.57599999999999996,"15min":0.58699999999999997},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.753,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.627,"15min":0.64300000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.90500000000000003,"5min":1.087,"15min":1.087},"last":0.76200000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.76000000000000001,"15min":0.77600000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.71499999999999997,"15min":0.72999999999999998},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84299999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.78300000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.48899999999999999,"5min":0.436,"15min":0.436},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73299999999999998,"15min":0.74199999999999999},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.82399999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.91500000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.876},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.88700000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1081894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.44900000000000001,"15min":0.439},"min":{"1min":0.28399999999999997,"5min":0.249,"15min":0.249},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.38200000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54200000000000004},"min":{"1min":0.41399999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.71099999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.61299999999999999,"15min":0.58899999999999997},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.40999999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73999999999999999,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52700000000000002,"15min":0.52200000000000002},"min":{"1min":0.38,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51600000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.66500000000000004,"15min":0.64900000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.49299999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.66200000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.626},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68700000000000006,"15min":0.68000000000000005},"min":{"1min":0.48199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81399999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:31:43 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72699999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.59199999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.32500000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.62,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64200000000000002,"15min":0.63200000000000001},"min":{"1min":0.502,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85999999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.64200000000000002,"15min":0.62},"min":{"1min":0.52600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.625}]},{"osd":2,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.64900000000000002,"15min":0.629},"min":{"1min":0.48799999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92700000000000005,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.752}]},{"osd":3,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.95599999999999996,"5min":1.002,"15min":1.002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.629,"15min":0.626},"min":{"1min":0.40699999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89000000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.60899999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.54000000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.50600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.94399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.72899999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0349999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.85599999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.65500000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0600000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.55500000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.042,"5min":1.232,"15min":1.232},"last":0.76200000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:31:39 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78500000000000003,"15min":0.79100000000000004},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.107},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":1.1000000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0549999999999999,"5min":1.133,"15min":1.133},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85099999999999998,"15min":0.85699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0449999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.77500000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.90400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":525241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.56100000000000005,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.63},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.54600000000000004,"15min":0.53000000000000003},"min":{"1min":0.378,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.77700000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59099999999999997,"15min":0.55600000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.77200000000000002,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55600000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.42999999999999999,"15min":0.41199999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.436,"15min":0.42399999999999999},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40999999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60999999999999999,"15min":0.60599999999999998},"min":{"1min":0.44900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84399999999999997,"5min":1.181,"15min":1.181},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82399999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.51500000000000001,"15min":0.498},"min":{"1min":0.38100000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.505},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60999999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68400000000000005,"15min":0.67400000000000004},"min":{"1min":0.49099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96199999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70199999999999996,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70899999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.69799999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.746},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.79200000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.90500000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.437,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86899999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.621,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94799999999999995,"5min":1.202,"15min":1.202},"last":0.60599999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.501,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.85799999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.36599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84699999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.45700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:31:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.53800000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:31:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.66800000000000004,"15min":0.64200000000000002},"min":{"1min":0.48499999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.95799999999999996,"5min":1.101,"15min":1.101},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47699999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85699999999999998,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.41799999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:32:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60799999999999998,"15min":0.58799999999999997},"min":{"1min":0.437,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.83099999999999996,"5min":1.127,"15min":1.127},"last":0.505},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.72199999999999998},"min":{"1min":0.5,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0429999999999999,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.79700000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59299999999999997}]},{"osd":8,"last update":"Mon Oct 16 14:31:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.84199999999999997},"min":{"1min":0.70499999999999996,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1020000000000001,"5min":1.274,"15min":1.274},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.75},"min":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.51600000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.86899999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.94499999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":527254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48999999999999999},"min":{"1min":0.34499999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.502,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.501}]},{"osd":1,"last update":"Mon Oct 16 14:31:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.54300000000000004,"15min":0.496},"min":{"1min":0.43099999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52600000000000002},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":1.022,"15min":1.022},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93600000000000005,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.376,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.75600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61499999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:31:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.57299999999999995,"15min":0.57199999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.754,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90600000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.68100000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:31:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.434},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.44700000000000001,"15min":0.436},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:31:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.83099999999999996,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.71599999999999997,"15min":0.65400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.88100000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:31:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.67700000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.92100000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:31:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.36399999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.0129999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:31:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.98499999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.433,"15min":0.41599999999999998},"min":{"1min":0.31900000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.438},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.34499999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70899999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.40000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.59799999999999998,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60699999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.79700000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.63400000000000001,"15min":0.61599999999999999},"min":{"1min":0.41899999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.72999999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:31:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64100000000000001,"15min":0.63500000000000001},"min":{"1min":0.47999999999999998,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.79400000000000004,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.751},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92600000000000005,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.82299999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.84699999999999998,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.64100000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.755,"15min":0.747},"min":{"1min":0.59699999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1579999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.878},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73299999999999998},"min":{"1min":0.48599999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.85199999999999998,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.71099999999999997}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.81200000000000006},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80500000000000005,"5min":1.214,"15min":1.214},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.111,"5min":1.327,"15min":1.327},"last":0.77400000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.84499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.39400000000000002,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.40999999999999998,"15min":0.39800000000000002},"min":{"1min":0.317,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56000000000000005,"15min":0.54800000000000004},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83099999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.57599999999999996,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.88,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79100000000000004,"5min":0.872,"15min":0.872},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:31:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60099999999999998,"15min":0.58299999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61699999999999999,"15min":0.60899999999999999},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73899999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.84299999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:21 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.57299999999999995,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.61299999999999999,"15min":0.59699999999999998},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79800000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:31:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92100000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64600000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.754},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:31:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1613824,"data_stored":1078668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.55000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92500000000000004,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.53100000000000003},"min":{"1min":0.40999999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84499999999999997,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.872,"5min":1.425,"15min":1.425},"last":0.59099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27900000000000003,"5min":0.221,"15min":0.221},"max":{"1min":0.52000000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40000000000000002,"15min":0.39800000000000002},"min":{"1min":0.27500000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.55500000000000005},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84099999999999997,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59399999999999997,"15min":0.57499999999999996},"min":{"1min":0.42399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.83399999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58099999999999996,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89200000000000002,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56399999999999995,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.69299999999999995}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.71499999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94299999999999995,"5min":1.196,"15min":1.196},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.77400000000000002,"5min":0.878,"15min":0.878},"last":0.80100000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:31:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.624},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.91900000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.58599999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.88300000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:31:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.92000000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":44916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:32:18.031 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T14:32:18.032 INFO:tasks.cephadm:Setting up client nodes... 2023-10-16T14:32:18.032 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-10-16T14:32:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:17 smithi186 ceph-mon[134892]: pgmap v294: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 58 B/s, 0 objects/s recovering 2023-10-16T14:32:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:17 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:17 smithi187 ceph-mon[135297]: pgmap v294: 129 pgs: 129 active+clean; 623 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 15 op/s; 58 B/s, 0 objects/s recovering 2023-10-16T14:32:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:17 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:32:18.794 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:19.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:18 smithi178 ceph-mon[98495]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:19.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:18 smithi186 ceph-mon[134892]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:19.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:18 smithi187 ceph-mon[135297]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:32:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:19 smithi178 ceph-mon[98495]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 15 op/s; 53 B/s, 0 objects/s recovering 2023-10-16T14:32:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:19 smithi186 ceph-mon[134892]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 15 op/s; 53 B/s, 0 objects/s recovering 2023-10-16T14:32:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:19 smithi187 ceph-mon[135297]: pgmap v295: 129 pgs: 129 active+clean; 639 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.4 KiB/s wr, 15 op/s; 53 B/s, 0 objects/s recovering 2023-10-16T14:32:20.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 -- 172.21.15.178:0/95385255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a8104d80 msgr2=0x7f09a810b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:20.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 --2- 172.21.15.178:0/95385255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a8104d80 0x7f09a810b630 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f09a000b0d0 tx=0x7f09a000b3e0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 -- 172.21.15.178:0/95385255 shutdown_connections 2023-10-16T14:32:20.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 --2- 172.21.15.178:0/95385255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a8104d80 0x7f09a810b630 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 --2- 172.21.15.178:0/95385255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a81031e0 0x7f09a8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 --2- 172.21.15.178:0/95385255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81043e0 0x7f09a8104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 -- 172.21.15.178:0/95385255 >> 172.21.15.178:0/95385255 conn(0x7f09a80fe780 msgr2=0x7f09a8100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:20.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 -- 172.21.15.178:0/95385255 shutdown_connections 2023-10-16T14:32:20.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.510+0000 7f09add41700 1 -- 172.21.15.178:0/95385255 wait complete. 2023-10-16T14:32:20.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.511+0000 7f09add41700 1 Processor -- start 2023-10-16T14:32:20.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.511+0000 7f09add41700 1 -- start start 2023-10-16T14:32:20.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81031e0 0x7f09a8198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:20.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:20.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a8104d80 0x7f09a81a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:20.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a0b10 con 0x7f09a81043e0 2023-10-16T14:32:20.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a0c50 con 0x7f09a81031e0 2023-10-16T14:32:20.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09add41700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a81a0de0 con 0x7f09a8104d80 2023-10-16T14:32:20.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:20.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39636/0 (socket says 172.21.15.178:39636) 2023-10-16T14:32:20.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 -- 172.21.15.178:0/1725070340 learned_addr learned my addr 172.21.15.178:0/1725070340 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:20.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a7fff700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a8104d80 0x7f09a81a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:20.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a77fe700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81031e0 0x7f09a8198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:20.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 -- 172.21.15.178:0/1725070340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a8104d80 msgr2=0x7f09a81a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:20.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a8104d80 0x7f09a81a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 -- 172.21.15.178:0/1725070340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81031e0 msgr2=0x7f09a8198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:20.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81031e0 0x7f09a8198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.512+0000 7f09a6ffd700 1 -- 172.21.15.178:0/1725070340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009d00 con 0x7f09a81043e0 2023-10-16T14:32:20.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09a6ffd700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e9d0 tx=0x7f099c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:20.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c009790 con 0x7f09a81043e0 2023-10-16T14:32:20.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f099c00ee70 con 0x7f09a81043e0 2023-10-16T14:32:20.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a81a10c0 con 0x7f09a81043e0 2023-10-16T14:32:20.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c010ea0 con 0x7f09a81043e0 2023-10-16T14:32:20.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.513+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a81a1610 con 0x7f09a81043e0 2023-10-16T14:32:20.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.514+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a810a740 con 0x7f09a81043e0 2023-10-16T14:32:20.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.516+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f099c00cbd0 con 0x7f09a81043e0 2023-10-16T14:32:20.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.517+0000 7f09a4ff9700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 0x7f09900a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:20.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.518+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f099c014070 con 0x7f09a81043e0 2023-10-16T14:32:20.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.518+0000 7f09a77fe700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 0x7f09900a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:20.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.518+0000 7f09a77fe700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 0x7f09900a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09a8104240 tx=0x7f0998007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:20.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.521+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f099c052100 con 0x7f09a81043e0 2023-10-16T14:32:20.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.801+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f09a810ec30 con 0x7f09a81043e0 2023-10-16T14:32:20.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.804+0000 7f09a4ff9700 1 -- 172.21.15.178:0/1725070340 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f099c0964a0 con 0x7f09a81043e0 2023-10-16T14:32:20.808 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2023-10-16T14:32:20.808 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQB0SS1l1yTqLxAAbDFpHcNUEQ6iVYMq03BAmg== 2023-10-16T14:32:20.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 msgr2=0x7f09900a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:20.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 0x7f09900a3a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f09a8104240 tx=0x7f0998007400 comp rx=0 tx=0).stop 2023-10-16T14:32:20.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 msgr2=0x7f09a81994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:20.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f099c00e9d0 tx=0x7f099c00ece0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 shutdown_connections 2023-10-16T14:32:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f09900a1590 0x7f09900a3a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f09a8104d80 0x7f09a81a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f09a81031e0 0x7f09a8198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 --2- 172.21.15.178:0/1725070340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09a81043e0 0x7f09a81994c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:20.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 >> 172.21.15.178:0/1725070340 conn(0x7f09a80fe780 msgr2=0x7f09a8107880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:20.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 shutdown_connections 2023-10-16T14:32:20.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:20.807+0000 7f09add41700 1 -- 172.21.15.178:0/1725070340 wait complete. 2023-10-16T14:32:21.290 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:32:21.291 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-10-16T14:32:21.291 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T14:32:21.393 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T14:32:21.394 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-10-16T14:32:21.394 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph mgr dump --format=json 2023-10-16T14:32:21.493 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:21 smithi178 ceph-mon[98495]: pgmap v296: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:21.494 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:21 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-16T14:32:21.494 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:21 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-16T14:32:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:21 smithi186 ceph-mon[134892]: pgmap v296: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:21 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-16T14:32:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:21 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-16T14:32:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:21 smithi187 ceph-mon[135297]: pgmap v296: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 8.7 KiB/s wr, 14 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:21 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-10-16T14:32:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:21 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1725070340' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-10-16T14:32:22.169 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:23.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 -- 172.21.15.178:0/3413702148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4101940 msgr2=0x7f10b410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:23.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/3413702148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4101940 0x7f10b410ddb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b0d0 tx=0x7f10ac00b3e0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 -- 172.21.15.178:0/3413702148 shutdown_connections 2023-10-16T14:32:23.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/3413702148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4101940 0x7f10b410ddb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/3413702148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4106f90 0x7f10b4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/3413702148 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4100f80 0x7f10b4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 -- 172.21.15.178:0/3413702148 >> 172.21.15.178:0/3413702148 conn(0x7f10b40780a0 msgr2=0x7f10b40784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:23.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 -- 172.21.15.178:0/3413702148 shutdown_connections 2023-10-16T14:32:23.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.830+0000 7f10bbcc0700 1 -- 172.21.15.178:0/3413702148 wait complete. 2023-10-16T14:32:23.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.831+0000 7f10bbcc0700 1 Processor -- start 2023-10-16T14:32:23.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.831+0000 7f10bbcc0700 1 -- start start 2023-10-16T14:32:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4101940 0x7f10b40736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:23.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 0x7f10b4073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a5560 con 0x7f10b4100f80 2023-10-16T14:32:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a56d0 con 0x7f10b4106f90 2023-10-16T14:32:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10bbcc0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41a5890 con 0x7f10b4101940 2023-10-16T14:32:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:23.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39652/0 (socket says 172.21.15.178:39652) 2023-10-16T14:32:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 -- 172.21.15.178:0/2374598210 learned_addr learned my addr 172.21.15.178:0/2374598210 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b925b700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4101940 0x7f10b40736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10ba25d700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 0x7f10b4073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:23.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 -- 172.21.15.178:0/2374598210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4101940 msgr2=0x7f10b40736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4101940 0x7f10b40736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 -- 172.21.15.178:0/2374598210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 msgr2=0x7f10b4073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 0x7f10b4073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 -- 172.21.15.178:0/2374598210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac009d00 con 0x7f10b4100f80 2023-10-16T14:32:23.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10ba25d700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 0x7f10b4073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.832+0000 7f10b9a5c700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9a0 tx=0x7f10b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.833+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0009790 con 0x7f10b4100f80 2023-10-16T14:32:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.833+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10b000cbd0 con 0x7f10b4100f80 2023-10-16T14:32:23.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.833+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0018680 con 0x7f10b4100f80 2023-10-16T14:32:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.833+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b41a5b70 con 0x7f10b4100f80 2023-10-16T14:32:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.833+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b41a6020 con 0x7f10b4100f80 2023-10-16T14:32:23.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.834+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b404ebb0 con 0x7f10b4100f80 2023-10-16T14:32:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.836+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f10b000cd40 con 0x7f10b4100f80 2023-10-16T14:32:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.838+0000 7f10aaffd700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 0x7f10a00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:23.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.838+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10b0014070 con 0x7f10b4100f80 2023-10-16T14:32:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.838+0000 7f10b925b700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 0x7f10a00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.839+0000 7f10b925b700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 0x7f10a00a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10b40749a0 tx=0x7f10a4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:23.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:23.842+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f10b0052100 con 0x7f10b4100f80 2023-10-16T14:32:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:23 smithi187 ceph-mon[135297]: pgmap v297: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:24.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:23 smithi178 ceph-mon[98495]: pgmap v297: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:23 smithi186 ceph-mon[134892]: pgmap v297: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.8 KiB/s wr, 10 op/s; 48 B/s, 0 objects/s recovering 2023-10-16T14:32:24.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.099+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f10b40746e0 con 0x7f10b4100f80 2023-10-16T14:32:24.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.104+0000 7f10aaffd700 1 -- 172.21.15.178:0/2374598210 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260410 (secure 0 0 0) 0x7f10b0065950 con 0x7f10b4100f80 2023-10-16T14:32:24.113 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:24.115 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 msgr2=0x7f10a00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:24.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 0x7f10a00a3a50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f10b40749a0 tx=0x7f10a4009260 comp rx=0 tx=0).stop 2023-10-16T14:32:24.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 msgr2=0x7f10b40731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:24.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f10b000e9a0 tx=0x7f10b000ecb0 comp rx=0 tx=0).stop 2023-10-16T14:32:24.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 shutdown_connections 2023-10-16T14:32:24.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10a00a1590 0x7f10a00a3a50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:24.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f10b4106f90 0x7f10b4073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:24.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10b4100f80 0x7f10b40731b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:24.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 --2- 172.21.15.178:0/2374598210 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10b4101940 0x7f10b40736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:24.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.108+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 >> 172.21.15.178:0/2374598210 conn(0x7f10b40780a0 msgr2=0x7f10b40ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:24.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.109+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 shutdown_connections 2023-10-16T14:32:24.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:24.109+0000 7f10bbcc0700 1 -- 172.21.15.178:0/2374598210 wait complete. 2023-10-16T14:32:24.667 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":17,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":706630784},{"type":"v1","addr":"172.21.15.178:6801","nonce":706630784}]},"active_addr":"172.21.15.178:6801/706630784","active_change":"2023-10-16T14:25:06.501968+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.178:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":3582286393}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":664493085}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":48642049}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":2750452472}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.178:0","nonce":174843256}]}]} 2023-10-16T14:32:24.674 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-10-16T14:32:24.674 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-10-16T14:32:24.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-16T14:32:24.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:24 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2374598210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-16T14:32:25.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:24 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2374598210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-16T14:32:25.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:24 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2374598210' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-10-16T14:32:25.413 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:25 smithi187 ceph-mon[135297]: pgmap v298: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-10-16T14:32:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:25 smithi178 ceph-mon[98495]: pgmap v298: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-10-16T14:32:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:25 smithi186 ceph-mon[134892]: pgmap v298: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2023-10-16T14:32:27.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 -- 172.21.15.178:0/3533873867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc100f80 msgr2=0x7fc0dc101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:27.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/3533873867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc100f80 0x7fc0dc101400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00099c0 tx=0x7fc0d0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 -- 172.21.15.178:0/3533873867 shutdown_connections 2023-10-16T14:32:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/3533873867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc101940 0x7fc0dc10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/3533873867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/3533873867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc100f80 0x7fc0dc101400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 -- 172.21.15.178:0/3533873867 >> 172.21.15.178:0/3533873867 conn(0x7fc0dc0780a0 msgr2=0x7fc0dc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:27.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 -- 172.21.15.178:0/3533873867 shutdown_connections 2023-10-16T14:32:27.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.147+0000 7fc0e0d79700 1 -- 172.21.15.178:0/3533873867 wait complete. 2023-10-16T14:32:27.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.148+0000 7fc0e0d79700 1 Processor -- start 2023-10-16T14:32:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.148+0000 7fc0e0d79700 1 -- start start 2023-10-16T14:32:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc100f80 0x7fc0dc0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:27.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a54d0 con 0x7fc0dc101940 2023-10-16T14:32:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a5640 con 0x7fc0dc106f90 2023-10-16T14:32:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0e0d79700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1a5800 con 0x7fc0dc100f80 2023-10-16T14:32:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0d9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0d9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39674/0 (socket says 172.21.15.178:39674) 2023-10-16T14:32:27.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0d9d9b700 1 -- 172.21.15.178:0/2824133792 learned_addr learned my addr 172.21.15.178:0/2824133792 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0dad9d700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.149+0000 7fc0da59c700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc100f80 0x7fc0dc0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 -- 172.21.15.178:0/2824133792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc100f80 msgr2=0x7fc0dc0731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc100f80 0x7fc0dc0731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 -- 172.21.15.178:0/2824133792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 msgr2=0x7fc0dc073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 -- 172.21.15.178:0/2824133792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0dc101940 2023-10-16T14:32:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0dad9d700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0d9d9b700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00098e0 tx=0x7fc0d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:27.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d001d070 con 0x7fc0dc101940 2023-10-16T14:32:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d0022470 con 0x7fc0dc101940 2023-10-16T14:32:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0017650 con 0x7fc0dc101940 2023-10-16T14:32:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc1a5a80 con 0x7fc0dc101940 2023-10-16T14:32:27.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.150+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc1a5e90 con 0x7fc0dc101940 2023-10-16T14:32:27.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.151+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc04ebb0 con 0x7fc0dc101940 2023-10-16T14:32:27.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.154+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fc0d0022850 con 0x7fc0dc101940 2023-10-16T14:32:27.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.155+0000 7fc0cb7fe700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 0x7fc0c40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:27.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.155+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc0d00ce190 con 0x7fc0dc101940 2023-10-16T14:32:27.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.155+0000 7fc0da59c700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 0x7fc0c40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:27.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.156+0000 7fc0da59c700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 0x7fc0c40a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009be0 tx=0x7fc0cc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:27.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.158+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0d00974c0 con 0x7fc0dc101940 2023-10-16T14:32:27.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.358+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0dc0746e0 con 0x7fc0dc101940 2023-10-16T14:32:27.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.360+0000 7fc0cb7fe700 1 -- 172.21.15.178:0/2824133792 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24546 (secure 0 0 0) 0x7fc0d009aae0 con 0x7fc0dc101940 2023-10-16T14:32:27.364 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:27.365 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":89,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:32:07.664080+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:32:27.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 msgr2=0x7fc0c40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:27.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 0x7fc0c40a3a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009be0 tx=0x7fc0cc009260 comp rx=0 tx=0).stop 2023-10-16T14:32:27.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 msgr2=0x7fc0dc0736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:27.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00098e0 tx=0x7fc0d000f690 comp rx=0 tx=0).stop 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 shutdown_connections 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c40a1590 0x7fc0c40a3a50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0dc106f90 0x7fc0dc073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0dc100f80 0x7fc0dc0731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 --2- 172.21.15.178:0/2824133792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0dc101940 0x7fc0dc0736f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:27.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 >> 172.21.15.178:0/2824133792 conn(0x7fc0dc0780a0 msgr2=0x7fc0dc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:27.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 shutdown_connections 2023-10-16T14:32:27.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:27.363+0000 7fc0e0d79700 1 -- 172.21.15.178:0/2824133792 wait complete. 2023-10-16T14:32:27.912 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-10-16T14:32:27.912 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd dump --format=json 2023-10-16T14:32:27.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:27 smithi187 ceph-mon[135297]: pgmap v299: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-10-16T14:32:27.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:27 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2824133792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:27.975 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:27 smithi178 ceph-mon[98495]: pgmap v299: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-10-16T14:32:27.975 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:27 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2824133792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:27 smithi186 ceph-mon[134892]: pgmap v299: 129 pgs: 129 active+clean; 649 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.0 KiB/s wr, 8 op/s 2023-10-16T14:32:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:27 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2824133792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:28.651 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:29.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:29 smithi178 ceph-mon[98495]: pgmap v300: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2023-10-16T14:32:29.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:29 smithi187 ceph-mon[135297]: pgmap v300: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2023-10-16T14:32:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:29 smithi186 ceph-mon[134892]: pgmap v300: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2023-10-16T14:32:30.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 -- 172.21.15.178:0/2927524980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 msgr2=0x7fb110107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:30.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 --2- 172.21.15.178:0/2927524980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb110107370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb1000099c0 tx=0x7fb100009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 -- 172.21.15.178:0/2927524980 shutdown_connections 2023-10-16T14:32:30.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 --2- 172.21.15.178:0/2927524980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb11010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 --2- 172.21.15.178:0/2927524980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb110107370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 --2- 172.21.15.178:0/2927524980 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 0x7fb110101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 -- 172.21.15.178:0/2927524980 >> 172.21.15.178:0/2927524980 conn(0x7fb1100780a0 msgr2=0x7fb1100784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:30.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 -- 172.21.15.178:0/2927524980 shutdown_connections 2023-10-16T14:32:30.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.171+0000 7fb115710700 1 -- 172.21.15.178:0/2927524980 wait complete. 2023-10-16T14:32:30.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.172+0000 7fb115710700 1 Processor -- start 2023-10-16T14:32:30.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.172+0000 7fb115710700 1 -- start start 2023-10-16T14:32:30.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 0x7fb1100731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:30.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb1100736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:30.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:30.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb110073fe0 con 0x7fb110106f90 2023-10-16T14:32:30.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101a5710 con 0x7fb110100f80 2023-10-16T14:32:30.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb115710700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1101a5880 con 0x7fb110101940 2023-10-16T14:32:30.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:30.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 0x7fb1100731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:30.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55830/0 (socket says 172.21.15.178:55830) 2023-10-16T14:32:30.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb1100736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:30.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10f7fe700 1 -- 172.21.15.178:0/458007318 learned_addr learned my addr 172.21.15.178:0/458007318 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:30.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 -- 172.21.15.178:0/458007318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 msgr2=0x7fb1100731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:30.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 0x7fb1100731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 -- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 msgr2=0x7fb1101a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:30.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10e7fc700 1 -- 172.21.15.178:0/458007318 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000095f0 con 0x7fb110101940 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.173+0000 7fb10f7fe700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb10e7fc700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb1100736f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9f0 tx=0x7fb10400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400c9c0 con 0x7fb110101940 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1101a5b00 con 0x7fb110101940 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb104010430 con 0x7fb110101940 2023-10-16T14:32:30.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb104018680 con 0x7fb110101940 2023-10-16T14:32:30.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.174+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1101a6050 con 0x7fb110101940 2023-10-16T14:32:30.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.175+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11004ebb0 con 0x7fb110101940 2023-10-16T14:32:30.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.177+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fb104010860 con 0x7fb110101940 2023-10-16T14:32:30.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.178+0000 7fb0f7fff700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 0x7fb0f80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:30.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.179+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb104014070 con 0x7fb110101940 2023-10-16T14:32:30.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.179+0000 7fb10effd700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 0x7fb0f80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:30.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.179+0000 7fb10effd700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 0x7fb0f80a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb100009990 tx=0x7fb100005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:30.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.181+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb1040963f0 con 0x7fb110101940 2023-10-16T14:32:30.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.381+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb110110fa0 con 0x7fb110101940 2023-10-16T14:32:30.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.383+0000 7fb0f7fff700 1 -- 172.21.15.178:0/458007318 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24546 (secure 0 0 0) 0x7fb104099a10 con 0x7fb110101940 2023-10-16T14:32:30.388 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:32:30.388 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":89,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:32:07.664080+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:32:30.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 msgr2=0x7fb0f80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 0x7fb0f80a3a50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb100009990 tx=0x7fb100005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 msgr2=0x7fb1100736f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb1100736f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb10400e9f0 tx=0x7fb10400ed00 comp rx=0 tx=0).stop 2023-10-16T14:32:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 shutdown_connections 2023-10-16T14:32:30.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0f80a1590 0x7fb0f80a3a50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb110106f90 0x7fb1101a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb110100f80 0x7fb1100731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 --2- 172.21.15.178:0/458007318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb110101940 0x7fb1100736f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:30.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 >> 172.21.15.178:0/458007318 conn(0x7fb1100780a0 msgr2=0x7fb1100ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 shutdown_connections 2023-10-16T14:32:30.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:30.386+0000 7fb115710700 1 -- 172.21.15.178:0/458007318 wait complete. 2023-10-16T14:32:30.949 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.0 flush_pg_stats 2023-10-16T14:32:30.949 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.1 flush_pg_stats 2023-10-16T14:32:30.950 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.2 flush_pg_stats 2023-10-16T14:32:30.950 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.3 flush_pg_stats 2023-10-16T14:32:30.951 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.4 flush_pg_stats 2023-10-16T14:32:30.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.5 flush_pg_stats 2023-10-16T14:32:30.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.6 flush_pg_stats 2023-10-16T14:32:30.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.7 flush_pg_stats 2023-10-16T14:32:30.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.8 flush_pg_stats 2023-10-16T14:32:30.954 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.9 flush_pg_stats 2023-10-16T14:32:30.954 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.10 flush_pg_stats 2023-10-16T14:32:30.955 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph tell osd.11 flush_pg_stats 2023-10-16T14:32:31.556 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:31 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/458007318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:31.557 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:31 smithi178 ceph-mon[98495]: pgmap v301: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-10-16T14:32:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:31 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/458007318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:31 smithi186 ceph-mon[134892]: pgmap v301: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-10-16T14:32:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:31 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/458007318' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:32:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:31 smithi187 ceph-mon[135297]: pgmap v301: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-10-16T14:32:32.271 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:32.538 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:32.555 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:32.896 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:33.528 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:33.529 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:33.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:33 smithi187 ceph-mon[135297]: pgmap v302: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-10-16T14:32:34.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:33 smithi178 ceph-mon[98495]: pgmap v302: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-10-16T14:32:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:33 smithi186 ceph-mon[134892]: pgmap v302: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-10-16T14:32:34.148 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:34.262 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:34.388 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:35.712 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:35 smithi178 ceph-mon[98495]: pgmap v303: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:35.814 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:35.857 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:35.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:35 smithi187 ceph-mon[135297]: pgmap v303: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:35 smithi186 ceph-mon[134892]: pgmap v303: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:36.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.172+0000 7fd7dd649700 1 -- 172.21.15.178:0/75589768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 msgr2=0x7fd7d8101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.172+0000 7fd7dd649700 1 --2- 172.21.15.178:0/75589768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8101400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0099c0 tx=0x7fd7cc009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 -- 172.21.15.178:0/75589768 shutdown_connections 2023-10-16T14:32:36.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 --2- 172.21.15.178:0/75589768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d8101940 0x7fd7d810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 --2- 172.21.15.178:0/75589768 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d8106f90 0x7fd7d8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 --2- 172.21.15.178:0/75589768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8101400 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 -- 172.21.15.178:0/75589768 >> 172.21.15.178:0/75589768 conn(0x7fd7d80780a0 msgr2=0x7fd7d80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:36.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 -- 172.21.15.178:0/75589768 shutdown_connections 2023-10-16T14:32:36.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.173+0000 7fd7dd649700 1 -- 172.21.15.178:0/75589768 wait complete. 2023-10-16T14:32:36.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 Processor -- start 2023-10-16T14:32:36.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 -- start start 2023-10-16T14:32:36.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d8101940 0x7fd7d81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d8106f90 0x7fd7d819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d819e140 con 0x7fd7d8100f80 2023-10-16T14:32:36.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d819e2b0 con 0x7fd7d8106f90 2023-10-16T14:32:36.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7dd649700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d819e470 con 0x7fd7d8101940 2023-10-16T14:32:36.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7d6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7d6ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55848/0 (socket says 172.21.15.178:55848) 2023-10-16T14:32:36.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7d6ffd700 1 -- 172.21.15.178:0/3110060106 learned_addr learned my addr 172.21.15.178:0/3110060106 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:36.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.174+0000 7fd7d67fc700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d8101940 0x7fd7d81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d77fe700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d8106f90 0x7fd7d819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 -- 172.21.15.178:0/3110060106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d8101940 msgr2=0x7fd7d81996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d8101940 0x7fd7d81996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 -- 172.21.15.178:0/3110060106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d8106f90 msgr2=0x7fd7d819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd7d8106f90 0x7fd7d819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0095f0 con 0x7fd7d8100f80 2023-10-16T14:32:36.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7d6ffd700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8199180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e9a0 tx=0x7fd7c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8009790 con 0x7fd7d8100f80 2023-10-16T14:32:36.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c8010430 con 0x7fd7d8100f80 2023-10-16T14:32:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d819e750 con 0x7fd7d8100f80 2023-10-16T14:32:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8018680 con 0x7fd7d8100f80 2023-10-16T14:32:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.175+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d81a5fb0 con 0x7fd7d8100f80 2023-10-16T14:32:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.176+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd7d8111110 con 0x7fd7d8100f80 2023-10-16T14:32:36.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.178+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fd7c80105a0 con 0x7fd7d8100f80 2023-10-16T14:32:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7bffff700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd7c00a1510 0x7fd7c00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd7c8014070 con 0x7fd7d8100f80 2023-10-16T14:32:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7d67fc700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd7c00a1510 0x7fd7c00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7bffff700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] conn(0x7fd7c00ac530 0x7fd7c00ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd7c00af0a0 con 0x7fd7c00ac530 2023-10-16T14:32:36.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c80cd960 con 0x7fd7d8100f80 2023-10-16T14:32:36.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7d77fe700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] conn(0x7fd7c00ac530 0x7fd7c00ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.180+0000 7fd7d67fc700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd7c00a1510 0x7fd7c00a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.181+0000 7fd7d77fe700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] conn(0x7fd7c00ac530 0x7fd7c00ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.182+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== osd.9 v2:172.21.15.178:6826/2115000831 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fd7c00af0a0 con 0x7fd7c00ac530 2023-10-16T14:32:36.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.198+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 --> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd7d804ebb0 con 0x7fd7c00ac530 2023-10-16T14:32:36.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7bffff700 1 -- 172.21.15.178:0/3110060106 <== osd.9 v2:172.21.15.178:6826/2115000831 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd7d804ebb0 con 0x7fd7c00ac530 2023-10-16T14:32:36.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] conn(0x7fd7c00ac530 msgr2=0x7fd7c00ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] conn(0x7fd7c00ac530 0x7fd7c00ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd7c00a1510 msgr2=0x7fd7c00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd7c00a1510 0x7fd7c00a39d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009990 tx=0x7fd7cc005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 msgr2=0x7fd7d8199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 --2- 172.21.15.178:0/3110060106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd7d8100f80 0x7fd7d8199180 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e9a0 tx=0x7fd7c800ecb0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7d77fe700 1 -- 172.21.15.178:0/3110060106 reap_dead start 2023-10-16T14:32:36.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 shutdown_connections 2023-10-16T14:32:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 >> 172.21.15.178:0/3110060106 conn(0x7fd7d80780a0 msgr2=0x7fd7d810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 shutdown_connections 2023-10-16T14:32:36.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.199+0000 7fd7dd649700 1 -- 172.21.15.178:0/3110060106 wait complete. 2023-10-16T14:32:36.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 -- 172.21.15.178:0/1400397821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40073620 msgr2=0x7fcf40073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 --2- 172.21.15.178:0/1400397821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40073620 0x7fcf40073a80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009990 tx=0x7fcf30009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 -- 172.21.15.178:0/1400397821 shutdown_connections 2023-10-16T14:32:36.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 --2- 172.21.15.178:0/1400397821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40111bc0 0x7fcf40113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 --2- 172.21.15.178:0/1400397821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf40074ef0 0x7fcf400730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 --2- 172.21.15.178:0/1400397821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40073620 0x7fcf40073a80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.761+0000 7fcf484de700 1 -- 172.21.15.178:0/1400397821 >> 172.21.15.178:0/1400397821 conn(0x7fcf400780a0 msgr2=0x7fcf400784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:36.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.762+0000 7fcf484de700 1 -- 172.21.15.178:0/1400397821 shutdown_connections 2023-10-16T14:32:36.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.762+0000 7fcf484de700 1 -- 172.21.15.178:0/1400397821 wait complete. 2023-10-16T14:32:36.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.762+0000 7fcf484de700 1 Processor -- start 2023-10-16T14:32:36.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 -- start start 2023-10-16T14:32:36.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 0x7fcf4019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40074ef0 0x7fcf4019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf40111bc0 0x7fcf401a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401a2620 con 0x7fcf40074ef0 2023-10-16T14:32:36.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401a2790 con 0x7fcf40073620 2023-10-16T14:32:36.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf484de700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401a2950 con 0x7fcf40111bc0 2023-10-16T14:32:36.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf45a79700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40074ef0 0x7fcf4019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf4627a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 0x7fcf4019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf4627a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 0x7fcf4019d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52016/0 (socket says 172.21.15.178:52016) 2023-10-16T14:32:36.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.763+0000 7fcf4627a700 1 -- 172.21.15.178:0/3809399223 learned_addr learned my addr 172.21.15.178:0/3809399223 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:36.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf46a7b700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf40111bc0 0x7fcf401a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 -- 172.21.15.178:0/3809399223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf40111bc0 msgr2=0x7fcf401a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf40111bc0 0x7fcf401a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 -- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40074ef0 msgr2=0x7fcf4019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40074ef0 0x7fcf4019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300095f0 con 0x7fcf40073620 2023-10-16T14:32:36.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf45a79700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf40074ef0 0x7fcf4019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:36.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf4627a700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 0x7fcf4019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9a0 tx=0x7fcf3c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c009790 con 0x7fcf40073620 2023-10-16T14:32:36.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf401a2c30 con 0x7fcf40073620 2023-10-16T14:32:36.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf3c010430 con 0x7fcf40073620 2023-10-16T14:32:36.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.764+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c018680 con 0x7fcf40073620 2023-10-16T14:32:36.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.765+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401aa490 con 0x7fcf40073620 2023-10-16T14:32:36.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.765+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf4004ebb0 con 0x7fcf40073620 2023-10-16T14:32:36.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.767+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fcf3c0105a0 con 0x7fcf40073620 2023-10-16T14:32:36.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf377fe700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf2c0a1510 0x7fcf2c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf3c014070 con 0x7fcf40073620 2023-10-16T14:32:36.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf45a79700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf2c0a1510 0x7fcf2c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf377fe700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] conn(0x7fcf2c0ac530 0x7fcf2c0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:36.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf2c0af0a0 con 0x7fcf2c0ac530 2023-10-16T14:32:36.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf3c0cd9d0 con 0x7fcf40073620 2023-10-16T14:32:36.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.769+0000 7fcf46a7b700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] conn(0x7fcf2c0ac530 0x7fcf2c0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:36.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.770+0000 7fcf45a79700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf2c0a1510 0x7fcf2c0a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf4019ec80 tx=0x7fcf30005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.770+0000 7fcf46a7b700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] conn(0x7fcf2c0ac530 0x7fcf2c0ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:36.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.771+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== osd.6 v2:172.21.15.178:6818/113445947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fcf2c0af0a0 con 0x7fcf2c0ac530 2023-10-16T14:32:36.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.788+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 --> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf401177a0 con 0x7fcf2c0ac530 2023-10-16T14:32:36.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.788+0000 7fcf377fe700 1 -- 172.21.15.178:0/3809399223 <== osd.6 v2:172.21.15.178:6818/113445947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcf3800f040 con 0x7fcf2c0ac530 2023-10-16T14:32:36.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.788+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] conn(0x7fcf2c0ac530 msgr2=0x7fcf2c0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.788+0000 7fcf484de700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] conn(0x7fcf2c0ac530 0x7fcf2c0ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf2c0a1510 msgr2=0x7fcf2c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf2c0a1510 0x7fcf2c0a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcf4019ec80 tx=0x7fcf30005fd0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.794 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 msgr2=0x7fcf4019d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:36.794 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 --2- 172.21.15.178:0/3809399223 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf40073620 0x7fcf4019d660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e9a0 tx=0x7fcf3c00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:32:36.794 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf46a7b700 1 -- 172.21.15.178:0/3809399223 reap_dead start 2023-10-16T14:32:36.794 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 shutdown_connections 2023-10-16T14:32:36.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 >> 172.21.15.178:0/3809399223 conn(0x7fcf400780a0 msgr2=0x7fcf40110640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:36.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 shutdown_connections 2023-10-16T14:32:36.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:36.789+0000 7fcf484de700 1 -- 172.21.15.178:0/3809399223 wait complete. 2023-10-16T14:32:37.883 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:37.886 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:37 smithi178 ceph-mon[98495]: pgmap v304: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:37.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:37 smithi187 ceph-mon[135297]: pgmap v304: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:37 smithi186 ceph-mon[134892]: pgmap v304: 129 pgs: 129 active+clean; 653 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:32:38.914 INFO:teuthology.orchestra.run.smithi178.stdout:197568495663 2023-10-16T14:32:38.915 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.6 2023-10-16T14:32:39.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:39 smithi187 ceph-mon[135297]: pgmap v305: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-16T14:32:39.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:39 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:32:40.023 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:39 smithi178 ceph-mon[98495]: pgmap v305: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-16T14:32:40.024 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:39 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:32:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:39 smithi186 ceph-mon[134892]: pgmap v305: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 6 op/s 2023-10-16T14:32:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:39 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:32:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 -- 172.21.15.178:0/1086948202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 msgr2=0x7f1e2010fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 --2- 172.21.15.178:0/1086948202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 0x7f1e2010fee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800b0d0 tx=0x7f1e1800b3e0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 -- 172.21.15.178:0/1086948202 shutdown_connections 2023-10-16T14:32:40.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 --2- 172.21.15.178:0/1086948202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 0x7f1e2010fee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 --2- 172.21.15.178:0/1086948202 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e200734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 --2- 172.21.15.178:0/1086948202 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e20073a00 0x7f1e2010d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 -- 172.21.15.178:0/1086948202 >> 172.21.15.178:0/1086948202 conn(0x7f1e20078f30 msgr2=0x7f1e20079340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:40.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 -- 172.21.15.178:0/1086948202 shutdown_connections 2023-10-16T14:32:40.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.372+0000 7f1e2514c700 1 -- 172.21.15.178:0/1086948202 wait complete. 2023-10-16T14:32:40.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 Processor -- start 2023-10-16T14:32:40.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 -- start start 2023-10-16T14:32:40.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e20199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e20073a00 0x7f1e201998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 0x7f1e2019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2019e320 con 0x7f1e20073a00 2023-10-16T14:32:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2019e490 con 0x7f1e200730e0 2023-10-16T14:32:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.374+0000 7f1e2514c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2019e650 con 0x7f1e2010daf0 2023-10-16T14:32:40.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e20199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e20073a00 0x7f1e201998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e20199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:49614/0 (socket says 172.21.15.178:49614) 2023-10-16T14:32:40.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 -- 172.21.15.178:0/2345663770 learned_addr learned my addr 172.21.15.178:0/2345663770 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:40.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1f59e700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 0x7f1e2019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 -- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 msgr2=0x7f1e2019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e2010daf0 0x7f1e2019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 -- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e20073a00 msgr2=0x7f1e201998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e20073a00 0x7f1e201998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.375+0000 7f1e1ed9d700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10009520 con 0x7f1e200730e0 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e1ed9d700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e20199360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eb30 tx=0x7f1e1000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10009880 con 0x7f1e200730e0 2023-10-16T14:32:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e100044e0 con 0x7f1e200730e0 2023-10-16T14:32:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10003d50 con 0x7f1e200730e0 2023-10-16T14:32:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e18009d00 con 0x7f1e200730e0 2023-10-16T14:32:40.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.376+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201a5fc0 con 0x7f1e200730e0 2023-10-16T14:32:40.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.377+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e20113320 con 0x7f1e200730e0 2023-10-16T14:32:40.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.379+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f1e10003ef0 con 0x7f1e200730e0 2023-10-16T14:32:40.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.380+0000 7f1e07fff700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1e080a1510 0x7f1e080a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1e10014070 con 0x7f1e200730e0 2023-10-16T14:32:40.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e1e59c700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1e080a1510 0x7f1e080a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e07fff700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] conn(0x7f1e080ac530 0x7f1e080ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e080af0a0 con 0x7f1e080ac530 2023-10-16T14:32:40.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e100cfaa0 con 0x7f1e200730e0 2023-10-16T14:32:40.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.381+0000 7f1e1f59e700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] conn(0x7f1e080ac530 0x7f1e080ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.382+0000 7f1e1e59c700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1e080a1510 0x7f1e080a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e2019a980 tx=0x7f1e14009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.382+0000 7f1e1f59e700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] conn(0x7f1e080ac530 0x7f1e080ae9d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.383+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== osd.11 v2:172.21.15.187:6824/433464338 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f1e080af0a0 con 0x7f1e080ac530 2023-10-16T14:32:40.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.411+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 --> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e2004ebb0 con 0x7f1e080ac530 2023-10-16T14:32:40.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e07fff700 1 -- 172.21.15.178:0/2345663770 <== osd.11 v2:172.21.15.187:6824/433464338 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1e2004ebb0 con 0x7f1e080ac530 2023-10-16T14:32:40.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] conn(0x7f1e080ac530 msgr2=0x7f1e080ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] conn(0x7f1e080ac530 0x7f1e080ae9d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1e080a1510 msgr2=0x7f1e080a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1e080a1510 0x7f1e080a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e2019a980 tx=0x7f1e14009260 comp rx=0 tx=0).stop 2023-10-16T14:32:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 msgr2=0x7f1e20199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e2514c700 1 --2- 172.21.15.178:0/2345663770 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1e200730e0 0x7f1e20199360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e1000eb30 tx=0x7f1e1000ee40 comp rx=0 tx=0).stop 2023-10-16T14:32:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.412+0000 7f1e1f59e700 1 -- 172.21.15.178:0/2345663770 reap_dead start 2023-10-16T14:32:40.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.413+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 shutdown_connections 2023-10-16T14:32:40.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.413+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 >> 172.21.15.178:0/2345663770 conn(0x7f1e20078f30 msgr2=0x7f1e20105550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:40.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.413+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 shutdown_connections 2023-10-16T14:32:40.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.413+0000 7f1e2514c700 1 -- 172.21.15.178:0/2345663770 wait complete. 2023-10-16T14:32:40.914 INFO:teuthology.orchestra.run.smithi178.stdout:283467841569 2023-10-16T14:32:40.914 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.9 2023-10-16T14:32:40.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 -- 172.21.15.178:0/2183775737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc074ef0 msgr2=0x7fcddc0730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 --2- 172.21.15.178:0/2183775737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc074ef0 0x7fcddc0730e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009990 tx=0x7fcdd8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 -- 172.21.15.178:0/2183775737 shutdown_connections 2023-10-16T14:32:40.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 --2- 172.21.15.178:0/2183775737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcddc111bc0 0x7fcddc113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 --2- 172.21.15.178:0/2183775737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc074ef0 0x7fcddc0730e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 --2- 172.21.15.178:0/2183775737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc073620 0x7fcddc073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 -- 172.21.15.178:0/2183775737 >> 172.21.15.178:0/2183775737 conn(0x7fcddc0780a0 msgr2=0x7fcddc0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:40.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 -- 172.21.15.178:0/2183775737 shutdown_connections 2023-10-16T14:32:40.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.238+0000 7fcde4640700 1 -- 172.21.15.178:0/2183775737 wait complete. 2023-10-16T14:32:40.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.239+0000 7fcde4640700 1 Processor -- start 2023-10-16T14:32:40.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.239+0000 7fcde4640700 1 -- start start 2023-10-16T14:32:40.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 0x7fcddc19d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc074ef0 0x7fcddc19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcddc111bc0 0x7fcddc1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1a25e0 con 0x7fcddc074ef0 2023-10-16T14:32:40.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1a2750 con 0x7fcddc111bc0 2023-10-16T14:32:40.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde4640700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc1a2910 con 0x7fcddc073620 2023-10-16T14:32:40.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 0x7fcddc19d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 0x7fcddc19d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:41446/0 (socket says 172.21.15.178:41446) 2023-10-16T14:32:40.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 -- 172.21.15.178:0/3868751759 learned_addr learned my addr 172.21.15.178:0/3868751759 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:40.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde2bdd700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcddc111bc0 0x7fcddc1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 -- 172.21.15.178:0/3868751759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcddc111bc0 msgr2=0x7fcddc1a1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcddc111bc0 0x7fcddc1a1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 -- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc074ef0 msgr2=0x7fcddc19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde1bdb700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc074ef0 0x7fcddc19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc074ef0 0x7fcddc19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.240+0000 7fcde23dc700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd4009c40 con 0x7fcddc073620 2023-10-16T14:32:40.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcde1bdb700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcddc074ef0 0x7fcddc19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:32:40.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcde23dc700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 0x7fcddc19d620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd801d070 con 0x7fcddc073620 2023-10-16T14:32:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdd8022470 con 0x7fcddc073620 2023-10-16T14:32:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd80095f0 con 0x7fcddc073620 2023-10-16T14:32:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8017650 con 0x7fcddc073620 2023-10-16T14:32:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.241+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc1aa280 con 0x7fcddc073620 2023-10-16T14:32:40.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.242+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcddc1177a0 con 0x7fcddc073620 2023-10-16T14:32:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.244+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fcdd800fa50 con 0x7fcddc073620 2023-10-16T14:32:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcdd37fe700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcdc80a1510 0x7fcdc80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcdd80cf060 con 0x7fcddc073620 2023-10-16T14:32:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcde1bdb700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcdc80a1510 0x7fcdc80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcdd37fe700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] conn(0x7fcdc80ac530 0x7fcdc80ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdc80af0a0 con 0x7fcdc80ac530 2023-10-16T14:32:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdd80cf470 con 0x7fcddc073620 2023-10-16T14:32:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.246+0000 7fcde2bdd700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] conn(0x7fcdc80ac530 0x7fcdc80ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.247+0000 7fcde1bdb700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcdc80a1510 0x7fcdc80a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcddc19ec40 tx=0x7fcdcc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.247+0000 7fcde2bdd700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] conn(0x7fcdc80ac530 0x7fcdc80ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:40.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.248+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== osd.0 v2:172.21.15.178:6802/1189154044 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fcdc80af0a0 con 0x7fcdc80ac530 2023-10-16T14:32:40.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.269+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 --> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcddc04ebb0 con 0x7fcdc80ac530 2023-10-16T14:32:40.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcdd37fe700 1 -- 172.21.15.178:0/3868751759 <== osd.0 v2:172.21.15.178:6802/1189154044 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcddc04ebb0 con 0x7fcdc80ac530 2023-10-16T14:32:40.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] conn(0x7fcdc80ac530 msgr2=0x7fcdc80ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] conn(0x7fcdc80ac530 0x7fcdc80ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:40.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcdc80a1510 msgr2=0x7fcdc80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcdc80a1510 0x7fcdc80a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcddc19ec40 tx=0x7fcdcc009260 comp rx=0 tx=0).stop 2023-10-16T14:32:40.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 msgr2=0x7fcddc19d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:40.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde4640700 1 --2- 172.21.15.178:0/3868751759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcddc073620 0x7fcddc19d620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8000c00 tx=0x7fcdd8005740 comp rx=0 tx=0).stop 2023-10-16T14:32:40.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.270+0000 7fcde2bdd700 1 -- 172.21.15.178:0/3868751759 reap_dead start 2023-10-16T14:32:40.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.271+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 shutdown_connections 2023-10-16T14:32:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.271+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 >> 172.21.15.178:0/3868751759 conn(0x7fcddc0780a0 msgr2=0x7fcddc1106c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.271+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 shutdown_connections 2023-10-16T14:32:40.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.271+0000 7fcde4640700 1 -- 172.21.15.178:0/3868751759 wait complete. 2023-10-16T14:32:41.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 -- 172.21.15.178:0/1887566262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0101940 msgr2=0x7f21a010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 --2- 172.21.15.178:0/1887566262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0101940 0x7f21a010ddb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f219800b0a0 tx=0x7f219800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 -- 172.21.15.178:0/1887566262 shutdown_connections 2023-10-16T14:32:41.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 --2- 172.21.15.178:0/1887566262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0101940 0x7f21a010ddb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 --2- 172.21.15.178:0/1887566262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0106f90 0x7f21a0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 --2- 172.21.15.178:0/1887566262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a0100f80 0x7f21a0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 -- 172.21.15.178:0/1887566262 >> 172.21.15.178:0/1887566262 conn(0x7f21a00780a0 msgr2=0x7f21a00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:41.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 -- 172.21.15.178:0/1887566262 shutdown_connections 2023-10-16T14:32:41.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.546+0000 7f21a52d2700 1 -- 172.21.15.178:0/1887566262 wait complete. 2023-10-16T14:32:41.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.547+0000 7f21a52d2700 1 Processor -- start 2023-10-16T14:32:41.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.547+0000 7f21a52d2700 1 -- start start 2023-10-16T14:32:41.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0100f80 0x7f21a01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a0101940 0x7f21a0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 0x7f21a019dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019e2b0 con 0x7f21a0100f80 2023-10-16T14:32:41.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019e420 con 0x7f21a0101940 2023-10-16T14:32:41.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f21a52d2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a019e5e0 con 0x7f21a0106f90 2023-10-16T14:32:41.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 0x7f21a019dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f219f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 0x7f21a019dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:41488/0 (socket says 172.21.15.178:41488) 2023-10-16T14:32:41.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.548+0000 7f219f7fe700 1 -- 172.21.15.178:0/2114219120 learned_addr learned my addr 172.21.15.178:0/2114219120 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:41.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219effd700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0100f80 0x7f21a01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 -- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a0101940 msgr2=0x7f21a0199830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T14:32:41.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f21a0101940 0x7f21a0199830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 -- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0100f80 msgr2=0x7f21a01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0100f80 0x7f21a01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2198009d00 con 0x7f21a0106f90 2023-10-16T14:32:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219effd700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21a0100f80 0x7f21a01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:32:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f219f7fe700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 0x7f21a019dca0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2198009fd0 tx=0x7f21980078a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800e040 con 0x7f21a0106f90 2023-10-16T14:32:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2198004e60 con 0x7f21a0106f90 2023-10-16T14:32:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198005280 con 0x7f21a0106f90 2023-10-16T14:32:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.549+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a019e860 con 0x7f21a0106f90 2023-10-16T14:32:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.550+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a01a6120 con 0x7f21a0106f90 2023-10-16T14:32:41.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.551+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f21a019eb40 con 0x7f21a0106f90 2023-10-16T14:32:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.553+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f21980053e0 con 0x7f21a0106f90 2023-10-16T14:32:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.555+0000 7f2187fff700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f21880a1590 0x7f21880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f21980ce900 con 0x7f21a0106f90 2023-10-16T14:32:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f219effd700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f21880a1590 0x7f21880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f2187fff700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] conn(0x7f21880ac530 0x7f21880ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f21880af0a0 con 0x7f21880ac530 2023-10-16T14:32:41.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21980cec70 con 0x7f21a0106f90 2023-10-16T14:32:41.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.556+0000 7f219e7fc700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] conn(0x7f21880ac530 0x7f21880ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.557+0000 7f219effd700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f21880a1590 0x7f21880a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2190005fd0 tx=0x7f2190005ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.557+0000 7f219e7fc700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] conn(0x7f21880ac530 0x7f21880ae9d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.558+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== osd.4 v2:172.21.15.186:6808/699064619 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f21880af0a0 con 0x7f21880ac530 2023-10-16T14:32:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.584+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 --> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f21a01111a0 con 0x7f21880ac530 2023-10-16T14:32:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.584+0000 7f2187fff700 1 -- 172.21.15.178:0/2114219120 <== osd.4 v2:172.21.15.186:6808/699064619 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2194009b00 con 0x7f21880ac530 2023-10-16T14:32:41.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] conn(0x7f21880ac530 msgr2=0x7f21880ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] conn(0x7f21880ac530 0x7f21880ae9d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f21880a1590 msgr2=0x7f21880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f21880a1590 0x7f21880a3a50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2190005fd0 tx=0x7f2190005ee0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 msgr2=0x7f21a019dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 --2- 172.21.15.178:0/2114219120 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f21a0106f90 0x7f21a019dca0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2198009fd0 tx=0x7f21980078a0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f219f7fe700 1 -- 172.21.15.178:0/2114219120 reap_dead start 2023-10-16T14:32:41.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 shutdown_connections 2023-10-16T14:32:41.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 >> 172.21.15.178:0/2114219120 conn(0x7f21a00780a0 msgr2=0x7f21a00ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:41.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 shutdown_connections 2023-10-16T14:32:41.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.585+0000 7f21a52d2700 1 -- 172.21.15.178:0/2114219120 wait complete. 2023-10-16T14:32:41.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 -- 172.21.15.178:0/1072663606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec100f80 msgr2=0x7f20ec101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 --2- 172.21.15.178:0/1072663606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec100f80 0x7f20ec101400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f20e00099c0 tx=0x7f20e0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 -- 172.21.15.178:0/1072663606 shutdown_connections 2023-10-16T14:32:41.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 --2- 172.21.15.178:0/1072663606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec101940 0x7f20ec10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 --2- 172.21.15.178:0/1072663606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec106f90 0x7f20ec107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 --2- 172.21.15.178:0/1072663606 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec100f80 0x7f20ec101400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 -- 172.21.15.178:0/1072663606 >> 172.21.15.178:0/1072663606 conn(0x7f20ec0780a0 msgr2=0x7f20ec0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 -- 172.21.15.178:0/1072663606 shutdown_connections 2023-10-16T14:32:41.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.565+0000 7f20f2393700 1 -- 172.21.15.178:0/1072663606 wait complete. 2023-10-16T14:32:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.566+0000 7f20f2393700 1 Processor -- start 2023-10-16T14:32:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.566+0000 7f20f2393700 1 -- start start 2023-10-16T14:32:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec100f80 0x7f20ec199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 0x7f20ec199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec106f90 0x7f20ec19dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec19e1f0 con 0x7f20ec101940 2023-10-16T14:32:41.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec19e360 con 0x7f20ec106f90 2023-10-16T14:32:41.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20f2393700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec19e520 con 0x7f20ec100f80 2023-10-16T14:32:41.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 0x7f20ec199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20eb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 0x7f20ec199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60206/0 (socket says 172.21.15.178:60206) 2023-10-16T14:32:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20eb7fe700 1 -- 172.21.15.178:0/182287539 learned_addr learned my addr 172.21.15.178:0/182287539 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.567+0000 7f20ebfff700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec100f80 0x7f20ec199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 -- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec106f90 msgr2=0x7f20ec19dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f20ec106f90 0x7f20ec19dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 -- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 msgr2=0x7f20ec199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 0x7f20ec199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00095f0 con 0x7f20ec100f80 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20eb7fe700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20ec101940 0x7f20ec199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20ebfff700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec100f80 0x7f20ec199230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00b450 tx=0x7f20dc00b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc011830 con 0x7f20ec100f80 2023-10-16T14:32:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20dc00be90 con 0x7f20ec100f80 2023-10-16T14:32:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec19e800 con 0x7f20ec100f80 2023-10-16T14:32:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.568+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc010d50 con 0x7f20ec100f80 2023-10-16T14:32:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.569+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec1a6060 con 0x7f20ec100f80 2023-10-16T14:32:41.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.570+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20ec1111e0 con 0x7f20ec100f80 2023-10-16T14:32:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.573+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f20dc010380 con 0x7f20ec100f80 2023-10-16T14:32:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.574+0000 7f20e97fa700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f20d40a1510 0x7f20d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.574+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20dc0cc9e0 con 0x7f20ec100f80 2023-10-16T14:32:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20eb7fe700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f20d40a1510 0x7f20d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20e97fa700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] conn(0x7f20d40ac530 0x7f20d40ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20d40af0a0 con 0x7f20d40ac530 2023-10-16T14:32:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20dc0d12c0 con 0x7f20ec100f80 2023-10-16T14:32:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20eb7fe700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f20d40a1510 0x7f20d40a39d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20ec19a850 tx=0x7f20e0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.575+0000 7f20f0930700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] conn(0x7f20d40ac530 0x7f20d40ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:41.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.576+0000 7f20f0930700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] conn(0x7f20d40ac530 0x7f20d40ae9d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.577+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== osd.7 v2:172.21.15.186:6816/4049803928 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f20d40af0a0 con 0x7f20d40ac530 2023-10-16T14:32:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.595+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 --> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20ec04ebb0 con 0x7f20d40ac530 2023-10-16T14:32:41.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.596+0000 7f20e97fa700 1 -- 172.21.15.178:0/182287539 <== osd.7 v2:172.21.15.186:6816/4049803928 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f20ec04ebb0 con 0x7f20d40ac530 2023-10-16T14:32:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.596+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] conn(0x7f20d40ac530 msgr2=0x7f20d40ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.596+0000 7f20f2393700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] conn(0x7f20d40ac530 0x7f20d40ae9d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f20d40a1510 msgr2=0x7f20d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f20d40a1510 0x7f20d40a39d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f20ec19a850 tx=0x7f20e0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:32:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec100f80 msgr2=0x7f20ec199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 --2- 172.21.15.178:0/182287539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20ec100f80 0x7f20ec199230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00b450 tx=0x7f20dc00b760 comp rx=0 tx=0).stop 2023-10-16T14:32:41.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f0930700 1 -- 172.21.15.178:0/182287539 reap_dead start 2023-10-16T14:32:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 shutdown_connections 2023-10-16T14:32:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 >> 172.21.15.178:0/182287539 conn(0x7f20ec0780a0 msgr2=0x7f20ec073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 shutdown_connections 2023-10-16T14:32:41.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:40.597+0000 7f20f2393700 1 -- 172.21.15.178:0/182287539 wait complete. 2023-10-16T14:32:41.463 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:41 smithi178 ceph-mon[98495]: pgmap v306: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:41 smithi186 ceph-mon[134892]: pgmap v306: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:41 smithi187 ceph-mon[135297]: pgmap v306: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:42.338 INFO:teuthology.orchestra.run.smithi178.stdout:219043332139 2023-10-16T14:32:42.339 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.7 2023-10-16T14:32:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 -- 172.21.15.178:0/611194963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 msgr2=0x7f3d40073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 --2- 172.21.15.178:0/611194963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 0x7f3d40073a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3d34009990 tx=0x7f3d34009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 -- 172.21.15.178:0/611194963 shutdown_connections 2023-10-16T14:32:42.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 --2- 172.21.15.178:0/611194963 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40111bc0 0x7f3d40113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 --2- 172.21.15.178:0/611194963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d40074ef0 0x7f3d400730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 --2- 172.21.15.178:0/611194963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 0x7f3d40073a80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 -- 172.21.15.178:0/611194963 >> 172.21.15.178:0/611194963 conn(0x7f3d400780a0 msgr2=0x7f3d400784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 -- 172.21.15.178:0/611194963 shutdown_connections 2023-10-16T14:32:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.236+0000 7f3d44c54700 1 -- 172.21.15.178:0/611194963 wait complete. 2023-10-16T14:32:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.237+0000 7f3d44c54700 1 Processor -- start 2023-10-16T14:32:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 -- start start 2023-10-16T14:32:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 0x7f3d4019d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 0x7f3d4019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d40111bc0 0x7f3d401a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401a2610 con 0x7f3d40073620 2023-10-16T14:32:42.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401a2780 con 0x7f3d40074ef0 2023-10-16T14:32:42.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d44c54700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d401a2940 con 0x7f3d40111bc0 2023-10-16T14:32:42.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d3dd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 0x7f3d4019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:42.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d3dd9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 0x7f3d4019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:49670/0 (socket says 172.21.15.178:49670) 2023-10-16T14:32:42.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d3dd9b700 1 -- 172.21.15.178:0/779798356 learned_addr learned my addr 172.21.15.178:0/779798356 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.238+0000 7f3d3e59c700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 0x7f3d4019d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3ed9d700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d40111bc0 0x7f3d401a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 -- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d40111bc0 msgr2=0x7f3d401a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d40111bc0 0x7f3d401a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 -- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 msgr2=0x7f3d4019d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d40073620 0x7f3d4019d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340095f0 con 0x7f3d40074ef0 2023-10-16T14:32:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d3dd9b700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 0x7f3d4019db90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005910 tx=0x7f3d34005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3401d070 con 0x7f3d40074ef0 2023-10-16T14:32:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d34022470 con 0x7f3d40074ef0 2023-10-16T14:32:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d401a2bc0 con 0x7f3d40074ef0 2023-10-16T14:32:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d34017650 con 0x7f3d40074ef0 2023-10-16T14:32:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.239+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d401aa480 con 0x7f3d40074ef0 2023-10-16T14:32:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.240+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d4004ebb0 con 0x7f3d40074ef0 2023-10-16T14:32:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.242+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f3d3400fb00 con 0x7f3d40074ef0 2023-10-16T14:32:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.243+0000 7f3d2f7fe700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3d280a1590 0x7f3d280a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d340ce570 con 0x7f3d40074ef0 2023-10-16T14:32:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d3e59c700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3d280a1590 0x7f3d280a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d2f7fe700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] conn(0x7f3d280ac480 0x7f3d280ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d280aeff0 con 0x7f3d280ac480 2023-10-16T14:32:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d340d0b50 con 0x7f3d40074ef0 2023-10-16T14:32:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d3ed9d700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] conn(0x7f3d280ac480 0x7f3d280ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.244+0000 7f3d3e59c700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3d280a1590 0x7f3d280a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009c60 tx=0x7f3d30009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:42.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.245+0000 7f3d3ed9d700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] conn(0x7f3d280ac480 0x7f3d280ae920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:42.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.246+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== osd.2 v2:172.21.15.187:6800/2818376874 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f3d280aeff0 con 0x7f3d280ac480 2023-10-16T14:32:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.263+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 --> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d401177f0 con 0x7f3d280ac480 2023-10-16T14:32:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.264+0000 7f3d2f7fe700 1 -- 172.21.15.178:0/779798356 <== osd.2 v2:172.21.15.187:6800/2818376874 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d3800f040 con 0x7f3d280ac480 2023-10-16T14:32:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.264+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] conn(0x7f3d280ac480 msgr2=0x7f3d280ae920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.264+0000 7f3d44c54700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] conn(0x7f3d280ac480 0x7f3d280ae920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3d280a1590 msgr2=0x7f3d280a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3d280a1590 0x7f3d280a3a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009c60 tx=0x7f3d30009260 comp rx=0 tx=0).stop 2023-10-16T14:32:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 msgr2=0x7f3d4019db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 --2- 172.21.15.178:0/779798356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3d40074ef0 0x7f3d4019db90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3d34005910 tx=0x7f3d34005720 comp rx=0 tx=0).stop 2023-10-16T14:32:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d3ed9d700 1 -- 172.21.15.178:0/779798356 reap_dead start 2023-10-16T14:32:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 shutdown_connections 2023-10-16T14:32:42.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 >> 172.21.15.178:0/779798356 conn(0x7f3d400780a0 msgr2=0x7f3d40110620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:42.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 shutdown_connections 2023-10-16T14:32:42.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:41.265+0000 7f3d44c54700 1 -- 172.21.15.178:0/779798356 wait complete. 2023-10-16T14:32:42.966 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:42.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:42.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:42.967 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:42 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:42 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:42 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:32:43.205 INFO:teuthology.orchestra.run.smithi178.stdout:34359738443 2023-10-16T14:32:43.205 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.0 2023-10-16T14:32:43.773 INFO:teuthology.orchestra.run.smithi178.stdout:141733920826 2023-10-16T14:32:43.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.4 2023-10-16T14:32:44.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:43 smithi178 ceph-mon[98495]: pgmap v307: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-16T14:32:44.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:43 smithi186 ceph-mon[134892]: pgmap v307: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-16T14:32:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:43 smithi187 ceph-mon[135297]: pgmap v307: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 5 op/s 2023-10-16T14:32:44.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.009+0000 7fab62483700 1 -- 172.21.15.178:0/1331430308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c1033b0 msgr2=0x7fab5c103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.009+0000 7fab62483700 1 --2- 172.21.15.178:0/1331430308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c1033b0 0x7fab5c103790 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0098f0 tx=0x7fab4c009c00 comp rx=0 tx=0).stop 2023-10-16T14:32:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 -- 172.21.15.178:0/1331430308 shutdown_connections 2023-10-16T14:32:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 --2- 172.21.15.178:0/1331430308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c108490 0x7fab5c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 --2- 172.21.15.178:0/1331430308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c1033b0 0x7fab5c103790 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 --2- 172.21.15.178:0/1331430308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c103d60 0x7fab5c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 -- 172.21.15.178:0/1331430308 >> 172.21.15.178:0/1331430308 conn(0x7fab5c0fec20 msgr2=0x7fab5c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 -- 172.21.15.178:0/1331430308 shutdown_connections 2023-10-16T14:32:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.010+0000 7fab62483700 1 -- 172.21.15.178:0/1331430308 wait complete. 2023-10-16T14:32:44.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.011+0000 7fab62483700 1 Processor -- start 2023-10-16T14:32:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.011+0000 7fab62483700 1 -- start start 2023-10-16T14:32:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c1033b0 0x7fab5c0753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c103d60 0x7fab5c075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 0x7fab5c075e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:44.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a99d0 con 0x7fab5c1033b0 2023-10-16T14:32:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a9b10 con 0x7fab5c108490 2023-10-16T14:32:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab62483700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1a9cd0 con 0x7fab5c103d60 2023-10-16T14:32:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c1033b0 0x7fab5c0753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab60a20700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 0x7fab5c075e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:44.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab5bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c1033b0 0x7fab5c0753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60248/0 (socket says 172.21.15.178:60248) 2023-10-16T14:32:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab5bfff700 1 -- 172.21.15.178:0/1218342799 learned_addr learned my addr 172.21.15.178:0/1218342799 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab5b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c103d60 0x7fab5c075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.013+0000 7fab60a20700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 0x7fab5c075e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:49680/0 (socket says 172.21.15.178:49680) 2023-10-16T14:32:44.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 -- 172.21.15.178:0/1218342799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c103d60 msgr2=0x7fab5c075900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c103d60 0x7fab5c075900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 -- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c1033b0 msgr2=0x7fab5c0753c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab5c1033b0 0x7fab5c0753c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c009520 con 0x7fab5c108490 2023-10-16T14:32:44.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab5b7fe700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab5c103d60 0x7fab5c075900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab60a20700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 0x7fab5c075e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab5400efc0 tx=0x7fab5400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54018070 con 0x7fab5c108490 2023-10-16T14:32:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab5400f040 con 0x7fab5c108490 2023-10-16T14:32:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5400cd30 con 0x7fab5c108490 2023-10-16T14:32:44.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.014+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c1a9f50 con 0x7fab5c108490 2023-10-16T14:32:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.015+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c1aa3d0 con 0x7fab5c108490 2023-10-16T14:32:44.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.016+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab5c113330 con 0x7fab5c108490 2023-10-16T14:32:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.019+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fab54003a00 con 0x7fab5c108490 2023-10-16T14:32:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.022+0000 7fab597fa700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fab440a1590 0x7fab440a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:44.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.022+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fab540cc880 con 0x7fab5c108490 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.022+0000 7fab597fa700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] conn(0x7fab440ac5b0 0x7fab440aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.022+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab440af120 con 0x7fab440ac5b0 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.022+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab540cee60 con 0x7fab5c108490 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.023+0000 7fab5bfff700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fab440a1590 0x7fab440a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.023+0000 7fab5b7fe700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] conn(0x7fab440ac5b0 0x7fab440aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:44.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.024+0000 7fab5b7fe700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] conn(0x7fab440ac5b0 0x7fab440aea50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.024+0000 7fab5bfff700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fab440a1590 0x7fab440a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009fd0 tx=0x7fab4c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.025+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== osd.10 v2:172.21.15.186:6824/2093939166 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fab440af120 con 0x7fab440ac5b0 2023-10-16T14:32:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.045+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 --> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab5c04ebb0 con 0x7fab440ac5b0 2023-10-16T14:32:44.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.046+0000 7fab597fa700 1 -- 172.21.15.178:0/1218342799 <== osd.10 v2:172.21.15.186:6824/2093939166 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fab5c04ebb0 con 0x7fab440ac5b0 2023-10-16T14:32:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] conn(0x7fab440ac5b0 msgr2=0x7fab440aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] conn(0x7fab440ac5b0 0x7fab440aea50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fab440a1590 msgr2=0x7fab440a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fab440a1590 0x7fab440a3a50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009fd0 tx=0x7fab4c019040 comp rx=0 tx=0).stop 2023-10-16T14:32:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 msgr2=0x7fab5c075e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 --2- 172.21.15.178:0/1218342799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fab5c108490 0x7fab5c075e40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fab5400efc0 tx=0x7fab5400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab60a20700 1 -- 172.21.15.178:0/1218342799 reap_dead start 2023-10-16T14:32:44.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 shutdown_connections 2023-10-16T14:32:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 >> 172.21.15.178:0/1218342799 conn(0x7fab5c0fec20 msgr2=0x7fab5c0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.047+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 shutdown_connections 2023-10-16T14:32:44.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:44.048+0000 7fab62483700 1 -- 172.21.15.178:0/1218342799 wait complete. 2023-10-16T14:32:44.376 INFO:teuthology.orchestra.run.smithi178.stdout:335007449114 2023-10-16T14:32:44.377 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.11 2023-10-16T14:32:45.275 INFO:teuthology.orchestra.run.smithi178.stdout:304942678046 2023-10-16T14:32:45.275 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.10 2023-10-16T14:32:45.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 -- 172.21.15.178:0/3757356230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0101940 msgr2=0x7fafc010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 --2- 172.21.15.178:0/3757356230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0101940 0x7fafc010ddb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009fc0 tx=0x7fafb800b330 comp rx=0 tx=0).stop 2023-10-16T14:32:45.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 -- 172.21.15.178:0/3757356230 shutdown_connections 2023-10-16T14:32:45.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 --2- 172.21.15.178:0/3757356230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0101940 0x7fafc010ddb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 --2- 172.21.15.178:0/3757356230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafc0106f90 0x7fafc0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 --2- 172.21.15.178:0/3757356230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc0100f80 0x7fafc0101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 -- 172.21.15.178:0/3757356230 >> 172.21.15.178:0/3757356230 conn(0x7fafc00780a0 msgr2=0x7fafc00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:45.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 -- 172.21.15.178:0/3757356230 shutdown_connections 2023-10-16T14:32:45.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.679+0000 7fafc742d700 1 -- 172.21.15.178:0/3757356230 wait complete. 2023-10-16T14:32:45.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.680+0000 7fafc742d700 1 Processor -- start 2023-10-16T14:32:45.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.680+0000 7fafc742d700 1 -- start start 2023-10-16T14:32:45.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 0x7fafc01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:45.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafc0101940 0x7fafc0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:45.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc0106f90 0x7fafc019ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:45.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019c0a0 con 0x7fafc0101940 2023-10-16T14:32:45.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019c210 con 0x7fafc0100f80 2023-10-16T14:32:45.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc742d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc019c3d0 con 0x7fafc0106f90 2023-10-16T14:32:45.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 0x7fafc01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:45.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc49c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafc0101940 0x7fafc0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:45.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 0x7fafc01970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:49688/0 (socket says 172.21.15.178:49688) 2023-10-16T14:32:45.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 -- 172.21.15.178:0/937560309 learned_addr learned my addr 172.21.15.178:0/937560309 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:45.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc59ca700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc0106f90 0x7fafc019ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:45.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 -- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc0106f90 msgr2=0x7fafc019ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc0106f90 0x7fafc019ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 -- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafc0101940 msgr2=0x7fafc0197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafc0101940 0x7fafc0197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.681+0000 7fafc51c9700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009c40 con 0x7fafc0100f80 2023-10-16T14:32:45.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafc51c9700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 0x7fafc01970e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00e9c0 tx=0x7fafbc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:45.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc004a30 con 0x7fafc0100f80 2023-10-16T14:32:45.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafbc00cc80 con 0x7fafc0100f80 2023-10-16T14:32:45.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc019c6b0 con 0x7fafc0100f80 2023-10-16T14:32:45.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0054e0 con 0x7fafc0100f80 2023-10-16T14:32:45.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.682+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc01a3f10 con 0x7fafc0100f80 2023-10-16T14:32:45.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.683+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fafc0111190 con 0x7fafc0100f80 2023-10-16T14:32:45.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.685+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fafbc004b90 con 0x7fafc0100f80 2023-10-16T14:32:45.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.686+0000 7fafb67fc700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fafac0a1510 0x7fafac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:45.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fafbc014070 con 0x7fafc0100f80 2023-10-16T14:32:45.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafc49c8700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fafac0a1510 0x7fafac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:45.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafb67fc700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] conn(0x7fafac0ac530 0x7fafac0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:45.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fafac0af0a0 con 0x7fafac0ac530 2023-10-16T14:32:45.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafbc0cd8d0 con 0x7fafc0100f80 2023-10-16T14:32:45.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafc49c8700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fafac0a1510 0x7fafac0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fafc0198700 tx=0x7fafb0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:45.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.687+0000 7fafc59ca700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] conn(0x7fafac0ac530 0x7fafac0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:45.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.688+0000 7fafc59ca700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] conn(0x7fafac0ac530 0x7fafac0ae9d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:45.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.689+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== osd.8 v2:172.21.15.187:6816/1521942991 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fafac0af0a0 con 0x7fafac0ac530 2023-10-16T14:32:45.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.706+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 --> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fafc004ebb0 con 0x7fafac0ac530 2023-10-16T14:32:45.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafb67fc700 1 -- 172.21.15.178:0/937560309 <== osd.8 v2:172.21.15.187:6816/1521942991 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fafc004ebb0 con 0x7fafac0ac530 2023-10-16T14:32:45.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] conn(0x7fafac0ac530 msgr2=0x7fafac0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] conn(0x7fafac0ac530 0x7fafac0ae9d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fafac0a1510 msgr2=0x7fafac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fafac0a1510 0x7fafac0a39d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fafc0198700 tx=0x7fafb0009260 comp rx=0 tx=0).stop 2023-10-16T14:32:45.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 msgr2=0x7fafc01970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:45.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 --2- 172.21.15.178:0/937560309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafc0100f80 0x7fafc01970e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00e9c0 tx=0x7fafbc00ecd0 comp rx=0 tx=0).stop 2023-10-16T14:32:45.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc59ca700 1 -- 172.21.15.178:0/937560309 reap_dead start 2023-10-16T14:32:45.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 shutdown_connections 2023-10-16T14:32:45.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.707+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 >> 172.21.15.178:0/937560309 conn(0x7fafc00780a0 msgr2=0x7fafc00ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:45.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.708+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 shutdown_connections 2023-10-16T14:32:45.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.708+0000 7fafc742d700 1 -- 172.21.15.178:0/937560309 wait complete. 2023-10-16T14:32:45.944 INFO:teuthology.orchestra.run.smithi178.stdout:77309411395 2023-10-16T14:32:45.945 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.2 2023-10-16T14:32:46.017 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:45 smithi178 ceph-mon[98495]: pgmap v308: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-16T14:32:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:45 smithi186 ceph-mon[134892]: pgmap v308: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-16T14:32:46.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 -- 172.21.15.178:0/4225614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c074ef0 msgr2=0x7efc7c0730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 --2- 172.21.15.178:0/4225614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c074ef0 0x7efc7c0730e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009990 tx=0x7efc6c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 -- 172.21.15.178:0/4225614029 shutdown_connections 2023-10-16T14:32:46.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 --2- 172.21.15.178:0/4225614029 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7c111bc0 0x7efc7c113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 --2- 172.21.15.178:0/4225614029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c074ef0 0x7efc7c0730e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 --2- 172.21.15.178:0/4225614029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 0x7efc7c073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 -- 172.21.15.178:0/4225614029 >> 172.21.15.178:0/4225614029 conn(0x7efc7c0780a0 msgr2=0x7efc7c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 -- 172.21.15.178:0/4225614029 shutdown_connections 2023-10-16T14:32:46.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.790+0000 7efc822af700 1 -- 172.21.15.178:0/4225614029 wait complete. 2023-10-16T14:32:46.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.791+0000 7efc822af700 1 Processor -- start 2023-10-16T14:32:46.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 -- start start 2023-10-16T14:32:46.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 0x7efc7c19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7c074ef0 0x7efc7c19dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 0x7efc7c1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c1a1580 con 0x7efc7c111bc0 2023-10-16T14:32:46.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c1a16f0 con 0x7efc7c074ef0 2023-10-16T14:32:46.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc822af700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7c1a18b0 con 0x7efc7c073620 2023-10-16T14:32:46.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc8084c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 0x7efc7c1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc8084c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 0x7efc7c1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60298/0 (socket says 172.21.15.178:60298) 2023-10-16T14:32:46.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc8084c700 1 -- 172.21.15.178:0/4583172 learned_addr learned my addr 172.21.15.178:0/4583172 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:46.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc7bfff700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 0x7efc7c19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.792+0000 7efc7b7fe700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7c074ef0 0x7efc7c19dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 -- 172.21.15.178:0/4583172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7c074ef0 msgr2=0x7efc7c19dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc7c074ef0 0x7efc7c19dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 -- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 msgr2=0x7efc7c1a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 0x7efc7c1a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009c40 con 0x7efc7c073620 2023-10-16T14:32:46.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc8084c700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc7c111bc0 0x7efc7c1a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:46.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc7bfff700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 0x7efc7c19d680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000c00 tx=0x7efc6c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c01d070 con 0x7efc7c073620 2023-10-16T14:32:46.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc6c021410 con 0x7efc7c073620 2023-10-16T14:32:46.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc6c0095f0 con 0x7efc7c073620 2023-10-16T14:32:46.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c017650 con 0x7efc7c073620 2023-10-16T14:32:46.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.793+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7c1aa310 con 0x7efc7c073620 2023-10-16T14:32:46.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.794+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc7c1177f0 con 0x7efc7c073620 2023-10-16T14:32:46.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.796+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7efc6c0217f0 con 0x7efc7c073620 2023-10-16T14:32:46.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.797+0000 7efc797fa700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efc640a1510 0x7efc640a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efc6c0ce490 con 0x7efc7c073620 2023-10-16T14:32:46.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc7b7fe700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efc640a1510 0x7efc640a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc797fa700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] conn(0x7efc640ac530 0x7efc640ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc640af0a0 con 0x7efc640ac530 2023-10-16T14:32:46.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc6c0d0a70 con 0x7efc7c073620 2023-10-16T14:32:46.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc8084c700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] conn(0x7efc640ac530 0x7efc640ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.798+0000 7efc7b7fe700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efc640a1510 0x7efc640a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc7c19eca0 tx=0x7efc70009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.799+0000 7efc8084c700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] conn(0x7efc640ac530 0x7efc640ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.800+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== osd.3 v2:172.21.15.178:6810/4190845983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7efc640af0a0 con 0x7efc640ac530 2023-10-16T14:32:46.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.824+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 --> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc7c04ebb0 con 0x7efc640ac530 2023-10-16T14:32:46.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc797fa700 1 -- 172.21.15.178:0/4583172 <== osd.3 v2:172.21.15.178:6810/4190845983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efc7c04ebb0 con 0x7efc640ac530 2023-10-16T14:32:46.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] conn(0x7efc640ac530 msgr2=0x7efc640ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] conn(0x7efc640ac530 0x7efc640ae9d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efc640a1510 msgr2=0x7efc640a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efc640a1510 0x7efc640a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc7c19eca0 tx=0x7efc70009260 comp rx=0 tx=0).stop 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 msgr2=0x7efc7c19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.825+0000 7efc822af700 1 --2- 172.21.15.178:0/4583172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc7c073620 0x7efc7c19d680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efc6c000c00 tx=0x7efc6c005740 comp rx=0 tx=0).stop 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.826+0000 7efc8084c700 1 -- 172.21.15.178:0/4583172 reap_dead start 2023-10-16T14:32:46.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.826+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 shutdown_connections 2023-10-16T14:32:46.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.826+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 >> 172.21.15.178:0/4583172 conn(0x7efc7c0780a0 msgr2=0x7efc7c110640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.826+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 shutdown_connections 2023-10-16T14:32:46.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:45.826+0000 7efc822af700 1 -- 172.21.15.178:0/4583172 wait complete. 2023-10-16T14:32:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:45 smithi187 ceph-mon[135297]: pgmap v308: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-10-16T14:32:46.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 -- 172.21.15.178:0/3548444612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184101940 msgr2=0x7fc18410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 --2- 172.21.15.178:0/3548444612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184101940 0x7fc18410ddb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b0d0 tx=0x7fc17c00b3e0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 -- 172.21.15.178:0/3548444612 shutdown_connections 2023-10-16T14:32:46.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 --2- 172.21.15.178:0/3548444612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184101940 0x7fc18410ddb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 --2- 172.21.15.178:0/3548444612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc184106f90 0x7fc184107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 --2- 172.21.15.178:0/3548444612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 -- 172.21.15.178:0/3548444612 >> 172.21.15.178:0/3548444612 conn(0x7fc1840780a0 msgr2=0x7fc1840784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 -- 172.21.15.178:0/3548444612 shutdown_connections 2023-10-16T14:32:46.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.516+0000 7fc18a247700 1 -- 172.21.15.178:0/3548444612 wait complete. 2023-10-16T14:32:46.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.517+0000 7fc18a247700 1 Processor -- start 2023-10-16T14:32:46.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 -- start start 2023-10-16T14:32:46.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc184101940 0x7fc1841997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184106f90 0x7fc18419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e250 con 0x7fc184100f80 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e3c0 con 0x7fc184106f90 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc18a247700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18419e580 con 0x7fc184101940 2023-10-16T14:32:46.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc1837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc1837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60320/0 (socket says 172.21.15.178:60320) 2023-10-16T14:32:46.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.518+0000 7fc1837fe700 1 -- 172.21.15.178:0/112211212 learned_addr learned my addr 172.21.15.178:0/112211212 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:46.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc183fff700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184106f90 0x7fc18419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc182ffd700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc184101940 0x7fc1841997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 -- 172.21.15.178:0/112211212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc184101940 msgr2=0x7fc1841997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc184101940 0x7fc1841997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 -- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184106f90 msgr2=0x7fc18419dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184106f90 0x7fc18419dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009d00 con 0x7fc184100f80 2023-10-16T14:32:46.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc183fff700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc184106f90 0x7fc18419dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:46.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.519+0000 7fc1837fe700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184199290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc17400e9c0 tx=0x7fc17400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.520+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740097b0 con 0x7fc184100f80 2023-10-16T14:32:46.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.520+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc17400ee60 con 0x7fc184100f80 2023-10-16T14:32:46.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.520+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1740054f0 con 0x7fc184100f80 2023-10-16T14:32:46.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.520+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18419e860 con 0x7fc184100f80 2023-10-16T14:32:46.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.520+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1841a60c0 con 0x7fc184100f80 2023-10-16T14:32:46.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.521+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc184111180 con 0x7fc184100f80 2023-10-16T14:32:46.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.523+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fc174004750 con 0x7fc184100f80 2023-10-16T14:32:46.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.525+0000 7fc180ff9700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc16c0a1590 0x7fc16c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc174014070 con 0x7fc184100f80 2023-10-16T14:32:46.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc182ffd700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc16c0a1590 0x7fc16c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc180ff9700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] conn(0x7fc16c0ac5b0 0x7fc16c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc16c0af120 con 0x7fc16c0ac5b0 2023-10-16T14:32:46.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1740cf830 con 0x7fc184100f80 2023-10-16T14:32:46.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc182ffd700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc16c0a1590 0x7fc16c0a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc18419a8b0 tx=0x7fc178009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.526+0000 7fc183fff700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] conn(0x7fc16c0ac5b0 0x7fc16c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.527+0000 7fc183fff700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] conn(0x7fc16c0ac5b0 0x7fc16c0aea50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.528+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== osd.1 v2:172.21.15.186:6800/4108518172 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7fc16c0af120 con 0x7fc16c0ac5b0 2023-10-16T14:32:46.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.550+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 --> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc18404ebb0 con 0x7fc16c0ac5b0 2023-10-16T14:32:46.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.550+0000 7fc180ff9700 1 -- 172.21.15.178:0/112211212 <== osd.1 v2:172.21.15.186:6800/4108518172 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc18404ebb0 con 0x7fc16c0ac5b0 2023-10-16T14:32:46.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] conn(0x7fc16c0ac5b0 msgr2=0x7fc16c0aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] conn(0x7fc16c0ac5b0 0x7fc16c0aea50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc16c0a1590 msgr2=0x7fc16c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc16c0a1590 0x7fc16c0a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc18419a8b0 tx=0x7fc178009260 comp rx=0 tx=0).stop 2023-10-16T14:32:46.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 msgr2=0x7fc184199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 --2- 172.21.15.178:0/112211212 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc184100f80 0x7fc184199290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc17400e9c0 tx=0x7fc17400ecd0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc183fff700 1 -- 172.21.15.178:0/112211212 reap_dead start 2023-10-16T14:32:46.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 shutdown_connections 2023-10-16T14:32:46.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 >> 172.21.15.178:0/112211212 conn(0x7fc1840780a0 msgr2=0x7fc18410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 shutdown_connections 2023-10-16T14:32:46.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.551+0000 7fc18a247700 1 -- 172.21.15.178:0/112211212 wait complete. 2023-10-16T14:32:46.592 INFO:teuthology.orchestra.run.smithi178.stdout:249108103209 2023-10-16T14:32:46.593 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.8 2023-10-16T14:32:46.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 -- 172.21.15.178:0/3865375799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac073620 msgr2=0x7f75ac073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 --2- 172.21.15.178:0/3865375799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac073620 0x7f75ac073a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7598009990 tx=0x7f7598009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 -- 172.21.15.178:0/3865375799 shutdown_connections 2023-10-16T14:32:46.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 --2- 172.21.15.178:0/3865375799 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac111bc0 0x7f75ac113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 --2- 172.21.15.178:0/3865375799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac074ef0 0x7f75ac0730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 --2- 172.21.15.178:0/3865375799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac073620 0x7f75ac073a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.846+0000 7f75b2f58700 1 -- 172.21.15.178:0/3865375799 >> 172.21.15.178:0/3865375799 conn(0x7f75ac0780a0 msgr2=0x7f75ac0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.847+0000 7f75b2f58700 1 -- 172.21.15.178:0/3865375799 shutdown_connections 2023-10-16T14:32:46.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.847+0000 7f75b2f58700 1 -- 172.21.15.178:0/3865375799 wait complete. 2023-10-16T14:32:46.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 Processor -- start 2023-10-16T14:32:46.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 -- start start 2023-10-16T14:32:46.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac073620 0x7f75ac19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac074ef0 0x7f75ac19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 0x7f75ac1a2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac1a2630 con 0x7f75ac111bc0 2023-10-16T14:32:46.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac1a27a0 con 0x7f75ac073620 2023-10-16T14:32:46.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b2f58700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac1a2960 con 0x7f75ac074ef0 2023-10-16T14:32:46.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.848+0000 7f75b14f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 0x7f75ac1a2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b0cf4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac073620 0x7f75ac19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 0x7f75ac1a2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60346/0 (socket says 172.21.15.178:60346) 2023-10-16T14:32:46.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 -- 172.21.15.178:0/1544130378 learned_addr learned my addr 172.21.15.178:0/1544130378 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:46.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75abfff700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac074ef0 0x7f75ac19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 -- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac074ef0 msgr2=0x7f75ac19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75ac074ef0 0x7f75ac19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 -- 172.21.15.178:0/1544130378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac073620 msgr2=0x7f75ac19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac073620 0x7f75ac19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980095f0 con 0x7f75ac111bc0 2023-10-16T14:32:46.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b0cf4700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f75ac073620 0x7f75ac19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:46.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.849+0000 7f75b14f5700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 0x7f75ac1a2020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75a400ef90 tx=0x7f75a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.850+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4018070 con 0x7f75ac111bc0 2023-10-16T14:32:46.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.850+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75a400f040 con 0x7f75ac111bc0 2023-10-16T14:32:46.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.850+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400ccc0 con 0x7f75ac111bc0 2023-10-16T14:32:46.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.850+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac1a2be0 con 0x7f75ac111bc0 2023-10-16T14:32:46.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.850+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac1aa4a0 con 0x7f75ac111bc0 2023-10-16T14:32:46.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.851+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f75ac04ebb0 con 0x7f75ac111bc0 2023-10-16T14:32:46.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.853+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f75a4003a00 con 0x7f75ac111bc0 2023-10-16T14:32:46.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.855+0000 7f75a9ffb700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f759c0a1590 0x7f759c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f75a40cd4e0 con 0x7f75ac111bc0 2023-10-16T14:32:46.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75b0cf4700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f759c0a1590 0x7f759c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75a9ffb700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] conn(0x7f759c0ac5b0 0x7f759c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:46.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f759c0af120 con 0x7f759c0ac5b0 2023-10-16T14:32:46.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a40cd850 con 0x7f75ac111bc0 2023-10-16T14:32:46.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.856+0000 7f75b0cf4700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f759c0a1590 0x7f759c0a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75a0011490 tx=0x7f75a0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.857+0000 7f75abfff700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] conn(0x7f759c0ac5b0 0x7f759c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:46.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.857+0000 7f75abfff700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] conn(0x7f759c0ac5b0 0x7f759c0aea50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:46.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.858+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== osd.5 v2:172.21.15.187:6808/3921910211 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25755 (crc 0 0 0) 0x7f759c0af120 con 0x7f759c0ac5b0 2023-10-16T14:32:46.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.876+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 --> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75ac1177f0 con 0x7f759c0ac5b0 2023-10-16T14:32:46.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75a9ffb700 1 -- 172.21.15.178:0/1544130378 <== osd.5 v2:172.21.15.187:6808/3921910211 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f759801e040 con 0x7f759c0ac5b0 2023-10-16T14:32:46.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] conn(0x7f759c0ac5b0 msgr2=0x7f759c0aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] conn(0x7f759c0ac5b0 0x7f759c0aea50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f759c0a1590 msgr2=0x7f759c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f759c0a1590 0x7f759c0a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75a0011490 tx=0x7f75a0009260 comp rx=0 tx=0).stop 2023-10-16T14:32:46.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 msgr2=0x7f75ac1a2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:46.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 --2- 172.21.15.178:0/1544130378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75ac111bc0 0x7f75ac1a2020 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f75a400ef90 tx=0x7f75a400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:46.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b14f5700 1 -- 172.21.15.178:0/1544130378 reap_dead start 2023-10-16T14:32:46.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 shutdown_connections 2023-10-16T14:32:46.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.877+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 >> 172.21.15.178:0/1544130378 conn(0x7f75ac0780a0 msgr2=0x7f75ac1106c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:46.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.878+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 shutdown_connections 2023-10-16T14:32:46.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:46.878+0000 7f75b2f58700 1 -- 172.21.15.178:0/1544130378 wait complete. 2023-10-16T14:32:47.389 INFO:teuthology.orchestra.run.smithi178.stdout:111669149759 2023-10-16T14:32:47.389 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.3 2023-10-16T14:32:47.796 INFO:teuthology.orchestra.run.smithi178.stdout:55834574918 2023-10-16T14:32:47.796 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.1 2023-10-16T14:32:48.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:47 smithi178 ceph-mon[98495]: pgmap v309: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:47 smithi186 ceph-mon[134892]: pgmap v309: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:48.132 INFO:teuthology.orchestra.run.smithi178.stdout:163208757302 2023-10-16T14:32:48.132 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph osd last-stat-seq osd.5 2023-10-16T14:32:48.134 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:48.142 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:48.167 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:47 smithi187 ceph-mon[135297]: pgmap v309: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:48.369 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:48.370 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:48.592 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:49.344 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:49.345 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:49.874 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:49 smithi178 ceph-mon[98495]: pgmap v310: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:49 smithi186 ceph-mon[134892]: pgmap v310: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:50.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:49 smithi187 ceph-mon[135297]: pgmap v310: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:32:51.044 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:51.466 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:51 smithi178 ceph-mon[98495]: pgmap v311: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-10-16T14:32:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:51 smithi186 ceph-mon[134892]: pgmap v311: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-10-16T14:32:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:51 smithi187 ceph-mon[135297]: pgmap v311: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2023-10-16T14:32:52.872 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:53.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.049+0000 7ffa94881700 1 -- 172.21.15.178:0/372624234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 msgr2=0x7ffa90101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:53.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.049+0000 7ffa94881700 1 --2- 172.21.15.178:0/372624234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa90101400 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa840099c0 tx=0x7ffa84009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 -- 172.21.15.178:0/372624234 shutdown_connections 2023-10-16T14:32:53.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 --2- 172.21.15.178:0/372624234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90101940 0x7ffa9010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 --2- 172.21.15.178:0/372624234 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90106f90 0x7ffa90107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 --2- 172.21.15.178:0/372624234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa90101400 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 -- 172.21.15.178:0/372624234 >> 172.21.15.178:0/372624234 conn(0x7ffa900780a0 msgr2=0x7ffa900784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 -- 172.21.15.178:0/372624234 shutdown_connections 2023-10-16T14:32:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.050+0000 7ffa94881700 1 -- 172.21.15.178:0/372624234 wait complete. 2023-10-16T14:32:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 Processor -- start 2023-10-16T14:32:53.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 -- start start 2023-10-16T14:32:53.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa900731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:53.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90101940 0x7ffa900736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:53.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:53.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa901a54d0 con 0x7ffa90100f80 2023-10-16T14:32:53.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa901a5640 con 0x7ffa90101940 2023-10-16T14:32:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.051+0000 7ffa94881700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa901a5800 con 0x7ffa90106f90 2023-10-16T14:32:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa900731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57556/0 (socket says 172.21.15.178:57556) 2023-10-16T14:32:53.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 -- 172.21.15.178:0/3936754652 learned_addr learned my addr 172.21.15.178:0/3936754652 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:53.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8dd9b700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90101940 0x7ffa900736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:53.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 -- 172.21.15.178:0/3936754652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90101940 msgr2=0x7ffa900736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:53.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90101940 0x7ffa900736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 -- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 msgr2=0x7ffa900731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:53.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa900731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 -- 172.21.15.178:0/3936754652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840095f0 con 0x7ffa90106f90 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8e59c700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa900731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.052+0000 7ffa8ed9d700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800efc0 tx=0x7ffa8800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.053+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa88018070 con 0x7ffa90106f90 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.053+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa8800f040 con 0x7ffa90106f90 2023-10-16T14:32:53.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.053+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa901a5a80 con 0x7ffa90106f90 2023-10-16T14:32:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.053+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8800cc10 con 0x7ffa90106f90 2023-10-16T14:32:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.053+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa901a5f50 con 0x7ffa90106f90 2023-10-16T14:32:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.054+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa9004ebb0 con 0x7ffa90106f90 2023-10-16T14:32:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.056+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7ffa8800cd90 con 0x7ffa90106f90 2023-10-16T14:32:53.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.057+0000 7ffa7f7fe700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 0x7ffa780a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:53.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.057+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa880ccb80 con 0x7ffa90106f90 2023-10-16T14:32:53.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.057+0000 7ffa8e59c700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 0x7ffa780a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:53.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.058+0000 7ffa8e59c700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 0x7ffa780a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa80005950 tx=0x7ffa8000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:53.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.061+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffa88095da0 con 0x7ffa90106f90 2023-10-16T14:32:53.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.283+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7ffa90111110 con 0x7ffa90106f90 2023-10-16T14:32:53.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.284+0000 7ffa7f7fe700 1 -- 172.21.15.178:0/3936754652 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa880993c0 con 0x7ffa90106f90 2023-10-16T14:32:53.287 INFO:teuthology.orchestra.run.smithi178.stdout:283467841573 2023-10-16T14:32:53.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 msgr2=0x7ffa780a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 0x7ffa780a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa80005950 tx=0x7ffa8000b410 comp rx=0 tx=0).stop 2023-10-16T14:32:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 msgr2=0x7ffa90073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800efc0 tx=0x7ffa8800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 shutdown_connections 2023-10-16T14:32:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffa780a1510 0x7ffa780a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa90106f90 0x7ffa90073c70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa90100f80 0x7ffa900731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 --2- 172.21.15.178:0/3936754652 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffa90101940 0x7ffa900736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.286+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 >> 172.21.15.178:0/3936754652 conn(0x7ffa900780a0 msgr2=0x7ffa900ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.287+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 shutdown_connections 2023-10-16T14:32:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:53.287+0000 7ffa94881700 1 -- 172.21.15.178:0/3936754652 wait complete. 2023-10-16T14:32:53.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:53 smithi187 ceph-mon[135297]: pgmap v312: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-16T14:32:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:53 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3936754652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-16T14:32:53.949 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:53 smithi178 ceph-mon[98495]: pgmap v312: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-16T14:32:53.949 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:53 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3936754652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-16T14:32:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:53 smithi186 ceph-mon[134892]: pgmap v312: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2023-10-16T14:32:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:53 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3936754652' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-10-16T14:32:54.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 -- 172.21.15.178:0/1887989949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944106f90 msgr2=0x7fc944107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:54.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 --2- 172.21.15.178:0/1887989949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944106f90 0x7fc944107370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc9400099c0 tx=0x7fc940009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 -- 172.21.15.178:0/1887989949 shutdown_connections 2023-10-16T14:32:54.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 --2- 172.21.15.178:0/1887989949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944101940 0x7fc94410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 --2- 172.21.15.178:0/1887989949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944106f90 0x7fc944107370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 --2- 172.21.15.178:0/1887989949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944100f80 0x7fc944101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 -- 172.21.15.178:0/1887989949 >> 172.21.15.178:0/1887989949 conn(0x7fc9440780a0 msgr2=0x7fc9440784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:54.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 -- 172.21.15.178:0/1887989949 shutdown_connections 2023-10-16T14:32:54.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.206+0000 7fc94b812700 1 -- 172.21.15.178:0/1887989949 wait complete. 2023-10-16T14:32:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 Processor -- start 2023-10-16T14:32:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 -- start start 2023-10-16T14:32:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:54.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944101940 0x7fc9441996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 0x7fc94419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94419e150 con 0x7fc944100f80 2023-10-16T14:32:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94419e2c0 con 0x7fc944106f90 2023-10-16T14:32:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.207+0000 7fc94b812700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94419e480 con 0x7fc944101940 2023-10-16T14:32:54.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc948dad700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944101940 0x7fc9441996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56898/0 (socket says 172.21.15.178:56898) 2023-10-16T14:32:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 -- 172.21.15.178:0/2585395741 learned_addr learned my addr 172.21.15.178:0/2585395741 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc949daf700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 0x7fc94419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:54.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 -- 172.21.15.178:0/2585395741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944101940 msgr2=0x7fc9441996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944101940 0x7fc9441996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 -- 172.21.15.178:0/2585395741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 msgr2=0x7fc94419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 0x7fc94419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 -- 172.21.15.178:0/2585395741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9400095f0 con 0x7fc944100f80 2023-10-16T14:32:54.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc949daf700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 0x7fc94419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:32:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc9495ae700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b580 tx=0x7fc94000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.208+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94001d070 con 0x7fc944100f80 2023-10-16T14:32:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.209+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94419e700 con 0x7fc944100f80 2023-10-16T14:32:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.209+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc940005050 con 0x7fc944100f80 2023-10-16T14:32:54.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.209+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9441a5fc0 con 0x7fc944100f80 2023-10-16T14:32:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.209+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940004a60 con 0x7fc944100f80 2023-10-16T14:32:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.210+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc94404ebb0 con 0x7fc944100f80 2023-10-16T14:32:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.211+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fc9400051c0 con 0x7fc944100f80 2023-10-16T14:32:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.213+0000 7fc93a7fc700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:54.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.213+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9400cdf10 con 0x7fc944100f80 2023-10-16T14:32:54.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.213+0000 7fc948dad700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:54.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.214+0000 7fc948dad700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 0x7fc9300a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc94419a7b0 tx=0x7fc934009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:54.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.220+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc94005c100 con 0x7fc944100f80 2023-10-16T14:32:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.465+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc944110f40 con 0x7fc944100f80 2023-10-16T14:32:54.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.465+0000 7fc93a7fc700 1 -- 172.21.15.178:0/2585395741 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc940097130 con 0x7fc944100f80 2023-10-16T14:32:54.469 INFO:teuthology.orchestra.run.smithi178.stdout:141733920828 2023-10-16T14:32:54.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.468+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 msgr2=0x7fc9300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.468+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 0x7fc9300a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc94419a7b0 tx=0x7fc934009260 comp rx=0 tx=0).stop 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.468+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 msgr2=0x7fc944199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.468+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b580 tx=0x7fc94000bfd0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 shutdown_connections 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9300a1510 0x7fc9300a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc944106f90 0x7fc94419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc944100f80 0x7fc944199190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 --2- 172.21.15.178:0/2585395741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc944101940 0x7fc9441996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 >> 172.21.15.178:0/2585395741 conn(0x7fc9440780a0 msgr2=0x7fc94410c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 shutdown_connections 2023-10-16T14:32:54.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:54.469+0000 7fc94b812700 1 -- 172.21.15.178:0/2585395741 wait complete. 2023-10-16T14:32:54.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:54 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2585395741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-16T14:32:54.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:54 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2585395741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-16T14:32:55.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:54 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2585395741' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-10-16T14:32:55.601 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920826 got 141733920828 for osd.4 2023-10-16T14:32:55.602 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:55.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 -- 172.21.15.178:0/1460748608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40689a0 msgr2=0x7ff2a4068e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:55.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1460748608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40689a0 0x7ff2a4068e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff290009990 tx=0x7ff290009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 -- 172.21.15.178:0/1460748608 shutdown_connections 2023-10-16T14:32:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1460748608 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1460748608 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a4106f90 0x7ff2a4107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1460748608 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a40689a0 0x7ff2a4068e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 -- 172.21.15.178:0/1460748608 >> 172.21.15.178:0/1460748608 conn(0x7ff2a4075e00 msgr2=0x7ff2a4076210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:55.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 -- 172.21.15.178:0/1460748608 shutdown_connections 2023-10-16T14:32:55.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.641+0000 7ff2aae79700 1 -- 172.21.15.178:0/1460748608 wait complete. 2023-10-16T14:32:55.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.642+0000 7ff2aae79700 1 Processor -- start 2023-10-16T14:32:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 -- start start 2023-10-16T14:32:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a40689a0 0x7ff2a4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4106f90 0x7ff2a419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419e100 con 0x7ff2a4069360 2023-10-16T14:32:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419e270 con 0x7ff2a4106f90 2023-10-16T14:32:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2aae79700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a419e430 con 0x7ff2a40689a0 2023-10-16T14:32:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2a3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56908/0 (socket says 172.21.15.178:56908) 2023-10-16T14:32:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2a3fff700 1 -- 172.21.15.178:0/1655531873 learned_addr learned my addr 172.21.15.178:0/1655531873 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2a9416700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4106f90 0x7ff2a419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.643+0000 7ff2a8c15700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a40689a0 0x7ff2a4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 -- 172.21.15.178:0/1655531873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a40689a0 msgr2=0x7ff2a4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a40689a0 0x7ff2a4199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 -- 172.21.15.178:0/1655531873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4106f90 msgr2=0x7ff2a419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4106f90 0x7ff2a419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 -- 172.21.15.178:0/1655531873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009c40 con 0x7ff2a4069360 2023-10-16T14:32:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a3fff700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff290000c00 tx=0x7ff290004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29001d070 con 0x7ff2a4069360 2023-10-16T14:32:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff290021410 con 0x7ff2a4069360 2023-10-16T14:32:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff298009520 con 0x7ff2a4069360 2023-10-16T14:32:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.644+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290017650 con 0x7ff2a4069360 2023-10-16T14:32:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.645+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2900095f0 con 0x7ff2a4069360 2023-10-16T14:32:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.646+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a404ebb0 con 0x7ff2a4069360 2023-10-16T14:32:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.648+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7ff290021580 con 0x7ff2a4069360 2023-10-16T14:32:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.650+0000 7ff2a1ffb700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 0x7ff2940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.650+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2900ce1e0 con 0x7ff2a4069360 2023-10-16T14:32:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.651+0000 7ff2a8c15700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 0x7ff2940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:55.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.652+0000 7ff2a8c15700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 0x7ff2940a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff298009df0 tx=0x7ff298009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:55.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.655+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff290097510 con 0x7ff2a4069360 2023-10-16T14:32:55.895 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:55 smithi178 ceph-mon[98495]: pgmap v313: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:32:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.900+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff2a4110e20 con 0x7ff2a4069360 2023-10-16T14:32:55.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.901+0000 7ff2a1ffb700 1 -- 172.21.15.178:0/1655531873 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff29009ab30 con 0x7ff2a4069360 2023-10-16T14:32:55.904 INFO:teuthology.orchestra.run.smithi178.stdout:77309411398 2023-10-16T14:32:55.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.903+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 msgr2=0x7ff2940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.903+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 0x7ff2940a3a50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff298009df0 tx=0x7ff298009260 comp rx=0 tx=0).stop 2023-10-16T14:32:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.903+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 msgr2=0x7ff2a41996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.903+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff290000c00 tx=0x7ff290004790 comp rx=0 tx=0).stop 2023-10-16T14:32:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 shutdown_connections 2023-10-16T14:32:55.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2940a1590 0x7ff2940a3a50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff2a4106f90 0x7ff2a419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2a40689a0 0x7ff2a4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 --2- 172.21.15.178:0/1655531873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2a4069360 0x7ff2a41996d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:55.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 >> 172.21.15.178:0/1655531873 conn(0x7ff2a4075e00 msgr2=0x7ff2a40fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 shutdown_connections 2023-10-16T14:32:55.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:55.904+0000 7ff2aae79700 1 -- 172.21.15.178:0/1655531873 wait complete. 2023-10-16T14:32:55.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:55 smithi187 ceph-mon[135297]: pgmap v313: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:32:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:55 smithi186 ceph-mon[134892]: pgmap v313: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:32:56.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 -- 172.21.15.178:0/1859073175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 msgr2=0x7f30140757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 --2- 172.21.15.178:0/1859073175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30140757c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3010009990 tx=0x7f3010009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 -- 172.21.15.178:0/1859073175 shutdown_connections 2023-10-16T14:32:56.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 --2- 172.21.15.178:0/1859073175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 0x7f30141141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 --2- 172.21.15.178:0/1859073175 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30140757c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 --2- 172.21.15.178:0/1859073175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 0x7f30141118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 -- 172.21.15.178:0/1859073175 >> 172.21.15.178:0/1859073175 conn(0x7f30140fec10 msgr2=0x7f3014101030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:56.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 -- 172.21.15.178:0/1859073175 shutdown_connections 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.100+0000 7f301c0e2700 1 -- 172.21.15.178:0/1859073175 wait complete. 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.101+0000 7f301c0e2700 1 Processor -- start 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 -- start start 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 0x7f30141a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 0x7f301419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301419dcf0 con 0x7f3014111de0 2023-10-16T14:32:56.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301419de60 con 0x7f30140753e0 2023-10-16T14:32:56.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301c0e2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301419e020 con 0x7f3014075d00 2023-10-16T14:32:56.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f3019e7e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f3019e7e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37068/0 (socket says 172.21.15.178:37068) 2023-10-16T14:32:56.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f3019e7e700 1 -- 172.21.15.178:0/3868786033 learned_addr learned my addr 172.21.15.178:0/3868786033 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:56.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301a67f700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 0x7f301419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.102+0000 7f301967d700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 0x7f30141a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 -- 172.21.15.178:0/3868786033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 msgr2=0x7f30141a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 0x7f30141a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 -- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 msgr2=0x7f301419d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 0x7f301419d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 -- 172.21.15.178:0/3868786033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100095f0 con 0x7f30140753e0 2023-10-16T14:32:56.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f3019e7e700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3010003bc0 tx=0x7f3010003bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:56.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301001d070 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f301000f460 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301419e2a0 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.103+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010017650 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.104+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30141aa5f0 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.105+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301410d000 con 0x7f30140753e0 2023-10-16T14:32:56.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.107+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f301000f5d0 con 0x7f30140753e0 2023-10-16T14:32:56.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.108+0000 7f300affd700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 0x7f30000a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.108+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f30100ce2f0 con 0x7f30140753e0 2023-10-16T14:32:56.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.108+0000 7f301967d700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 0x7f30000a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.109+0000 7f301967d700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 0x7f30000a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3014078110 tx=0x7f3004009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:56.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.112+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3010097510 con 0x7f30140753e0 2023-10-16T14:32:56.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.350+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f301404ebb0 con 0x7f30140753e0 2023-10-16T14:32:56.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.350+0000 7f300affd700 1 -- 172.21.15.178:0/3868786033 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f301009ab30 con 0x7f30140753e0 2023-10-16T14:32:56.353 INFO:teuthology.orchestra.run.smithi178.stdout:34359738446 2023-10-16T14:32:56.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 msgr2=0x7f30000a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 0x7f30000a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3014078110 tx=0x7f3004009190 comp rx=0 tx=0).stop 2023-10-16T14:32:56.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 msgr2=0x7f30141a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3010003bc0 tx=0x7f3010003bf0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 shutdown_connections 2023-10-16T14:32:56.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f30000a1510 0x7f30000a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3014111de0 0x7f301419d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f30140753e0 0x7f30141a3600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 --2- 172.21.15.178:0/3868786033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3014075d00 0x7f30141a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 >> 172.21.15.178:0/3868786033 conn(0x7f30140fec10 msgr2=0x7f3014112a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:56.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.353+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 shutdown_connections 2023-10-16T14:32:56.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.354+0000 7f301c0e2700 1 -- 172.21.15.178:0/3868786033 wait complete. 2023-10-16T14:32:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3720854739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b14073a00 msgr2=0x7f6b1410d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3720854739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b14073a00 0x7f6b1410d3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009990 tx=0x7f6b00009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3720854739 shutdown_connections 2023-10-16T14:32:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3720854739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b1410d930 0x7f6b1410fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3720854739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b140730e0 0x7f6b140734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3720854739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b14073a00 0x7f6b1410d3f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3720854739 >> 172.21.15.178:0/3720854739 conn(0x7f6b140780a0 msgr2=0x7f6b140784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3720854739 shutdown_connections 2023-10-16T14:32:56.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.400+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3720854739 wait complete. 2023-10-16T14:32:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.401+0000 7f6b1ace6700 1 Processor -- start 2023-10-16T14:32:56.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 -- start start 2023-10-16T14:32:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b14073a00 0x7f6b141997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 0x7f6b1419dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419e260 con 0x7f6b140730e0 2023-10-16T14:32:56.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419e3d0 con 0x7f6b14073a00 2023-10-16T14:32:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b1ace6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1419e590 con 0x7f6b1410d930 2023-10-16T14:32:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b18a82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b18a82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56936/0 (socket says 172.21.15.178:56936) 2023-10-16T14:32:56.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b18a82700 1 -- 172.21.15.178:0/3966123228 learned_addr learned my addr 172.21.15.178:0/3966123228 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b13fff700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b14073a00 0x7f6b141997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.402+0000 7f6b19283700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 0x7f6b1419dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 -- 172.21.15.178:0/3966123228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 msgr2=0x7f6b1419dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 0x7f6b1419dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 -- 172.21.15.178:0/3966123228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b14073a00 msgr2=0x7f6b141997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b14073a00 0x7f6b141997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 -- 172.21.15.178:0/3966123228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08009520 con 0x7f6b140730e0 2023-10-16T14:32:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b19283700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 0x7f6b1419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b18a82700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800eb30 tx=0x7f6b0800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:56.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08009880 con 0x7f6b140730e0 2023-10-16T14:32:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.403+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b000095f0 con 0x7f6b140730e0 2023-10-16T14:32:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.404+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b080044e0 con 0x7f6b140730e0 2023-10-16T14:32:56.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.404+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b08003d80 con 0x7f6b140730e0 2023-10-16T14:32:56.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.404+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b141a5f00 con 0x7f6b140730e0 2023-10-16T14:32:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.405+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1404ebb0 con 0x7f6b140730e0 2023-10-16T14:32:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.407+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f6b08003f20 con 0x7f6b140730e0 2023-10-16T14:32:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.408+0000 7f6b11ffb700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 0x7f6b040a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:56.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.408+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6b08014070 con 0x7f6b140730e0 2023-10-16T14:32:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.408+0000 7f6b13fff700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 0x7f6b040a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.409+0000 7f6b13fff700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 0x7f6b040a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b2e0 tx=0x7f6b00005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:56.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.412+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6b08096880 con 0x7f6b140730e0 2023-10-16T14:32:56.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.634+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b141131a0 con 0x7f6b140730e0 2023-10-16T14:32:56.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.634+0000 7f6b11ffb700 1 -- 172.21.15.178:0/3966123228 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b08099ea0 con 0x7f6b140730e0 2023-10-16T14:32:56.637 INFO:teuthology.orchestra.run.smithi178.stdout:111669149761 2023-10-16T14:32:56.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 msgr2=0x7f6b040a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 0x7f6b040a39d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000b2e0 tx=0x7f6b00005960 comp rx=0 tx=0).stop 2023-10-16T14:32:56.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 msgr2=0x7f6b141992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:56.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800eb30 tx=0x7f6b0800ee40 comp rx=0 tx=0).stop 2023-10-16T14:32:56.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 shutdown_connections 2023-10-16T14:32:56.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6b040a1510 0x7f6b040a39d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1410d930 0x7f6b1419dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b140730e0 0x7f6b141992a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 --2- 172.21.15.178:0/3966123228 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6b14073a00 0x7f6b141997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:56.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 >> 172.21.15.178:0/3966123228 conn(0x7f6b140780a0 msgr2=0x7f6b140ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:56.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 shutdown_connections 2023-10-16T14:32:56.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:56.638+0000 7f6b1ace6700 1 -- 172.21.15.178:0/3966123228 wait complete. 2023-10-16T14:32:56.919 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1655531873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-16T14:32:56.919 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3868786033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-16T14:32:56.920 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:56 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3966123228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-16T14:32:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1655531873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-16T14:32:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3868786033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-16T14:32:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:56 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3966123228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-16T14:32:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1655531873' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-10-16T14:32:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3868786033' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-10-16T14:32:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:56 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3966123228' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-10-16T14:32:57.270 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738443 got 34359738446 for osd.0 2023-10-16T14:32:57.271 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:57.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 -- 172.21.15.178:0/2464285022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa80730e0 msgr2=0x7f2fa80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:57.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 --2- 172.21.15.178:0/2464285022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa80730e0 0x7f2fa80734c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009990 tx=0x7f2fa4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 -- 172.21.15.178:0/2464285022 shutdown_connections 2023-10-16T14:32:57.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 --2- 172.21.15.178:0/2464285022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa810d930 0x7f2fa810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 --2- 172.21.15.178:0/2464285022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa80730e0 0x7f2fa80734c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 --2- 172.21.15.178:0/2464285022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa810d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 -- 172.21.15.178:0/2464285022 >> 172.21.15.178:0/2464285022 conn(0x7f2fa80780a0 msgr2=0x7f2fa80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:57.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 -- 172.21.15.178:0/2464285022 shutdown_connections 2023-10-16T14:32:57.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.414+0000 7f2fafbae700 1 -- 172.21.15.178:0/2464285022 wait complete. 2023-10-16T14:32:57.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.415+0000 7f2fafbae700 1 Processor -- start 2023-10-16T14:32:57.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 -- start start 2023-10-16T14:32:57.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 0x7f2fa81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:57.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:57.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 0x7f2fa819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:57.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819e280 con 0x7f2fa810d930 2023-10-16T14:32:57.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819e3f0 con 0x7f2fa8073a00 2023-10-16T14:32:57.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fafbae700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa819e5b0 con 0x7f2fa80730e0 2023-10-16T14:32:57.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fad94a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 0x7f2fa81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:57.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fad149700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:57.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fae14b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 0x7f2fa819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:57.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fad94a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 0x7f2fa81992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57618/0 (socket says 172.21.15.178:57618) 2023-10-16T14:32:57.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fad94a700 1 -- 172.21.15.178:0/3639178871 learned_addr learned my addr 172.21.15.178:0/3639178871 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:57.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.416+0000 7f2fad149700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37096/0 (socket says 172.21.15.178:37096) 2023-10-16T14:32:57.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 -- 172.21.15.178:0/3639178871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 msgr2=0x7f2fa81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:57.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 0x7f2fa81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 -- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 msgr2=0x7f2fa819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:57.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 0x7f2fa819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 -- 172.21.15.178:0/3639178871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40095f0 con 0x7f2fa8073a00 2023-10-16T14:32:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fae14b700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 0x7f2fa819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fad149700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800e9f0 tx=0x7f2f9800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9800c9c0 con 0x7f2fa8073a00 2023-10-16T14:32:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa819e890 con 0x7f2fa8073a00 2023-10-16T14:32:57.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f98010430 con 0x7f2fa8073a00 2023-10-16T14:32:57.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f98018680 con 0x7f2fa8073a00 2023-10-16T14:32:57.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.417+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa81a60f0 con 0x7f2fa8073a00 2023-10-16T14:32:57.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.418+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa804ebb0 con 0x7f2fa8073a00 2023-10-16T14:32:57.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.421+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f2f98010860 con 0x7f2fa8073a00 2023-10-16T14:32:57.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.422+0000 7f2f9effd700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 0x7f2f940a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:57.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.422+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2f98014070 con 0x7f2fa8073a00 2023-10-16T14:32:57.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.422+0000 7f2fad94a700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 0x7f2f940a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:57.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.423+0000 7f2fad94a700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 0x7f2f940a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006010 tx=0x7f2fa400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:57.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.424+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2f98096670 con 0x7f2fa8073a00 2023-10-16T14:32:57.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.636+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2fa81130e0 con 0x7f2fa8073a00 2023-10-16T14:32:57.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.636+0000 7f2f9effd700 1 -- 172.21.15.178:0/3639178871 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f98099c90 con 0x7f2fa8073a00 2023-10-16T14:32:57.642 INFO:teuthology.orchestra.run.smithi178.stdout:197568495667 2023-10-16T14:32:57.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 msgr2=0x7f2f940a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:57.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 0x7f2f940a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4006010 tx=0x7f2fa400b260 comp rx=0 tx=0).stop 2023-10-16T14:32:57.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 msgr2=0x7f2fa8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:57.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2f9800e9f0 tx=0x7f2f9800ed00 comp rx=0 tx=0).stop 2023-10-16T14:32:57.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 shutdown_connections 2023-10-16T14:32:57.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2f940a1510 0x7f2f940a39d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2fa810d930 0x7f2fa819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2fa80730e0 0x7f2fa81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 --2- 172.21.15.178:0/3639178871 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2fa8073a00 0x7f2fa8199800 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:57.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 >> 172.21.15.178:0/3639178871 conn(0x7f2fa80780a0 msgr2=0x7f2fa80fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:57.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 shutdown_connections 2023-10-16T14:32:57.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:57.639+0000 7f2fafbae700 1 -- 172.21.15.178:0/3639178871 wait complete. 2023-10-16T14:32:57.853 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149759 got 111669149761 for osd.3 2023-10-16T14:32:57.853 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:57.968 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:57 smithi178 ceph-mon[98495]: pgmap v314: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-10-16T14:32:57.969 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:57 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3639178871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-16T14:32:58.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3221142360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c100f80 msgr2=0x7f2c5c101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:58.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3221142360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c100f80 0x7f2c5c101400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00fb20 tx=0x7f2c4c00fe30 comp rx=0 tx=0).stop 2023-10-16T14:32:58.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3221142360 shutdown_connections 2023-10-16T14:32:58.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3221142360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c101940 0x7f2c5c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3221142360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c106f90 0x7f2c5c107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3221142360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c100f80 0x7f2c5c101400 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.035+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3221142360 >> 172.21.15.178:0/3221142360 conn(0x7f2c5c0780a0 msgr2=0x7f2c5c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:58.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.036+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3221142360 shutdown_connections 2023-10-16T14:32:58.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.036+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3221142360 wait complete. 2023-10-16T14:32:58.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.037+0000 7f2c63d3b700 1 Processor -- start 2023-10-16T14:32:58.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.037+0000 7f2c63d3b700 1 -- start start 2023-10-16T14:32:58.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:58.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c101940 0x7f2c5c195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:58.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c106f90 0x7f2c5c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:58.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c199d90 con 0x7f2c5c100f80 2023-10-16T14:32:58.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c199f00 con 0x7f2c5c106f90 2023-10-16T14:32:58.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c63d3b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c19a0c0 con 0x7f2c5c101940 2023-10-16T14:32:58.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:58.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56980/0 (socket says 172.21.15.178:56980) 2023-10-16T14:32:58.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 -- 172.21.15.178:0/3042083406 learned_addr learned my addr 172.21.15.178:0/3042083406 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:58.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c612d6700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c101940 0x7f2c5c195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:58.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c622d8700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c106f90 0x7f2c5c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:58.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 -- 172.21.15.178:0/3042083406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c101940 msgr2=0x7f2c5c195310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:58.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c101940 0x7f2c5c195310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 -- 172.21.15.178:0/3042083406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c106f90 msgr2=0x7f2c5c199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:58.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c106f90 0x7f2c5c199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.038+0000 7f2c61ad7700 1 -- 172.21.15.178:0/3042083406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54009c40 con 0x7f2c5c100f80 2023-10-16T14:32:58.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c61ad7700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:58.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800f890 con 0x7f2c5c100f80 2023-10-16T14:32:58.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c58010460 con 0x7f2c5c100f80 2023-10-16T14:32:58.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c00f750 con 0x7f2c5c100f80 2023-10-16T14:32:58.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800e610 con 0x7f2c5c100f80 2023-10-16T14:32:58.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.039+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c1a1a30 con 0x7f2c5c100f80 2023-10-16T14:32:58.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.040+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c04ebb0 con 0x7f2c5c100f80 2023-10-16T14:32:58.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.042+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f2c580108d0 con 0x7f2c5c100f80 2023-10-16T14:32:58.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.044+0000 7f2c52ffd700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 0x7f2c480a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:58.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.044+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2c580ccf90 con 0x7f2c5c100f80 2023-10-16T14:32:58.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.045+0000 7f2c612d6700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 0x7f2c480a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:58.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.045+0000 7f2c612d6700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 0x7f2c480a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00faf0 tx=0x7f2c4c01b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:58.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.049+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2c58096120 con 0x7f2c5c100f80 2023-10-16T14:32:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:57 smithi186 ceph-mon[134892]: pgmap v314: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-10-16T14:32:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:57 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3639178871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-16T14:32:58.102 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:57 smithi187 ceph-mon[135297]: pgmap v314: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-10-16T14:32:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:57 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3639178871' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-10-16T14:32:58.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.272+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f2c5c10ee50 con 0x7f2c5c100f80 2023-10-16T14:32:58.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.273+0000 7f2c52ffd700 1 -- 172.21.15.178:0/3042083406 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2c58016090 con 0x7f2c5c100f80 2023-10-16T14:32:58.276 INFO:teuthology.orchestra.run.smithi178.stdout:335007449117 2023-10-16T14:32:58.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 msgr2=0x7f2c480a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:58.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 0x7f2c480a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00faf0 tx=0x7f2c4c01b040 comp rx=0 tx=0).stop 2023-10-16T14:32:58.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 msgr2=0x7f2c5c194dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:58.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800d950 tx=0x7f2c5800dc60 comp rx=0 tx=0).stop 2023-10-16T14:32:58.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 shutdown_connections 2023-10-16T14:32:58.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2c480a1510 0x7f2c480a39d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2c5c106f90 0x7f2c5c199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c5c100f80 0x7f2c5c194dd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 --2- 172.21.15.178:0/3042083406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c5c101940 0x7f2c5c195310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:58.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 >> 172.21.15.178:0/3042083406 conn(0x7f2c5c0780a0 msgr2=0x7f2c5c104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:58.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.275+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 shutdown_connections 2023-10-16T14:32:58.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:58.276+0000 7f2c63d3b700 1 -- 172.21.15.178:0/3042083406 wait complete. 2023-10-16T14:32:58.660 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841569 got 283467841573 for osd.9 2023-10-16T14:32:58.660 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:32:58.661 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:58.662 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411398 for osd.2 2023-10-16T14:32:58.663 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:58.888 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:58 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3042083406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-16T14:32:59.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:58 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3042083406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-16T14:32:59.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:58 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3042083406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-10-16T14:32:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 -- 172.21.15.178:0/1172037876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c1033b0 msgr2=0x7ff74c103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1172037876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c1033b0 0x7ff74c103790 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff7480098f0 tx=0x7ff748009c00 comp rx=0 tx=0).stop 2023-10-16T14:32:59.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 -- 172.21.15.178:0/1172037876 shutdown_connections 2023-10-16T14:32:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1172037876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff74c108490 0x7ff74c110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1172037876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c1033b0 0x7ff74c103790 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1172037876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.421+0000 7ff753c0f700 1 -- 172.21.15.178:0/1172037876 >> 172.21.15.178:0/1172037876 conn(0x7ff74c0fec20 msgr2=0x7ff74c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.425+0000 7ff753c0f700 1 -- 172.21.15.178:0/1172037876 shutdown_connections 2023-10-16T14:32:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.425+0000 7ff753c0f700 1 -- 172.21.15.178:0/1172037876 wait complete. 2023-10-16T14:32:59.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.426+0000 7ff753c0f700 1 Processor -- start 2023-10-16T14:32:59.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.426+0000 7ff753c0f700 1 -- start start 2023-10-16T14:32:59.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff74c1033b0 0x7ff74c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 0x7ff74c19dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19e2b0 con 0x7ff74c103d60 2023-10-16T14:32:59.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19e420 con 0x7ff74c108490 2023-10-16T14:32:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff753c0f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c19e5e0 con 0x7ff74c1033b0 2023-10-16T14:32:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff7511aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff7511aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37848/0 (socket says 172.21.15.178:37848) 2023-10-16T14:32:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.427+0000 7ff7511aa700 1 -- 172.21.15.178:0/1590522678 learned_addr learned my addr 172.21.15.178:0/1590522678 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:59.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7521ac700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 0x7ff74c19dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 -- 172.21.15.178:0/1590522678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff74c1033b0 msgr2=0x7ff74c1992f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T14:32:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff74c1033b0 0x7ff74c1992f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 -- 172.21.15.178:0/1590522678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 msgr2=0x7ff74c19dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 0x7ff74c19dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 -- 172.21.15.178:0/1590522678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c0095f0 con 0x7ff74c103d60 2023-10-16T14:32:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7521ac700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 0x7ff74c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:32:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff7511aa700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00ec70 tx=0x7ff73c00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:59.436 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0099d0 con 0x7ff74c103d60 2023-10-16T14:32:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff73c0044e0 con 0x7ff74c103d60 2023-10-16T14:32:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff748009520 con 0x7ff74c103d60 2023-10-16T14:32:59.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.428+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c010690 con 0x7ff74c103d60 2023-10-16T14:32:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.429+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c06a8d0 con 0x7ff74c103d60 2023-10-16T14:32:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.430+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c04ebb0 con 0x7ff74c103d60 2023-10-16T14:32:59.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.434+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7ff73c003e20 con 0x7ff74c103d60 2023-10-16T14:32:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.436+0000 7ff742ffd700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 0x7ff7380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.436+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff73c014070 con 0x7ff74c103d60 2023-10-16T14:32:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.436+0000 7ff7519ab700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 0x7ff7380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.440 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.437+0000 7ff7519ab700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 0x7ff7380a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff748009230 tx=0x7ff7480058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:59.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.440+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff73c0969b0 con 0x7ff74c103d60 2023-10-16T14:32:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 -- 172.21.15.178:0/3093230452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8074ef0 msgr2=0x7f56e80730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 --2- 172.21.15.178:0/3093230452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8074ef0 0x7f56e80730e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f56e40098f0 tx=0x7f56e4009c00 comp rx=0 tx=0).stop 2023-10-16T14:32:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 -- 172.21.15.178:0/3093230452 shutdown_connections 2023-10-16T14:32:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 --2- 172.21.15.178:0/3093230452 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8111bc0 0x7f56e8113fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 --2- 172.21.15.178:0/3093230452 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8074ef0 0x7f56e80730e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 --2- 172.21.15.178:0/3093230452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e8073a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.337+0000 7f56efba0700 1 -- 172.21.15.178:0/3093230452 >> 172.21.15.178:0/3093230452 conn(0x7f56e80780a0 msgr2=0x7f56e80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:59.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.338+0000 7f56efba0700 1 -- 172.21.15.178:0/3093230452 shutdown_connections 2023-10-16T14:32:59.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.338+0000 7f56efba0700 1 -- 172.21.15.178:0/3093230452 wait complete. 2023-10-16T14:32:59.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.339+0000 7f56efba0700 1 Processor -- start 2023-10-16T14:32:59.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.339+0000 7f56efba0700 1 -- start start 2023-10-16T14:32:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 0x7f56e81a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8111bc0 0x7f56e819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e819dd00 con 0x7f56e8074ef0 2023-10-16T14:32:59.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e819de70 con 0x7f56e8111bc0 2023-10-16T14:32:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56efba0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e819e030 con 0x7f56e8073620 2023-10-16T14:32:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ed13b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 0x7f56e81a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ed93c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ed13b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 0x7f56e81a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37830/0 (socket says 172.21.15.178:37830) 2023-10-16T14:32:59.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ed93c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:38786/0 (socket says 172.21.15.178:38786) 2023-10-16T14:32:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ed13b700 1 -- 172.21.15.178:0/514718321 learned_addr learned my addr 172.21.15.178:0/514718321 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.340+0000 7f56ee13d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8111bc0 0x7f56e819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 -- 172.21.15.178:0/514718321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8111bc0 msgr2=0x7f56e819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8111bc0 0x7f56e819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 -- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 msgr2=0x7f56e81a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 0x7f56e81a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 -- 172.21.15.178:0/514718321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80095f0 con 0x7f56e8073620 2023-10-16T14:32:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56ed93c700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009810 tx=0x7f56e400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:59.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e401c070 con 0x7f56e8073620 2023-10-16T14:32:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e4009520 con 0x7f56e8073620 2023-10-16T14:32:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56e400bba0 con 0x7f56e8073620 2023-10-16T14:32:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.341+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4003970 con 0x7f56e8073620 2023-10-16T14:32:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.342+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e81aa310 con 0x7f56e8073620 2023-10-16T14:32:59.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.343+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e804ebb0 con 0x7f56e8073620 2023-10-16T14:32:59.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.345+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f56e4020030 con 0x7f56e8073620 2023-10-16T14:32:59.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.346+0000 7f56deffd700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 0x7f56d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.346+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f56e40ce1c0 con 0x7f56e8073620 2023-10-16T14:32:59.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.346+0000 7f56ed13b700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 0x7f56d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.347+0000 7f56ed13b700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 0x7f56d40a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f56e819edf0 tx=0x7f56d8009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:59.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.348+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f56e40973e0 con 0x7f56e8073620 2023-10-16T14:32:59.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.602+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f56e819ea80 con 0x7f56e8073620 2023-10-16T14:32:59.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.603+0000 7f56deffd700 1 -- 172.21.15.178:0/514718321 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f56e409aa00 con 0x7f56e8073620 2023-10-16T14:32:59.607 INFO:teuthology.orchestra.run.smithi178.stdout:304942678049 2023-10-16T14:32:59.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.606+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 msgr2=0x7f56d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.606+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 0x7f56d40a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f56e819edf0 tx=0x7f56d8009310 comp rx=0 tx=0).stop 2023-10-16T14:32:59.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.606+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 msgr2=0x7f56e81a3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.606+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009810 tx=0x7f56e400fa00 comp rx=0 tx=0).stop 2023-10-16T14:32:59.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 shutdown_connections 2023-10-16T14:32:59.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f56d40a1510 0x7f56d40a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f56e8111bc0 0x7f56e819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56e8073620 0x7f56e81a3560 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 --2- 172.21.15.178:0/514718321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56e8074ef0 0x7f56e81a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 >> 172.21.15.178:0/514718321 conn(0x7f56e80780a0 msgr2=0x7f56e8110720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:59.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 shutdown_connections 2023-10-16T14:32:59.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.607+0000 7f56efba0700 1 -- 172.21.15.178:0/514718321 wait complete. 2023-10-16T14:32:59.629 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495667 for osd.6 2023-10-16T14:32:59.629 DEBUG:teuthology.parallel:result is None 2023-10-16T14:32:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.717+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff74c19a650 con 0x7ff74c103d60 2023-10-16T14:32:59.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.718+0000 7ff742ffd700 1 -- 172.21.15.178:0/1590522678 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff73c099fd0 con 0x7ff74c103d60 2023-10-16T14:32:59.721 INFO:teuthology.orchestra.run.smithi178.stdout:219043332143 2023-10-16T14:32:59.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.720+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 msgr2=0x7ff7380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.720+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 0x7ff7380a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff748009230 tx=0x7ff7480058e0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.720+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 msgr2=0x7ff74c199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.720+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00ec70 tx=0x7ff73c00c5b0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 shutdown_connections 2023-10-16T14:32:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7380a1510 0x7ff7380a39d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff74c108490 0x7ff74c19dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff74c1033b0 0x7ff74c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 --2- 172.21.15.178:0/1590522678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff74c103d60 0x7ff74c199830 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 >> 172.21.15.178:0/1590522678 conn(0x7ff74c0fec20 msgr2=0x7ff74c106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.721+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 shutdown_connections 2023-10-16T14:32:59.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.722+0000 7ff753c0f700 1 -- 172.21.15.178:0/1590522678 wait complete. 2023-10-16T14:32:59.755 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:59 smithi178 ceph-mon[98495]: pgmap v315: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-10-16T14:32:59.755 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:32:59 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/514718321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-16T14:32:59.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 -- 172.21.15.178:0/2488103248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec100f80 msgr2=0x7fd3ec101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 --2- 172.21.15.178:0/2488103248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec100f80 0x7fd3ec101400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0099c0 tx=0x7fd3dc009cd0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 -- 172.21.15.178:0/2488103248 shutdown_connections 2023-10-16T14:32:59.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 --2- 172.21.15.178:0/2488103248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec101940 0x7fd3ec10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 --2- 172.21.15.178:0/2488103248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec106f90 0x7fd3ec107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 --2- 172.21.15.178:0/2488103248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec100f80 0x7fd3ec101400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.908+0000 7fd3f4575700 1 -- 172.21.15.178:0/2488103248 >> 172.21.15.178:0/2488103248 conn(0x7fd3ec0780a0 msgr2=0x7fd3ec0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:32:59.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.909+0000 7fd3f4575700 1 -- 172.21.15.178:0/2488103248 shutdown_connections 2023-10-16T14:32:59.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.909+0000 7fd3f4575700 1 -- 172.21.15.178:0/2488103248 wait complete. 2023-10-16T14:32:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 Processor -- start 2023-10-16T14:32:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 -- start start 2023-10-16T14:32:59.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 0x7fd3ec199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 0x7fd3ec19dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.910+0000 7fd3f4575700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec19e290 con 0x7fd3ec101940 2023-10-16T14:32:59.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f4575700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec19e400 con 0x7fd3ec100f80 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f4575700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec19e5c0 con 0x7fd3ec106f90 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f1b10700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 0x7fd3ec199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52402/0 (socket says 172.21.15.178:52402) 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 -- 172.21.15.178:0/1821792030 learned_addr learned my addr 172.21.15.178:0/1821792030 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:32:59.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2b12700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 0x7fd3ec19dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 -- 172.21.15.178:0/1821792030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 msgr2=0x7fd3ec19dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 0x7fd3ec19dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 -- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 msgr2=0x7fd3ec199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:32:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 0x7fd3ec199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:32:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2311700 1 -- 172.21.15.178:0/1821792030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0095f0 con 0x7fd3ec100f80 2023-10-16T14:32:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.911+0000 7fd3f2b12700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 0x7fd3ec19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3f1b10700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 0x7fd3ec199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:32:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3f2311700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9a0 tx=0x7fd3e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:32:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8009790 con 0x7fd3ec100f80 2023-10-16T14:32:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e8010430 con 0x7fd3ec100f80 2023-10-16T14:32:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec19e8a0 con 0x7fd3ec100f80 2023-10-16T14:32:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8018680 con 0x7fd3ec100f80 2023-10-16T14:32:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.912+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec1a6100 con 0x7fd3ec100f80 2023-10-16T14:32:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.914+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec04ebb0 con 0x7fd3ec100f80 2023-10-16T14:32:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.917+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fd3e80105a0 con 0x7fd3ec100f80 2023-10-16T14:32:59.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.919+0000 7fd3e37fe700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 0x7fd3d80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:32:59.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.919+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd3e8014070 con 0x7fd3ec100f80 2023-10-16T14:32:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.920+0000 7fd3f1b10700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 0x7fd3d80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:32:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.920+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd3e80969c0 con 0x7fd3ec100f80 2023-10-16T14:32:59.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:32:59.921+0000 7fd3f1b10700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 0x7fd3d80a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009990 tx=0x7fd3dc005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:59 smithi186 ceph-mon[134892]: pgmap v315: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-10-16T14:33:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:32:59 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/514718321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-16T14:33:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.158+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd3ec110f70 con 0x7fd3ec100f80 2023-10-16T14:33:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.159+0000 7fd3e37fe700 1 -- 172.21.15.178:0/1821792030 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3e8099fe0 con 0x7fd3ec100f80 2023-10-16T14:33:00.162 INFO:teuthology.orchestra.run.smithi178.stdout:249108103211 2023-10-16T14:33:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 msgr2=0x7fd3d80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 0x7fd3d80a3a50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009990 tx=0x7fd3dc005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 msgr2=0x7fd3ec1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e9a0 tx=0x7fd3e800ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 shutdown_connections 2023-10-16T14:33:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd3d80a1590 0x7fd3d80a3a50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3ec106f90 0x7fd3ec19dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd3ec100f80 0x7fd3ec1992d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 --2- 172.21.15.178:0/1821792030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3ec101940 0x7fd3ec199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 >> 172.21.15.178:0/1821792030 conn(0x7fd3ec0780a0 msgr2=0x7fd3ec0730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:00.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 shutdown_connections 2023-10-16T14:33:00.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:00.161+0000 7fd3f4575700 1 -- 172.21.15.178:0/1821792030 wait complete. 2023-10-16T14:33:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:59 smithi187 ceph-mon[135297]: pgmap v315: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-10-16T14:33:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:32:59 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/514718321' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-10-16T14:33:01.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1590522678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-16T14:33:01.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1821792030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-16T14:33:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1590522678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-16T14:33:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1821792030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-16T14:33:01.169 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103209 got 249108103211 for osd.8 2023-10-16T14:33:01.169 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1590522678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-10-16T14:33:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1821792030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-10-16T14:33:01.568 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449114 got 335007449117 for osd.11 2023-10-16T14:33:01.569 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:01.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332139 got 219043332143 for osd.7 2023-10-16T14:33:01.803 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:02.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:01 smithi178 ceph-mon[98495]: pgmap v316: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:02.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:01 smithi186 ceph-mon[134892]: pgmap v316: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:02.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:01 smithi187 ceph-mon[135297]: pgmap v316: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:02.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678049 for osd.10 2023-10-16T14:33:02.502 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:33:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:33:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 -- 172.21.15.178:0/2765997445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670101940 msgr2=0x7f067010ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 --2- 172.21.15.178:0/2765997445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670101940 0x7f067010ddb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f066800b0d0 tx=0x7f066800b3e0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 -- 172.21.15.178:0/2765997445 shutdown_connections 2023-10-16T14:33:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 --2- 172.21.15.178:0/2765997445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670101940 0x7f067010ddb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 --2- 172.21.15.178:0/2765997445 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670106f90 0x7f0670107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 --2- 172.21.15.178:0/2765997445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670100f80 0x7f0670101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 -- 172.21.15.178:0/2765997445 >> 172.21.15.178:0/2765997445 conn(0x7f06700780a0 msgr2=0x7f06700784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:02.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.911+0000 7f0675387700 1 -- 172.21.15.178:0/2765997445 shutdown_connections 2023-10-16T14:33:02.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.912+0000 7f0675387700 1 -- 172.21.15.178:0/2765997445 wait complete. 2023-10-16T14:33:02.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 Processor -- start 2023-10-16T14:33:02.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 -- start start 2023-10-16T14:33:02.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:02.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670101940 0x7f06701082e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:02.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 0x7f0670108820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:02.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06701a99f0 con 0x7f0670106f90 2023-10-16T14:33:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06701a9b60 con 0x7f0670100f80 2023-10-16T14:33:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.913+0000 7f0675387700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06701a9d20 con 0x7f0670101940 2023-10-16T14:33:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52416/0 (socket says 172.21.15.178:52416) 2023-10-16T14:33:02.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 -- 172.21.15.178:0/3382332102 learned_addr learned my addr 172.21.15.178:0/3382332102 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:02.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066e7fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670101940 0x7f06701082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:02.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066f7fe700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 0x7f0670108820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:02.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 -- 172.21.15.178:0/3382332102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670101940 msgr2=0x7f06701082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:02.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670101940 0x7f06701082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 -- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 msgr2=0x7f0670108820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 0x7f0670108820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 -- 172.21.15.178:0/3382332102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0668009d00 con 0x7f0670100f80 2023-10-16T14:33:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066f7fe700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 0x7f0670108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:33:02.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.914+0000 7f066effd700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f066000e9a0 tx=0x7f066000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.915+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660009790 con 0x7f0670100f80 2023-10-16T14:33:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.915+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f066000cbd0 con 0x7f0670100f80 2023-10-16T14:33:02.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.915+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0660018680 con 0x7f0670100f80 2023-10-16T14:33:02.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.917+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06701a9fa0 con 0x7f0670100f80 2023-10-16T14:33:02.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.917+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06701aa420 con 0x7f0670100f80 2023-10-16T14:33:02.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.918+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067004ebb0 con 0x7f0670100f80 2023-10-16T14:33:02.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.921+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f066000cd40 con 0x7f0670100f80 2023-10-16T14:33:02.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.923+0000 7f0657fff700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 0x7f06580a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:02.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.923+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0660014070 con 0x7f0670100f80 2023-10-16T14:33:02.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.923+0000 7f066e7fc700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 0x7f06580a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:02.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.924+0000 7f066e7fc700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 0x7f06580a3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0670109640 tx=0x7f0664009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:02.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:02.927+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f06600966e0 con 0x7f0670100f80 2023-10-16T14:33:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:33:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:33:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.148+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0670111120 con 0x7f0670100f80 2023-10-16T14:33:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.149+0000 7f0657fff700 1 -- 172.21.15.178:0/3382332102 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0660099d00 con 0x7f0670100f80 2023-10-16T14:33:03.152 INFO:teuthology.orchestra.run.smithi178.stdout:163208757305 2023-10-16T14:33:03.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 msgr2=0x7f06580a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 0x7f06580a3a50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0670109640 tx=0x7f0664009260 comp rx=0 tx=0).stop 2023-10-16T14:33:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 msgr2=0x7f0670107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f066000e9a0 tx=0x7f066000ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 shutdown_connections 2023-10-16T14:33:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f06580a1590 0x7f06580a3a50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0670106f90 0x7f0670108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0670100f80 0x7f0670107da0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 --2- 172.21.15.178:0/3382332102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0670101940 0x7f06701082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 >> 172.21.15.178:0/3382332102 conn(0x7f06700780a0 msgr2=0x7f067010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 shutdown_connections 2023-10-16T14:33:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.152+0000 7f0675387700 1 -- 172.21.15.178:0/3382332102 wait complete. 2023-10-16T14:33:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:33:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:33:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:33:03.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.251+0000 7f1531a9c700 1 -- 172.21.15.178:0/3916871364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c106f90 msgr2=0x7f152c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.251+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3916871364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c106f90 0x7f152c107370 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f151c009920 tx=0x7f151c009c30 comp rx=0 tx=0).stop 2023-10-16T14:33:03.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 -- 172.21.15.178:0/3916871364 shutdown_connections 2023-10-16T14:33:03.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3916871364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c101940 0x7f152c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3916871364 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c106f90 0x7f152c107370 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3916871364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c100f80 0x7f152c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 -- 172.21.15.178:0/3916871364 >> 172.21.15.178:0/3916871364 conn(0x7f152c0780a0 msgr2=0x7f152c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:03.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 -- 172.21.15.178:0/3916871364 shutdown_connections 2023-10-16T14:33:03.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.252+0000 7f1531a9c700 1 -- 172.21.15.178:0/3916871364 wait complete. 2023-10-16T14:33:03.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.253+0000 7f1531a9c700 1 Processor -- start 2023-10-16T14:33:03.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.253+0000 7f1531a9c700 1 -- start start 2023-10-16T14:33:03.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.253+0000 7f1531a9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:03.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.253+0000 7f1531a9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 0x7f152c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:03.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f1531a9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c106f90 0x7f152c073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:03.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f1531a9c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1a54a0 con 0x7f152c106f90 2023-10-16T14:33:03.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f1531a9c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1a5610 con 0x7f152c101940 2023-10-16T14:33:03.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f1531a9c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c1a57d0 con 0x7f152c100f80 2023-10-16T14:33:03.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c106f90 0x7f152c073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:03.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:03.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:38866/0 (socket says 172.21.15.178:38866) 2023-10-16T14:33:03.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 -- 172.21.15.178:0/3656052299 learned_addr learned my addr 172.21.15.178:0/3656052299 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:03.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152affd700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 0x7f152c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 -- 172.21.15.178:0/3656052299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 msgr2=0x7f152c0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 0x7f152c0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 -- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c106f90 msgr2=0x7f152c073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c106f90 0x7f152c073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152b7fe700 1 -- 172.21.15.178:0/3656052299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200095f0 con 0x7f152c100f80 2023-10-16T14:33:03.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.254+0000 7f152affd700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 0x7f152c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:03.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.255+0000 7f152b7fe700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c009840 tx=0x7f151c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:03.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.255+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c01c070 con 0x7f152c100f80 2023-10-16T14:33:03.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.255+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c009520 con 0x7f152c100f80 2023-10-16T14:33:03.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.255+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f151c00bba0 con 0x7f152c100f80 2023-10-16T14:33:03.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.255+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c0038c0 con 0x7f152c100f80 2023-10-16T14:33:03.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.256+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c1a5c80 con 0x7f152c100f80 2023-10-16T14:33:03.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.257+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c04ebb0 con 0x7f152c100f80 2023-10-16T14:33:03.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.258+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f151c020030 con 0x7f152c100f80 2023-10-16T14:33:03.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.259+0000 7f1528ff9700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 0x7f15140a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:03.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.260+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f151c0ce0b0 con 0x7f152c100f80 2023-10-16T14:33:03.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.260+0000 7f152affd700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 0x7f15140a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:03.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.260+0000 7f152affd700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 0x7f15140a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f152c0749a0 tx=0x7f1520009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:03.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.263+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f151c097330 con 0x7f152c100f80 2023-10-16T14:33:03.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.474+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f152c111110 con 0x7f152c100f80 2023-10-16T14:33:03.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.475+0000 7f1528ff9700 1 -- 172.21.15.178:0/3656052299 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f151c09a950 con 0x7f152c100f80 2023-10-16T14:33:03.478 INFO:teuthology.orchestra.run.smithi178.stdout:55834574921 2023-10-16T14:33:03.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 msgr2=0x7f15140a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 0x7f15140a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f152c0749a0 tx=0x7f1520009310 comp rx=0 tx=0).stop 2023-10-16T14:33:03.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 msgr2=0x7f152c0731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:03.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f151c009840 tx=0x7f151c00fa00 comp rx=0 tx=0).stop 2023-10-16T14:33:03.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 shutdown_connections 2023-10-16T14:33:03.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15140a1510 0x7f15140a39d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f152c106f90 0x7f152c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f152c100f80 0x7f152c0731b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 --2- 172.21.15.178:0/3656052299 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f152c101940 0x7f152c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:03.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 >> 172.21.15.178:0/3656052299 conn(0x7f152c0780a0 msgr2=0x7f152c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:03.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.477+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 shutdown_connections 2023-10-16T14:33:03.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:03.478+0000 7f1531a9c700 1 -- 172.21.15.178:0/3656052299 wait complete. 2023-10-16T14:33:03.715 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757305 for osd.5 2023-10-16T14:33:03.715 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:04.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574918 got 55834574921 for osd.1 2023-10-16T14:33:04.204 DEBUG:teuthology.parallel:result is None 2023-10-16T14:33:04.204 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-10-16T14:33:04.205 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-16T14:33:04.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:04 smithi178 ceph-mon[98495]: pgmap v317: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3382332102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-16T14:33:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3656052299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-16T14:33:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:04 smithi186 ceph-mon[134892]: pgmap v317: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3382332102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-16T14:33:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3656052299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-16T14:33:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:04 smithi187 ceph-mon[135297]: pgmap v317: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3382332102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-10-16T14:33:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3656052299' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-10-16T14:33:04.941 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:06.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:06 smithi178 ceph-mon[98495]: pgmap v318: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:06 smithi186 ceph-mon[134892]: pgmap v318: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:06.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:06 smithi187 ceph-mon[135297]: pgmap v318: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:06.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 -- 172.21.15.178:0/1453971520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 msgr2=0x7f2d6c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:06.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/1453971520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c0734c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009990 tx=0x7f2d5c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 -- 172.21.15.178:0/1453971520 shutdown_connections 2023-10-16T14:33:06.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/1453971520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 0x7f2d6c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/1453971520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c0734c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/1453971520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c10d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 -- 172.21.15.178:0/1453971520 >> 172.21.15.178:0/1453971520 conn(0x7f2d6c0780a0 msgr2=0x7f2d6c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:06.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 -- 172.21.15.178:0/1453971520 shutdown_connections 2023-10-16T14:33:06.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.582+0000 7f2d71f3c700 1 -- 172.21.15.178:0/1453971520 wait complete. 2023-10-16T14:33:06.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.583+0000 7f2d71f3c700 1 Processor -- start 2023-10-16T14:33:06.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.583+0000 7f2d71f3c700 1 -- start start 2023-10-16T14:33:06.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:06.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:06.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 0x7f2d6c19dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:06.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c19e270 con 0x7f2d6c0730e0 2023-10-16T14:33:06.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c19e3e0 con 0x7f2d6c10d930 2023-10-16T14:33:06.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d71f3c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c19e5a0 con 0x7f2d6c073a00 2023-10-16T14:33:06.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:06.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d6b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37912/0 (socket says 172.21.15.178:37912) 2023-10-16T14:33:06.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d6affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:06.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d6b7fe700 1 -- 172.21.15.178:0/3264913137 learned_addr learned my addr 172.21.15.178:0/3264913137 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:06.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.584+0000 7f2d6bfff700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 0x7f2d6c19dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:06.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 -- 172.21.15.178:0/3264913137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 msgr2=0x7f2d6c19dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 0x7f2d6c19dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 -- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 msgr2=0x7f2d6c1992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 -- 172.21.15.178:0/3264913137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0095f0 con 0x7f2d6c073a00 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6b7fe700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:06.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d6affd700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c1997f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9f0 tx=0x7f2d6000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:06.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000c9c0 con 0x7f2d6c073a00 2023-10-16T14:33:06.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c19e880 con 0x7f2d6c073a00 2023-10-16T14:33:06.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d60010430 con 0x7f2d6c073a00 2023-10-16T14:33:06.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60018680 con 0x7f2d6c073a00 2023-10-16T14:33:06.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.585+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c1a60e0 con 0x7f2d6c073a00 2023-10-16T14:33:06.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.586+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c04ebb0 con 0x7f2d6c073a00 2023-10-16T14:33:06.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.589+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f2d60010860 con 0x7f2d6c073a00 2023-10-16T14:33:06.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.590+0000 7f2d68ff9700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 0x7f2d540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:06.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.590+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d60014070 con 0x7f2d6c073a00 2023-10-16T14:33:06.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.590+0000 7f2d6b7fe700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 0x7f2d540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:06.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.591+0000 7f2d6b7fe700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 0x7f2d540a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c006010 tx=0x7f2d5c00b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:06.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.592+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2d60096670 con 0x7f2d6c073a00 2023-10-16T14:33:06.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.788+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d6c113250 con 0x7f2d540a1510 2023-10-16T14:33:06.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.802+0000 7f2d68ff9700 1 -- 172.21.15.178:0/3264913137 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427182 (secure 0 0 0) 0x7f2d6c113250 con 0x7f2d540a1510 2023-10-16T14:33:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:06.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 msgr2=0x7f2d540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:06.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 0x7f2d540a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c006010 tx=0x7f2d5c00b260 comp rx=0 tx=0).stop 2023-10-16T14:33:06.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 msgr2=0x7f2d6c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:06.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c1997f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9f0 tx=0x7f2d6000ed00 comp rx=0 tx=0).stop 2023-10-16T14:33:06.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 shutdown_connections 2023-10-16T14:33:06.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2d540a1510 0x7f2d540a39d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2d6c10d930 0x7f2d6c19dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d6c0730e0 0x7f2d6c1992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 --2- 172.21.15.178:0/3264913137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2d6c073a00 0x7f2d6c1997f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:06.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 >> 172.21.15.178:0/3264913137 conn(0x7f2d6c0780a0 msgr2=0x7f2d6c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:06.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 shutdown_connections 2023-10-16T14:33:06.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:06.808+0000 7f2d71f3c700 1 -- 172.21.15.178:0/3264913137 wait complete. 2023-10-16T14:33:06.818 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:33:07.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2023-10-16T14:33:07.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2023-10-16T14:33:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2023-10-16T14:33:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2023-10-16T14:33:07.541 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2023-10-16T14:33:06.609825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":480,"num_read_kb":368,"num_write":347,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337952,"kb_used_data":13748,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603872,"statfs":{"total":1151940427776,"available":1151594364928,"internally_reserved":0,"allocated":14077952,"data_stored":7826917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005803"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.668225+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:32:06.668225+0000","last_peered":"2023-10-16T14:32:06.668225+0000","last_clean":"2023-10-16T14:32:06.668225+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:32:06.668225+0000","last_undegraded":"2023-10-16T14:32:06.668225+0000","last_fullsized":"2023-10-16T14:32:06.668225+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675248+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:32:06.675248+0000","last_peered":"2023-10-16T14:32:06.675248+0000","last_clean":"2023-10-16T14:32:06.675248+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:32:06.675248+0000","last_undegraded":"2023-10-16T14:32:06.675248+0000","last_fullsized":"2023-10-16T14:32:06.675248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674667+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:06.674667+0000","last_peered":"2023-10-16T14:32:06.674667+0000","last_clean":"2023-10-16T14:32:06.674667+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:06.674667+0000","last_undegraded":"2023-10-16T14:32:06.674667+0000","last_fullsized":"2023-10-16T14:32:06.674667+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.669076+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:32:06.669076+0000","last_peered":"2023-10-16T14:32:06.669076+0000","last_clean":"2023-10-16T14:32:06.669076+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:32:06.669076+0000","last_undegraded":"2023-10-16T14:32:06.669076+0000","last_fullsized":"2023-10-16T14:32:06.669076+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675379+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:32:06.675379+0000","last_peered":"2023-10-16T14:32:06.675379+0000","last_clean":"2023-10-16T14:32:06.675379+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:32:06.675379+0000","last_undegraded":"2023-10-16T14:32:06.675379+0000","last_fullsized":"2023-10-16T14:32:06.675379+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671873+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:06.671873+0000","last_peered":"2023-10-16T14:32:06.671873+0000","last_clean":"2023-10-16T14:32:06.671873+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:06.671873+0000","last_undegraded":"2023-10-16T14:32:06.671873+0000","last_fullsized":"2023-10-16T14:32:06.671873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672442+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:06.672442+0000","last_peered":"2023-10-16T14:32:06.672442+0000","last_clean":"2023-10-16T14:32:06.672442+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:06.672442+0000","last_undegraded":"2023-10-16T14:32:06.672442+0000","last_fullsized":"2023-10-16T14:32:06.672442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:03.792526+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:33:03.792526+0000","last_peered":"2023-10-16T14:33:03.792526+0000","last_clean":"2023-10-16T14:33:03.792526+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:33:03.792526+0000","last_undegraded":"2023-10-16T14:33:03.792526+0000","last_fullsized":"2023-10-16T14:33:03.792526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":12,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702769+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:32:08.702769+0000","last_peered":"2023-10-16T14:32:08.702769+0000","last_clean":"2023-10-16T14:32:08.702769+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:32:08.702769+0000","last_undegraded":"2023-10-16T14:32:08.702769+0000","last_fullsized":"2023-10-16T14:32:08.702769+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:00.787186+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:33:00.787186+0000","last_peered":"2023-10-16T14:33:00.787186+0000","last_clean":"2023-10-16T14:33:00.787186+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:33:00.787186+0000","last_undegraded":"2023-10-16T14:33:00.787186+0000","last_fullsized":"2023-10-16T14:33:00.787186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674416+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:06.674416+0000","last_peered":"2023-10-16T14:32:06.674416+0000","last_clean":"2023-10-16T14:32:06.674416+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:06.674416+0000","last_undegraded":"2023-10-16T14:32:06.674416+0000","last_fullsized":"2023-10-16T14:32:06.674416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.924273+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:32:07.924273+0000","last_peered":"2023-10-16T14:32:07.924273+0000","last_clean":"2023-10-16T14:32:07.924273+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:32:07.924273+0000","last_undegraded":"2023-10-16T14:32:07.924273+0000","last_fullsized":"2023-10-16T14:32:07.924273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673223+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:06.673223+0000","last_peered":"2023-10-16T14:32:06.673223+0000","last_clean":"2023-10-16T14:32:06.673223+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:06.673223+0000","last_undegraded":"2023-10-16T14:32:06.673223+0000","last_fullsized":"2023-10-16T14:32:06.673223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671298+0000","last_change":"2023-10-16T14:31:41.088650+0000","last_active":"2023-10-16T14:32:06.671298+0000","last_peered":"2023-10-16T14:32:06.671298+0000","last_clean":"2023-10-16T14:32:06.671298+0000","last_became_active":"2023-10-16T14:31:41.088093+0000","last_became_peered":"2023-10-16T14:31:41.088093+0000","last_unstale":"2023-10-16T14:32:06.671298+0000","last_undegraded":"2023-10-16T14:32:06.671298+0000","last_fullsized":"2023-10-16T14:32:06.671298+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:45:26.392197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":167,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:01.788733+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:33:01.788733+0000","last_peered":"2023-10-16T14:33:01.788733+0000","last_clean":"2023-10-16T14:33:01.788733+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:33:01.788733+0000","last_undegraded":"2023-10-16T14:33:01.788733+0000","last_fullsized":"2023-10-16T14:33:01.788733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65228,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":114,"num_read_kb":121,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703074+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:08.703074+0000","last_peered":"2023-10-16T14:32:08.703074+0000","last_clean":"2023-10-16T14:32:08.703074+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:08.703074+0000","last_undegraded":"2023-10-16T14:32:08.703074+0000","last_fullsized":"2023-10-16T14:32:08.703074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674896+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:32:06.674896+0000","last_peered":"2023-10-16T14:32:06.674896+0000","last_clean":"2023-10-16T14:32:06.674896+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:32:06.674896+0000","last_undegraded":"2023-10-16T14:32:06.674896+0000","last_fullsized":"2023-10-16T14:32:06.674896+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:02.790257+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:33:02.790257+0000","last_peered":"2023-10-16T14:33:02.790257+0000","last_clean":"2023-10-16T14:33:02.790257+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:33:02.790257+0000","last_undegraded":"2023-10-16T14:33:02.790257+0000","last_fullsized":"2023-10-16T14:33:02.790257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674369+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:32:06.674369+0000","last_peered":"2023-10-16T14:32:06.674369+0000","last_clean":"2023-10-16T14:32:06.674369+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:32:06.674369+0000","last_undegraded":"2023-10-16T14:32:06.674369+0000","last_fullsized":"2023-10-16T14:32:06.674369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.706275+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:09.706275+0000","last_peered":"2023-10-16T14:32:09.706275+0000","last_clean":"2023-10-16T14:32:09.706275+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:09.706275+0000","last_undegraded":"2023-10-16T14:32:09.706275+0000","last_fullsized":"2023-10-16T14:32:09.706275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.669160+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:32:06.669160+0000","last_peered":"2023-10-16T14:32:06.669160+0000","last_clean":"2023-10-16T14:32:06.669160+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:32:06.669160+0000","last_undegraded":"2023-10-16T14:32:06.669160+0000","last_fullsized":"2023-10-16T14:32:06.669160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676695+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:32:07.676695+0000","last_peered":"2023-10-16T14:32:07.676695+0000","last_clean":"2023-10-16T14:32:07.676695+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:32:07.676695+0000","last_undegraded":"2023-10-16T14:32:07.676695+0000","last_fullsized":"2023-10-16T14:32:07.676695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.681861+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:32:07.681861+0000","last_peered":"2023-10-16T14:32:07.681861+0000","last_clean":"2023-10-16T14:32:07.681861+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:32:07.681861+0000","last_undegraded":"2023-10-16T14:32:07.681861+0000","last_fullsized":"2023-10-16T14:32:07.681861+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":80,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:04.792529+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:33:04.792529+0000","last_peered":"2023-10-16T14:33:04.792529+0000","last_clean":"2023-10-16T14:33:04.792529+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:33:04.792529+0000","last_undegraded":"2023-10-16T14:33:04.792529+0000","last_fullsized":"2023-10-16T14:33:04.792529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.677326+0000","last_change":"2023-10-16T14:31:41.086069+0000","last_active":"2023-10-16T14:32:07.677326+0000","last_peered":"2023-10-16T14:32:07.677326+0000","last_clean":"2023-10-16T14:32:07.677326+0000","last_became_active":"2023-10-16T14:31:41.085519+0000","last_became_peered":"2023-10-16T14:31:41.085519+0000","last_unstale":"2023-10-16T14:32:07.677326+0000","last_undegraded":"2023-10-16T14:32:07.677326+0000","last_fullsized":"2023-10-16T14:32:07.677326+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:55:27.007383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":314,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674219+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:32:06.674219+0000","last_peered":"2023-10-16T14:32:06.674219+0000","last_clean":"2023-10-16T14:32:06.674219+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:32:06.674219+0000","last_undegraded":"2023-10-16T14:32:06.674219+0000","last_fullsized":"2023-10-16T14:32:06.674219+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703372+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:32:08.703372+0000","last_peered":"2023-10-16T14:32:08.703372+0000","last_clean":"2023-10-16T14:32:08.703372+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:32:08.703372+0000","last_undegraded":"2023-10-16T14:32:08.703372+0000","last_fullsized":"2023-10-16T14:32:08.703372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:15.911140+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:15.911140+0000","last_peered":"2023-10-16T14:32:15.911140+0000","last_clean":"2023-10-16T14:32:15.911140+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:15.911140+0000","last_undegraded":"2023-10-16T14:32:15.911140+0000","last_fullsized":"2023-10-16T14:32:15.911140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689723+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:06.689723+0000","last_peered":"2023-10-16T14:32:06.689723+0000","last_clean":"2023-10-16T14:32:06.689723+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:06.689723+0000","last_undegraded":"2023-10-16T14:32:06.689723+0000","last_fullsized":"2023-10-16T14:32:06.689723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920458+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:07.920458+0000","last_peered":"2023-10-16T14:32:07.920458+0000","last_clean":"2023-10-16T14:32:07.920458+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:07.920458+0000","last_undegraded":"2023-10-16T14:32:07.920458+0000","last_fullsized":"2023-10-16T14:32:07.920458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:12.439616+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:12.439616+0000","last_peered":"2023-10-16T14:32:12.439616+0000","last_clean":"2023-10-16T14:32:12.439616+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:12.439616+0000","last_undegraded":"2023-10-16T14:32:12.439616+0000","last_fullsized":"2023-10-16T14:32:12.439616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674580+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:32:06.674580+0000","last_peered":"2023-10-16T14:32:06.674580+0000","last_clean":"2023-10-16T14:32:06.674580+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:32:06.674580+0000","last_undegraded":"2023-10-16T14:32:06.674580+0000","last_fullsized":"2023-10-16T14:32:06.674580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672009+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:06.672009+0000","last_peered":"2023-10-16T14:32:06.672009+0000","last_clean":"2023-10-16T14:32:06.672009+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:06.672009+0000","last_undegraded":"2023-10-16T14:32:06.672009+0000","last_fullsized":"2023-10-16T14:32:06.672009+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.705275+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:08.705275+0000","last_peered":"2023-10-16T14:32:08.705275+0000","last_clean":"2023-10-16T14:32:08.705275+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:08.705275+0000","last_undegraded":"2023-10-16T14:32:08.705275+0000","last_fullsized":"2023-10-16T14:32:08.705275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675372+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:32:06.675372+0000","last_peered":"2023-10-16T14:32:06.675372+0000","last_clean":"2023-10-16T14:32:06.675372+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:32:06.675372+0000","last_undegraded":"2023-10-16T14:32:06.675372+0000","last_fullsized":"2023-10-16T14:32:06.675372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675056+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:32:06.675056+0000","last_peered":"2023-10-16T14:32:06.675056+0000","last_clean":"2023-10-16T14:32:06.675056+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:32:06.675056+0000","last_undegraded":"2023-10-16T14:32:06.675056+0000","last_fullsized":"2023-10-16T14:32:06.675056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676145+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:32:06.676145+0000","last_peered":"2023-10-16T14:32:06.676145+0000","last_clean":"2023-10-16T14:32:06.676145+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:32:06.676145+0000","last_undegraded":"2023-10-16T14:32:06.676145+0000","last_fullsized":"2023-10-16T14:32:06.676145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672274+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:06.672274+0000","last_peered":"2023-10-16T14:32:06.672274+0000","last_clean":"2023-10-16T14:32:06.672274+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:06.672274+0000","last_undegraded":"2023-10-16T14:32:06.672274+0000","last_fullsized":"2023-10-16T14:32:06.672274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":354,"num_read_kb":259,"num_write":164,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":237678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.871,"15min":0.871},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73599999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.75}]},{"osd":9,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.256,"15min":1.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.82499999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78700000000000003,"15min":0.78400000000000003},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95499999999999996,"5min":1.214,"15min":1.214},"last":0.84799999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678050,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1687552,"data_stored":1139512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.70599999999999996}]},{"osd":1,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.64300000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.432}]},{"osd":4,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45000000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61699999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.52300000000000002}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.67100000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999}]},{"osd":11,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57099999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.038,"15min":1.038},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.72299999999999998}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56699999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.502},{"interface":"front","average":{"1min":0.433,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.754,"15min":0.77900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83899999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":1.262,"15min":1.262},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.73999999999999999,"5min":1.54,"15min":1.54},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84899999999999998,"5min":1.246,"15min":1.246},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":1.319,"15min":1.319},"last":0.442},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71599999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.81699999999999995,"15min":0.84199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.83499999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.82099999999999995,"15min":0.84999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.91700000000000004,"15min":0.95499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.876,"15min":0.91500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.452,"15min":1.452},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.96699999999999997,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.90600000000000003,"5min":1.474,"15min":1.474},"last":0.59899999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":1.637,"15min":1.637},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81100000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.747}]},{"osd":10,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.873,"15min":0.87},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.76900000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":486841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.38},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86699999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.056,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":1.02,"15min":1.02},"last":0.624},{"interface":"front","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70499999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.71599999999999997,"15min":0.68899999999999995},"min":{"1min":0.67800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.81999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.751},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86399999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1091892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53800000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.434},{"interface":"front","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73899999999999999,"5min":2.589,"15min":2.589},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49099999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56200000000000006}]},{"osd":6,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85499999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":3.2040000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73399999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":3.2759999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":3.246}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92500000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61899999999999999}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.746}]}]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.621},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.503,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.495}]},{"osd":1,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71799999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.625},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66900000000000004,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.139,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.012,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0489999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.03,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.58299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.232,"15min":1.232},"last":0.70899999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.107},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72599999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0549999999999999,"5min":1.133,"15min":1.133},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85099999999999998,"15min":0.85699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0449999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.84699999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.012,"15min":0.93200000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":11.375999999999999,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90900000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0680000000000001,"5min":1.157,"15min":1.157},"last":0.873}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":547293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.60399999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.621},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.97599999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.41899999999999998},"min":{"1min":0.35599999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.32700000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55000000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":1.181,"15min":1.181},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.624,"15min":0.58299999999999996},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.44400000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96499999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":2.528},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64100000000000001,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Mon Oct 16 14:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.873},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78800000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.995}]},{"osd":9,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84399999999999997,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":1.01},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96399999999999997,"5min":1.038,"15min":1.038},"last":0.92900000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.85999999999999999}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.94499999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.88400000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.65900000000000003,"15min":0.59399999999999997},"min":{"1min":0.56399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":1.202,"15min":1.202},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68000000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88600000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":1.101,"15min":1.101},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":1.006}]},{"osd":6,"last update":"Mon Oct 16 14:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.016,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.04},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95899999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.84599999999999997},"min":{"1min":0.74399999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.073,"5min":1.274,"15min":1.274},"last":1.018},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.069,"5min":1.216,"15min":1.216},"last":1.0640000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.97599999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":1.026},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.0820000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.075}]},{"osd":11,"last update":"Mon Oct 16 14:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.96399999999999997,"15min":0.96199999999999997},"min":{"1min":0.81899999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.095},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.079,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0840000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":559304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53700000000000003,"15min":0.5},"min":{"1min":0.40200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.435},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57899999999999996,"15min":0.50800000000000001},"min":{"1min":0.42999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77400000000000002,"5min":1.022,"15min":1.022},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63100000000000001,"15min":0.57699999999999996},"min":{"1min":0.55400000000000005,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38}]},{"osd":6,"last update":"Mon Oct 16 14:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.622},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.78200000000000003,"15min":0.67600000000000005},"min":{"1min":0.72299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.0190000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99299999999999999,"5min":1.044,"15min":1.044},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84099999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.746}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.42599999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.433},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65600000000000003,"15min":0.60699999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.53400000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83299999999999996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.81200000000000006},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80500000000000005,"5min":1.214,"15min":1.214},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.111,"5min":1.327,"15min":1.327},"last":0.84199999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80200000000000005}]},{"osd":11,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86299999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29199999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.63700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.308,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59099999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.85599999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63500000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81999999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.89000000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1088666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.501},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39700000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95099999999999996,"5min":1.425,"15min":1.425},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.39400000000000002},"min":{"1min":0.379,"5min":0.221,"15min":0.221},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.435}]},{"osd":3,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":1.196,"15min":1.196},"last":0.622},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.69399999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81599999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60499999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.58599999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76600000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.752},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.59899999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":11864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":11598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:33:07.551 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-16T14:33:08.291 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: pgmap v319: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-10-16T14:33:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-10-16T14:33:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2023-10-16T14:33:08.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:08 smithi178 ceph-mon[98495]: osdmap e90: 12 total, 12 up, 12 in 2023-10-16T14:33:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: pgmap v319: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-10-16T14:33:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-10-16T14:33:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2023-10-16T14:33:08.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:08 smithi186 ceph-mon[134892]: osdmap e90: 12 total, 12 up, 12 in 2023-10-16T14:33:08.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: pgmap v319: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:08.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:08.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2023-10-16T14:33:08.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2023-10-16T14:33:08.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2023-10-16T14:33:08.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:08 smithi187 ceph-mon[135297]: osdmap e90: 12 total, 12 up, 12 in 2023-10-16T14:33:09.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:09 smithi178 ceph-mon[98495]: osdmap e91: 12 total, 12 up, 12 in 2023-10-16T14:33:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:09 smithi178 ceph-mon[98495]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:09 smithi186 ceph-mon[134892]: osdmap e91: 12 total, 12 up, 12 in 2023-10-16T14:33:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:09 smithi186 ceph-mon[134892]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:09.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:09 smithi187 ceph-mon[135297]: osdmap e91: 12 total, 12 up, 12 in 2023-10-16T14:33:09.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:09 smithi187 ceph-mon[135297]: pgmap v322: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 -- 172.21.15.178:0/1254716850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60106f90 msgr2=0x7f6a60107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 --2- 172.21.15.178:0/1254716850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60106f90 0x7f6a60107370 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0099c0 tx=0x7f6a5c009cd0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 -- 172.21.15.178:0/1254716850 shutdown_connections 2023-10-16T14:33:09.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 --2- 172.21.15.178:0/1254716850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60101940 0x7f6a6010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 --2- 172.21.15.178:0/1254716850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60106f90 0x7f6a60107370 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 --2- 172.21.15.178:0/1254716850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 0x7f6a60101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 -- 172.21.15.178:0/1254716850 >> 172.21.15.178:0/1254716850 conn(0x7f6a600780a0 msgr2=0x7f6a600784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:09.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 -- 172.21.15.178:0/1254716850 shutdown_connections 2023-10-16T14:33:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.890+0000 7f6a67f50700 1 -- 172.21.15.178:0/1254716850 wait complete. 2023-10-16T14:33:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.891+0000 7f6a67f50700 1 Processor -- start 2023-10-16T14:33:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.891+0000 7f6a67f50700 1 -- start start 2023-10-16T14:33:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.891+0000 7f6a67f50700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 0x7f6a601991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:09.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.891+0000 7f6a67f50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a67f50700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60106f90 0x7f6a6019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a67f50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6019e1a0 con 0x7f6a60101940 2023-10-16T14:33:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a67f50700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6019e310 con 0x7f6a60100f80 2023-10-16T14:33:09.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a67f50700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6019e4d0 con 0x7f6a60106f90 2023-10-16T14:33:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34794/0 (socket says 172.21.15.178:34794) 2023-10-16T14:33:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 -- 172.21.15.178:0/422144902 learned_addr learned my addr 172.21.15.178:0/422144902 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a65cec700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 0x7f6a601991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:09.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a664ed700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60106f90 0x7f6a6019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 -- 172.21.15.178:0/422144902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60106f90 msgr2=0x7f6a6019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60106f90 0x7f6a6019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 -- 172.21.15.178:0/422144902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 msgr2=0x7f6a601991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 0x7f6a601991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:09.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.892+0000 7f6a654eb700 1 -- 172.21.15.178:0/422144902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0095f0 con 0x7f6a60101940 2023-10-16T14:33:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a654eb700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9a0 tx=0x7f6a5000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50009790 con 0x7f6a60101940 2023-10-16T14:33:09.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a50010430 con 0x7f6a60101940 2023-10-16T14:33:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50018680 con 0x7f6a60101940 2023-10-16T14:33:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6019e7b0 con 0x7f6a60101940 2023-10-16T14:33:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.893+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a601a6010 con 0x7f6a60101940 2023-10-16T14:33:09.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.894+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6019e9d0 con 0x7f6a60101940 2023-10-16T14:33:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.896+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f6a500105a0 con 0x7f6a60101940 2023-10-16T14:33:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.897+0000 7f6a56ffd700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 0x7f6a4c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.897+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6a50014070 con 0x7f6a60101940 2023-10-16T14:33:09.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.897+0000 7f6a65cec700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 0x7f6a4c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.898+0000 7f6a65cec700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 0x7f6a4c0a3a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:09.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:09.901+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6a500964e0 con 0x7f6a60101940 2023-10-16T14:33:10.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.098+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a60111050 con 0x7f6a4c0a1590 2023-10-16T14:33:10.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.110+0000 7f6a56ffd700 1 -- 172.21.15.178:0/422144902 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427077 (secure 0 0 0) 0x7f6a60111050 con 0x7f6a4c0a1590 2023-10-16T14:33:10.121 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:10.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 msgr2=0x7f6a4c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 0x7f6a4c0a3a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 msgr2=0x7f6a60199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000e9a0 tx=0x7f6a5000ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 shutdown_connections 2023-10-16T14:33:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6a4c0a1590 0x7f6a4c0a3a50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a60106f90 0x7f6a6019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6a60100f80 0x7f6a601991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 --2- 172.21.15.178:0/422144902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a60101940 0x7f6a60199720 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 >> 172.21.15.178:0/422144902 conn(0x7f6a600780a0 msgr2=0x7f6a60073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 shutdown_connections 2023-10-16T14:33:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:10.116+0000 7f6a67f50700 1 -- 172.21.15.178:0/422144902 wait complete. 2023-10-16T14:33:10.127 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:33:10.295 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:10 smithi178 ceph-mon[98495]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-16T14:33:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:10 smithi186 ceph-mon[134892]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-16T14:33:10.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:10 smithi187 ceph-mon[135297]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2023-10-16T14:33:10.583 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2023-10-16T14:33:08.610363+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":495,"num_read_kb":378,"num_write":347,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337992,"kb_used_data":13780,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603832,"statfs":{"total":1151940427776,"available":1151594323968,"internally_reserved":0,"allocated":14110720,"data_stored":7846285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002080"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677929+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:32:06.677929+0000","last_peered":"2023-10-16T14:32:06.677929+0000","last_clean":"2023-10-16T14:32:06.677929+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.677929+0000","last_undegraded":"2023-10-16T14:32:06.677929+0000","last_fullsized":"2023-10-16T14:32:06.677929+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667257+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:32:06.667257+0000","last_peered":"2023-10-16T14:32:06.667257+0000","last_clean":"2023-10-16T14:32:06.667257+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:32:06.667257+0000","last_undegraded":"2023-10-16T14:32:06.667257+0000","last_fullsized":"2023-10-16T14:32:06.667257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666481+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:32:06.666481+0000","last_peered":"2023-10-16T14:32:06.666481+0000","last_clean":"2023-10-16T14:32:06.666481+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:32:06.666481+0000","last_undegraded":"2023-10-16T14:32:06.666481+0000","last_fullsized":"2023-10-16T14:32:06.666481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671888+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:32:06.671888+0000","last_peered":"2023-10-16T14:32:06.671888+0000","last_clean":"2023-10-16T14:32:06.671888+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:32:06.671888+0000","last_undegraded":"2023-10-16T14:32:06.671888+0000","last_fullsized":"2023-10-16T14:32:06.671888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670704+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:32:06.670704+0000","last_peered":"2023-10-16T14:32:06.670704+0000","last_clean":"2023-10-16T14:32:06.670704+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:32:06.670704+0000","last_undegraded":"2023-10-16T14:32:06.670704+0000","last_fullsized":"2023-10-16T14:32:06.670704+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676353+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:32:07.676353+0000","last_peered":"2023-10-16T14:32:07.676353+0000","last_clean":"2023-10-16T14:32:07.676353+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:32:07.676353+0000","last_undegraded":"2023-10-16T14:32:07.676353+0000","last_fullsized":"2023-10-16T14:32:07.676353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043499+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:07.043499+0000","last_peered":"2023-10-16T14:33:07.043499+0000","last_clean":"2023-10-16T14:33:07.043499+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:07.043499+0000","last_undegraded":"2023-10-16T14:33:07.043499+0000","last_fullsized":"2023-10-16T14:33:07.043499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688119+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:32:06.688119+0000","last_peered":"2023-10-16T14:32:06.688119+0000","last_clean":"2023-10-16T14:32:06.688119+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:32:06.688119+0000","last_undegraded":"2023-10-16T14:32:06.688119+0000","last_fullsized":"2023-10-16T14:32:06.688119+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686611+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:07.686611+0000","last_peered":"2023-10-16T14:32:07.686611+0000","last_clean":"2023-10-16T14:32:07.686611+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:07.686611+0000","last_undegraded":"2023-10-16T14:32:07.686611+0000","last_fullsized":"2023-10-16T14:32:07.686611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676955+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:32:07.676955+0000","last_peered":"2023-10-16T14:32:07.676955+0000","last_clean":"2023-10-16T14:32:07.676955+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:32:07.676955+0000","last_undegraded":"2023-10-16T14:32:07.676955+0000","last_fullsized":"2023-10-16T14:32:07.676955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.045066+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:07.045066+0000","last_peered":"2023-10-16T14:33:07.045066+0000","last_clean":"2023-10-16T14:33:07.045066+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:07.045066+0000","last_undegraded":"2023-10-16T14:33:07.045066+0000","last_fullsized":"2023-10-16T14:33:07.045066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671802+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:32:06.671802+0000","last_peered":"2023-10-16T14:32:06.671802+0000","last_clean":"2023-10-16T14:32:06.671802+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:32:06.671802+0000","last_undegraded":"2023-10-16T14:32:06.671802+0000","last_fullsized":"2023-10-16T14:32:06.671802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672031+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:32:06.672031+0000","last_peered":"2023-10-16T14:32:06.672031+0000","last_clean":"2023-10-16T14:32:06.672031+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:32:06.672031+0000","last_undegraded":"2023-10-16T14:32:06.672031+0000","last_fullsized":"2023-10-16T14:32:06.672031+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666627+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:32:06.666627+0000","last_peered":"2023-10-16T14:32:06.666627+0000","last_clean":"2023-10-16T14:32:06.666627+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:32:06.666627+0000","last_undegraded":"2023-10-16T14:32:06.666627+0000","last_fullsized":"2023-10-16T14:32:06.666627+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666501+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:32:06.666501+0000","last_peered":"2023-10-16T14:32:06.666501+0000","last_clean":"2023-10-16T14:32:06.666501+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:32:06.666501+0000","last_undegraded":"2023-10-16T14:32:06.666501+0000","last_fullsized":"2023-10-16T14:32:06.666501+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686250+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:07.686250+0000","last_peered":"2023-10-16T14:32:07.686250+0000","last_clean":"2023-10-16T14:32:07.686250+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:07.686250+0000","last_undegraded":"2023-10-16T14:32:07.686250+0000","last_fullsized":"2023-10-16T14:32:07.686250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677862+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:32:06.677862+0000","last_peered":"2023-10-16T14:32:06.677862+0000","last_clean":"2023-10-16T14:32:06.677862+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:32:06.677862+0000","last_undegraded":"2023-10-16T14:32:06.677862+0000","last_fullsized":"2023-10-16T14:32:06.677862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672750+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:32:06.672750+0000","last_peered":"2023-10-16T14:32:06.672750+0000","last_clean":"2023-10-16T14:32:06.672750+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:32:06.672750+0000","last_undegraded":"2023-10-16T14:32:06.672750+0000","last_fullsized":"2023-10-16T14:32:06.672750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673670+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:32:06.673670+0000","last_peered":"2023-10-16T14:32:06.673670+0000","last_clean":"2023-10-16T14:32:06.673670+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:32:06.673670+0000","last_undegraded":"2023-10-16T14:32:06.673670+0000","last_fullsized":"2023-10-16T14:32:06.673670+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674053+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:32:06.674053+0000","last_peered":"2023-10-16T14:32:06.674053+0000","last_clean":"2023-10-16T14:32:06.674053+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:32:06.674053+0000","last_undegraded":"2023-10-16T14:32:06.674053+0000","last_fullsized":"2023-10-16T14:32:06.674053+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704302+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:32:08.704302+0000","last_peered":"2023-10-16T14:32:08.704302+0000","last_clean":"2023-10-16T14:32:08.704302+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:32:08.704302+0000","last_undegraded":"2023-10-16T14:32:08.704302+0000","last_fullsized":"2023-10-16T14:32:08.704302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689449+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:32:06.689449+0000","last_peered":"2023-10-16T14:32:06.689449+0000","last_clean":"2023-10-16T14:32:06.689449+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:32:06.689449+0000","last_undegraded":"2023-10-16T14:32:06.689449+0000","last_fullsized":"2023-10-16T14:32:06.689449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670390+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:32:06.670390+0000","last_peered":"2023-10-16T14:32:06.670390+0000","last_clean":"2023-10-16T14:32:06.670390+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:32:06.670390+0000","last_undegraded":"2023-10-16T14:32:06.670390+0000","last_fullsized":"2023-10-16T14:32:06.670390+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.675116+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:32:07.675116+0000","last_peered":"2023-10-16T14:32:07.675116+0000","last_clean":"2023-10-16T14:32:07.675116+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:32:07.675116+0000","last_undegraded":"2023-10-16T14:32:07.675116+0000","last_fullsized":"2023-10-16T14:32:07.675116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667101+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:32:06.667101+0000","last_peered":"2023-10-16T14:32:06.667101+0000","last_clean":"2023-10-16T14:32:06.667101+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:32:06.667101+0000","last_undegraded":"2023-10-16T14:32:06.667101+0000","last_fullsized":"2023-10-16T14:32:06.667101+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670768+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:32:06.670768+0000","last_peered":"2023-10-16T14:32:06.670768+0000","last_clean":"2023-10-16T14:32:06.670768+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:32:06.670768+0000","last_undegraded":"2023-10-16T14:32:06.670768+0000","last_fullsized":"2023-10-16T14:32:06.670768+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702413+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:32:08.702413+0000","last_peered":"2023-10-16T14:32:08.702413+0000","last_clean":"2023-10-16T14:32:08.702413+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:32:08.702413+0000","last_undegraded":"2023-10-16T14:32:08.702413+0000","last_fullsized":"2023-10-16T14:32:08.702413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686779+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:07.686779+0000","last_peered":"2023-10-16T14:32:07.686779+0000","last_clean":"2023-10-16T14:32:07.686779+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:07.686779+0000","last_undegraded":"2023-10-16T14:32:07.686779+0000","last_fullsized":"2023-10-16T14:32:07.686779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670658+0000","last_change":"2023-10-16T14:31:41.088979+0000","last_active":"2023-10-16T14:32:06.670658+0000","last_peered":"2023-10-16T14:32:06.670658+0000","last_clean":"2023-10-16T14:32:06.670658+0000","last_became_active":"2023-10-16T14:31:41.088553+0000","last_became_peered":"2023-10-16T14:31:41.088553+0000","last_unstale":"2023-10-16T14:32:06.670658+0000","last_undegraded":"2023-10-16T14:32:06.670658+0000","last_fullsized":"2023-10-16T14:32:06.670658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:46:03.249219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.692110+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:32:07.692110+0000","last_peered":"2023-10-16T14:32:07.692110+0000","last_clean":"2023-10-16T14:32:07.692110+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:32:07.692110+0000","last_undegraded":"2023-10-16T14:32:07.692110+0000","last_fullsized":"2023-10-16T14:32:07.692110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674794+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:32:06.674794+0000","last_peered":"2023-10-16T14:32:06.674794+0000","last_clean":"2023-10-16T14:32:06.674794+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:32:06.674794+0000","last_undegraded":"2023-10-16T14:32:06.674794+0000","last_fullsized":"2023-10-16T14:32:06.674794+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673526+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:32:06.673526+0000","last_peered":"2023-10-16T14:32:06.673526+0000","last_clean":"2023-10-16T14:32:06.673526+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:32:06.673526+0000","last_undegraded":"2023-10-16T14:32:06.673526+0000","last_fullsized":"2023-10-16T14:32:06.673526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":87,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:03.792526+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:33:03.792526+0000","last_peered":"2023-10-16T14:33:03.792526+0000","last_clean":"2023-10-16T14:33:03.792526+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:33:03.792526+0000","last_undegraded":"2023-10-16T14:33:03.792526+0000","last_fullsized":"2023-10-16T14:33:03.792526+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":59,"num_read_kb":12,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671167+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:32:06.671167+0000","last_peered":"2023-10-16T14:32:06.671167+0000","last_clean":"2023-10-16T14:32:06.671167+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:32:06.671167+0000","last_undegraded":"2023-10-16T14:32:06.671167+0000","last_fullsized":"2023-10-16T14:32:06.671167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673376+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:32:06.673376+0000","last_peered":"2023-10-16T14:32:06.673376+0000","last_clean":"2023-10-16T14:32:06.673376+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:32:06.673376+0000","last_undegraded":"2023-10-16T14:32:06.673376+0000","last_fullsized":"2023-10-16T14:32:06.673376+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702886+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:32:08.702886+0000","last_peered":"2023-10-16T14:32:08.702886+0000","last_clean":"2023-10-16T14:32:08.702886+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:32:08.702886+0000","last_undegraded":"2023-10-16T14:32:08.702886+0000","last_fullsized":"2023-10-16T14:32:08.702886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673963+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:32:06.673963+0000","last_peered":"2023-10-16T14:32:06.673963+0000","last_clean":"2023-10-16T14:32:06.673963+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:32:06.673963+0000","last_undegraded":"2023-10-16T14:32:06.673963+0000","last_fullsized":"2023-10-16T14:32:06.673963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677779+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:32:06.677779+0000","last_peered":"2023-10-16T14:32:06.677779+0000","last_clean":"2023-10-16T14:32:06.677779+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:32:06.677779+0000","last_undegraded":"2023-10-16T14:32:06.677779+0000","last_fullsized":"2023-10-16T14:32:06.677779+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704448+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:32:08.704448+0000","last_peered":"2023-10-16T14:32:08.704448+0000","last_clean":"2023-10-16T14:32:08.704448+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:32:08.704448+0000","last_undegraded":"2023-10-16T14:32:08.704448+0000","last_fullsized":"2023-10-16T14:32:08.704448+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688035+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:32:06.688035+0000","last_peered":"2023-10-16T14:32:06.688035+0000","last_clean":"2023-10-16T14:32:06.688035+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:32:06.688035+0000","last_undegraded":"2023-10-16T14:32:06.688035+0000","last_fullsized":"2023-10-16T14:32:06.688035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667134+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:32:06.667134+0000","last_peered":"2023-10-16T14:32:06.667134+0000","last_clean":"2023-10-16T14:32:06.667134+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:32:06.667134+0000","last_undegraded":"2023-10-16T14:32:06.667134+0000","last_fullsized":"2023-10-16T14:32:06.667134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690287+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:32:06.690287+0000","last_peered":"2023-10-16T14:32:06.690287+0000","last_clean":"2023-10-16T14:32:06.690287+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:32:06.690287+0000","last_undegraded":"2023-10-16T14:32:06.690287+0000","last_fullsized":"2023-10-16T14:32:06.690287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671071+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:32:06.671071+0000","last_peered":"2023-10-16T14:32:06.671071+0000","last_clean":"2023-10-16T14:32:06.671071+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:32:06.671071+0000","last_undegraded":"2023-10-16T14:32:06.671071+0000","last_fullsized":"2023-10-16T14:32:06.671071+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688621+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:32:06.688621+0000","last_peered":"2023-10-16T14:32:06.688621+0000","last_clean":"2023-10-16T14:32:06.688621+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:32:06.688621+0000","last_undegraded":"2023-10-16T14:32:06.688621+0000","last_fullsized":"2023-10-16T14:32:06.688621+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:06.796227+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:33:06.796227+0000","last_peered":"2023-10-16T14:33:06.796227+0000","last_clean":"2023-10-16T14:33:06.796227+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:33:06.796227+0000","last_undegraded":"2023-10-16T14:33:06.796227+0000","last_fullsized":"2023-10-16T14:33:06.796227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686884+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:07.686884+0000","last_peered":"2023-10-16T14:32:07.686884+0000","last_clean":"2023-10-16T14:32:07.686884+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:07.686884+0000","last_undegraded":"2023-10-16T14:32:07.686884+0000","last_fullsized":"2023-10-16T14:32:07.686884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670851+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:32:06.670851+0000","last_peered":"2023-10-16T14:32:06.670851+0000","last_clean":"2023-10-16T14:32:06.670851+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:32:06.670851+0000","last_undegraded":"2023-10-16T14:32:06.670851+0000","last_fullsized":"2023-10-16T14:32:06.670851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044405+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:07.044405+0000","last_peered":"2023-10-16T14:33:07.044405+0000","last_clean":"2023-10-16T14:33:07.044405+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:07.044405+0000","last_undegraded":"2023-10-16T14:33:07.044405+0000","last_fullsized":"2023-10-16T14:33:07.044405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667145+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:32:06.667145+0000","last_peered":"2023-10-16T14:32:06.667145+0000","last_clean":"2023-10-16T14:32:06.667145+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:32:06.667145+0000","last_undegraded":"2023-10-16T14:32:06.667145+0000","last_fullsized":"2023-10-16T14:32:06.667145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671252+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:32:06.671252+0000","last_peered":"2023-10-16T14:32:06.671252+0000","last_clean":"2023-10-16T14:32:06.671252+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:32:06.671252+0000","last_undegraded":"2023-10-16T14:32:06.671252+0000","last_fullsized":"2023-10-16T14:32:06.671252+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679407+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:32:07.679407+0000","last_peered":"2023-10-16T14:32:07.679407+0000","last_clean":"2023-10-16T14:32:07.679407+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:32:07.679407+0000","last_undegraded":"2023-10-16T14:32:07.679407+0000","last_fullsized":"2023-10-16T14:32:07.679407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686866+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:07.686866+0000","last_peered":"2023-10-16T14:32:07.686866+0000","last_clean":"2023-10-16T14:32:07.686866+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:07.686866+0000","last_undegraded":"2023-10-16T14:32:07.686866+0000","last_fullsized":"2023-10-16T14:32:07.686866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2023-10-16T14:31:41.073517+0000","last_change":"2023-10-16T14:33:07.654159+0000","last_active":"2023-10-16T14:31:40.061406+0000","last_peered":"2023-10-16T14:31:40.061406+0000","last_clean":"2023-10-16T14:31:40.061406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T14:31:41.073517+0000","last_undegraded":"2023-10-16T14:31:41.073517+0000","last_fullsized":"2023-10-16T14:31:41.073517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:06.796700+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:33:06.796700+0000","last_peered":"2023-10-16T14:33:06.796700+0000","last_clean":"2023-10-16T14:33:06.796700+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:33:06.796700+0000","last_undegraded":"2023-10-16T14:33:06.796700+0000","last_fullsized":"2023-10-16T14:33:06.796700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65228,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":124,"num_read_kb":126,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672141+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:32:06.672141+0000","last_peered":"2023-10-16T14:32:06.672141+0000","last_clean":"2023-10-16T14:32:06.672141+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:32:06.672141+0000","last_undegraded":"2023-10-16T14:32:06.672141+0000","last_fullsized":"2023-10-16T14:32:06.672141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.675125+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:32:06.675125+0000","last_peered":"2023-10-16T14:32:06.675125+0000","last_clean":"2023-10-16T14:32:06.675125+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:32:06.675125+0000","last_undegraded":"2023-10-16T14:32:06.675125+0000","last_fullsized":"2023-10-16T14:32:06.675125+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.703074+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:32:08.703074+0000","last_peered":"2023-10-16T14:32:08.703074+0000","last_clean":"2023-10-16T14:32:08.703074+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:32:08.703074+0000","last_undegraded":"2023-10-16T14:32:08.703074+0000","last_fullsized":"2023-10-16T14:32:08.703074+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672086+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:32:06.672086+0000","last_peered":"2023-10-16T14:32:06.672086+0000","last_clean":"2023-10-16T14:32:06.672086+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:32:06.672086+0000","last_undegraded":"2023-10-16T14:32:06.672086+0000","last_fullsized":"2023-10-16T14:32:06.672086+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677093+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:32:06.677093+0000","last_peered":"2023-10-16T14:32:06.677093+0000","last_clean":"2023-10-16T14:32:06.677093+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:32:06.677093+0000","last_undegraded":"2023-10-16T14:32:06.677093+0000","last_fullsized":"2023-10-16T14:32:06.677093+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.690537+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:32:06.690537+0000","last_peered":"2023-10-16T14:32:06.690537+0000","last_clean":"2023-10-16T14:32:06.690537+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:32:06.690537+0000","last_undegraded":"2023-10-16T14:32:06.690537+0000","last_fullsized":"2023-10-16T14:32:06.690537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043937+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:07.043937+0000","last_peered":"2023-10-16T14:33:07.043937+0000","last_clean":"2023-10-16T14:33:07.043937+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:07.043937+0000","last_undegraded":"2023-10-16T14:33:07.043937+0000","last_fullsized":"2023-10-16T14:33:07.043937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671497+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:32:06.671497+0000","last_peered":"2023-10-16T14:32:06.671497+0000","last_clean":"2023-10-16T14:32:06.671497+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:32:06.671497+0000","last_undegraded":"2023-10-16T14:32:06.671497+0000","last_fullsized":"2023-10-16T14:32:06.671497+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":79,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:02.790257+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:33:02.790257+0000","last_peered":"2023-10-16T14:33:02.790257+0000","last_clean":"2023-10-16T14:33:02.790257+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:33:02.790257+0000","last_undegraded":"2023-10-16T14:33:02.790257+0000","last_fullsized":"2023-10-16T14:33:02.790257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043458+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:07.043458+0000","last_peered":"2023-10-16T14:33:07.043458+0000","last_clean":"2023-10-16T14:33:07.043458+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:07.043458+0000","last_undegraded":"2023-10-16T14:33:07.043458+0000","last_fullsized":"2023-10-16T14:33:07.043458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673801+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:32:06.673801+0000","last_peered":"2023-10-16T14:32:06.673801+0000","last_clean":"2023-10-16T14:32:06.673801+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:32:06.673801+0000","last_undegraded":"2023-10-16T14:32:06.673801+0000","last_fullsized":"2023-10-16T14:32:06.673801+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.704638+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:32:08.704638+0000","last_peered":"2023-10-16T14:32:08.704638+0000","last_clean":"2023-10-16T14:32:08.704638+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:32:08.704638+0000","last_undegraded":"2023-10-16T14:32:08.704638+0000","last_fullsized":"2023-10-16T14:32:08.704638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672996+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:32:06.672996+0000","last_peered":"2023-10-16T14:32:06.672996+0000","last_clean":"2023-10-16T14:32:06.672996+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:32:06.672996+0000","last_undegraded":"2023-10-16T14:32:06.672996+0000","last_fullsized":"2023-10-16T14:32:06.672996+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672913+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:32:06.672913+0000","last_peered":"2023-10-16T14:32:06.672913+0000","last_clean":"2023-10-16T14:32:06.672913+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:32:06.672913+0000","last_undegraded":"2023-10-16T14:32:06.672913+0000","last_fullsized":"2023-10-16T14:32:06.672913+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.706275+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:32:09.706275+0000","last_peered":"2023-10-16T14:32:09.706275+0000","last_clean":"2023-10-16T14:32:09.706275+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:32:09.706275+0000","last_undegraded":"2023-10-16T14:32:09.706275+0000","last_fullsized":"2023-10-16T14:32:09.706275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677724+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:32:06.677724+0000","last_peered":"2023-10-16T14:32:06.677724+0000","last_clean":"2023-10-16T14:32:06.677724+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:32:06.677724+0000","last_undegraded":"2023-10-16T14:32:06.677724+0000","last_fullsized":"2023-10-16T14:32:06.677724+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671139+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:32:06.671139+0000","last_peered":"2023-10-16T14:32:06.671139+0000","last_clean":"2023-10-16T14:32:06.671139+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:32:06.671139+0000","last_undegraded":"2023-10-16T14:32:06.671139+0000","last_fullsized":"2023-10-16T14:32:06.671139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672304+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:32:06.672304+0000","last_peered":"2023-10-16T14:32:06.672304+0000","last_clean":"2023-10-16T14:32:06.672304+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:32:06.672304+0000","last_undegraded":"2023-10-16T14:32:06.672304+0000","last_fullsized":"2023-10-16T14:32:06.672304+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676812+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:32:07.676812+0000","last_peered":"2023-10-16T14:32:07.676812+0000","last_clean":"2023-10-16T14:32:07.676812+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:32:07.676812+0000","last_undegraded":"2023-10-16T14:32:07.676812+0000","last_fullsized":"2023-10-16T14:32:07.676812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672134+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:32:06.672134+0000","last_peered":"2023-10-16T14:32:06.672134+0000","last_clean":"2023-10-16T14:32:06.672134+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:32:06.672134+0000","last_undegraded":"2023-10-16T14:32:06.672134+0000","last_fullsized":"2023-10-16T14:32:06.672134+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043623+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:07.043623+0000","last_peered":"2023-10-16T14:33:07.043623+0000","last_clean":"2023-10-16T14:33:07.043623+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:07.043623+0000","last_undegraded":"2023-10-16T14:33:07.043623+0000","last_fullsized":"2023-10-16T14:33:07.043623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682383+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:32:07.682383+0000","last_peered":"2023-10-16T14:32:07.682383+0000","last_clean":"2023-10-16T14:32:07.682383+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:32:07.682383+0000","last_undegraded":"2023-10-16T14:32:07.682383+0000","last_fullsized":"2023-10-16T14:32:07.682383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673654+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:32:06.673654+0000","last_peered":"2023-10-16T14:32:06.673654+0000","last_clean":"2023-10-16T14:32:06.673654+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:32:06.673654+0000","last_undegraded":"2023-10-16T14:32:06.673654+0000","last_fullsized":"2023-10-16T14:32:06.673654+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671609+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:32:06.671609+0000","last_peered":"2023-10-16T14:32:06.671609+0000","last_clean":"2023-10-16T14:32:06.671609+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:32:06.671609+0000","last_undegraded":"2023-10-16T14:32:06.671609+0000","last_fullsized":"2023-10-16T14:32:06.671609+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":80,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:04.792529+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:33:04.792529+0000","last_peered":"2023-10-16T14:33:04.792529+0000","last_clean":"2023-10-16T14:33:04.792529+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:33:04.792529+0000","last_undegraded":"2023-10-16T14:33:04.792529+0000","last_fullsized":"2023-10-16T14:33:04.792529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":57,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050385+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050385+0000","last_peered":"2023-10-16T14:33:08.050385+0000","last_clean":"2023-10-16T14:33:08.050385+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050385+0000","last_undegraded":"2023-10-16T14:33:08.050385+0000","last_fullsized":"2023-10-16T14:33:08.050385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687959+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:32:06.687959+0000","last_peered":"2023-10-16T14:32:06.687959+0000","last_clean":"2023-10-16T14:32:06.687959+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:32:06.687959+0000","last_undegraded":"2023-10-16T14:32:06.687959+0000","last_fullsized":"2023-10-16T14:32:06.687959+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:09.705712+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:32:09.705712+0000","last_peered":"2023-10-16T14:32:09.705712+0000","last_clean":"2023-10-16T14:32:09.705712+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:32:09.705712+0000","last_undegraded":"2023-10-16T14:32:09.705712+0000","last_fullsized":"2023-10-16T14:32:09.705712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043337+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:07.043337+0000","last_peered":"2023-10-16T14:33:07.043337+0000","last_clean":"2023-10-16T14:33:07.043337+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:07.043337+0000","last_undegraded":"2023-10-16T14:33:07.043337+0000","last_fullsized":"2023-10-16T14:33:07.043337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.688480+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:32:07.688480+0000","last_peered":"2023-10-16T14:32:07.688480+0000","last_clean":"2023-10-16T14:32:07.688480+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:32:07.688480+0000","last_undegraded":"2023-10-16T14:32:07.688480+0000","last_fullsized":"2023-10-16T14:32:07.688480+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.685575+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:32:07.685575+0000","last_peered":"2023-10-16T14:32:07.685575+0000","last_clean":"2023-10-16T14:32:07.685575+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:32:07.685575+0000","last_undegraded":"2023-10-16T14:32:07.685575+0000","last_fullsized":"2023-10-16T14:32:07.685575+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043414+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:07.043414+0000","last_peered":"2023-10-16T14:33:07.043414+0000","last_clean":"2023-10-16T14:33:07.043414+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:07.043414+0000","last_undegraded":"2023-10-16T14:33:07.043414+0000","last_fullsized":"2023-10-16T14:33:07.043414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676506+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:32:06.676506+0000","last_peered":"2023-10-16T14:32:06.676506+0000","last_clean":"2023-10-16T14:32:06.676506+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:32:06.676506+0000","last_undegraded":"2023-10-16T14:32:06.676506+0000","last_fullsized":"2023-10-16T14:32:06.676506+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682051+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:32:07.682051+0000","last_peered":"2023-10-16T14:32:07.682051+0000","last_clean":"2023-10-16T14:32:07.682051+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:32:07.682051+0000","last_undegraded":"2023-10-16T14:32:07.682051+0000","last_fullsized":"2023-10-16T14:32:07.682051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672271+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:32:06.672271+0000","last_peered":"2023-10-16T14:32:06.672271+0000","last_clean":"2023-10-16T14:32:06.672271+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:32:06.672271+0000","last_undegraded":"2023-10-16T14:32:06.672271+0000","last_fullsized":"2023-10-16T14:32:06.672271+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.687941+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:32:06.687941+0000","last_peered":"2023-10-16T14:32:06.687941+0000","last_clean":"2023-10-16T14:32:06.687941+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:32:06.687941+0000","last_undegraded":"2023-10-16T14:32:06.687941+0000","last_fullsized":"2023-10-16T14:32:06.687941+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:15.911140+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:32:15.911140+0000","last_peered":"2023-10-16T14:32:15.911140+0000","last_clean":"2023-10-16T14:32:15.911140+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:32:15.911140+0000","last_undegraded":"2023-10-16T14:32:15.911140+0000","last_fullsized":"2023-10-16T14:32:15.911140+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689368+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:32:06.689368+0000","last_peered":"2023-10-16T14:32:06.689368+0000","last_clean":"2023-10-16T14:32:06.689368+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:32:06.689368+0000","last_undegraded":"2023-10-16T14:32:06.689368+0000","last_fullsized":"2023-10-16T14:32:06.689368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673946+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:32:06.673946+0000","last_peered":"2023-10-16T14:32:06.673946+0000","last_clean":"2023-10-16T14:32:06.673946+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:32:06.673946+0000","last_undegraded":"2023-10-16T14:32:06.673946+0000","last_fullsized":"2023-10-16T14:32:06.673946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689849+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:32:06.689849+0000","last_peered":"2023-10-16T14:32:06.689849+0000","last_clean":"2023-10-16T14:32:06.689849+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:32:06.689849+0000","last_undegraded":"2023-10-16T14:32:06.689849+0000","last_fullsized":"2023-10-16T14:32:06.689849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.667325+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:32:06.667325+0000","last_peered":"2023-10-16T14:32:06.667325+0000","last_clean":"2023-10-16T14:32:06.667325+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:32:06.667325+0000","last_undegraded":"2023-10-16T14:32:06.667325+0000","last_fullsized":"2023-10-16T14:32:06.667325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672195+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:32:06.672195+0000","last_peered":"2023-10-16T14:32:06.672195+0000","last_clean":"2023-10-16T14:32:06.672195+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:32:06.672195+0000","last_undegraded":"2023-10-16T14:32:06.672195+0000","last_fullsized":"2023-10-16T14:32:06.672195+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686357+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:07.686357+0000","last_peered":"2023-10-16T14:32:07.686357+0000","last_clean":"2023-10-16T14:32:07.686357+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:07.686357+0000","last_undegraded":"2023-10-16T14:32:07.686357+0000","last_fullsized":"2023-10-16T14:32:07.686357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.670616+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:32:06.670616+0000","last_peered":"2023-10-16T14:32:06.670616+0000","last_clean":"2023-10-16T14:32:06.670616+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:32:06.670616+0000","last_undegraded":"2023-10-16T14:32:06.670616+0000","last_fullsized":"2023-10-16T14:32:06.670616+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671667+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:32:06.671667+0000","last_peered":"2023-10-16T14:32:06.671667+0000","last_clean":"2023-10-16T14:32:06.671667+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:32:06.671667+0000","last_undegraded":"2023-10-16T14:32:06.671667+0000","last_fullsized":"2023-10-16T14:32:06.671667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.691771+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:32:06.691771+0000","last_peered":"2023-10-16T14:32:06.691771+0000","last_clean":"2023-10-16T14:32:06.691771+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:32:06.691771+0000","last_undegraded":"2023-10-16T14:32:06.691771+0000","last_fullsized":"2023-10-16T14:32:06.691771+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676318+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:32:06.676318+0000","last_peered":"2023-10-16T14:32:06.676318+0000","last_clean":"2023-10-16T14:32:06.676318+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:32:06.676318+0000","last_undegraded":"2023-10-16T14:32:06.676318+0000","last_fullsized":"2023-10-16T14:32:06.676318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920458+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:32:07.920458+0000","last_peered":"2023-10-16T14:32:07.920458+0000","last_clean":"2023-10-16T14:32:07.920458+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:32:07.920458+0000","last_undegraded":"2023-10-16T14:32:07.920458+0000","last_fullsized":"2023-10-16T14:32:07.920458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:12.439616+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:32:12.439616+0000","last_peered":"2023-10-16T14:32:12.439616+0000","last_clean":"2023-10-16T14:32:12.439616+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:32:12.439616+0000","last_undegraded":"2023-10-16T14:32:12.439616+0000","last_fullsized":"2023-10-16T14:32:12.439616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688389+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:32:06.688389+0000","last_peered":"2023-10-16T14:32:06.688389+0000","last_clean":"2023-10-16T14:32:06.688389+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:32:06.688389+0000","last_undegraded":"2023-10-16T14:32:06.688389+0000","last_fullsized":"2023-10-16T14:32:06.688389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671997+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:32:06.671997+0000","last_peered":"2023-10-16T14:32:06.671997+0000","last_clean":"2023-10-16T14:32:06.671997+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:32:06.671997+0000","last_undegraded":"2023-10-16T14:32:06.671997+0000","last_fullsized":"2023-10-16T14:32:06.671997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.926627+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:32:07.926627+0000","last_peered":"2023-10-16T14:32:07.926627+0000","last_clean":"2023-10-16T14:32:07.926627+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:32:07.926627+0000","last_undegraded":"2023-10-16T14:32:07.926627+0000","last_fullsized":"2023-10-16T14:32:07.926627+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043684+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:07.043684+0000","last_peered":"2023-10-16T14:33:07.043684+0000","last_clean":"2023-10-16T14:33:07.043684+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:07.043684+0000","last_undegraded":"2023-10-16T14:33:07.043684+0000","last_fullsized":"2023-10-16T14:33:07.043684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686854+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:07.686854+0000","last_peered":"2023-10-16T14:32:07.686854+0000","last_clean":"2023-10-16T14:32:07.686854+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:07.686854+0000","last_undegraded":"2023-10-16T14:32:07.686854+0000","last_fullsized":"2023-10-16T14:32:07.686854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.705275+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:32:08.705275+0000","last_peered":"2023-10-16T14:32:08.705275+0000","last_clean":"2023-10-16T14:32:08.705275+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:32:08.705275+0000","last_undegraded":"2023-10-16T14:32:08.705275+0000","last_fullsized":"2023-10-16T14:32:08.705275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044342+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:07.044342+0000","last_peered":"2023-10-16T14:33:07.044342+0000","last_clean":"2023-10-16T14:33:07.044342+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:07.044342+0000","last_undegraded":"2023-10-16T14:33:07.044342+0000","last_fullsized":"2023-10-16T14:33:07.044342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666727+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:32:06.666727+0000","last_peered":"2023-10-16T14:32:06.666727+0000","last_clean":"2023-10-16T14:32:06.666727+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:32:06.666727+0000","last_undegraded":"2023-10-16T14:32:06.666727+0000","last_fullsized":"2023-10-16T14:32:06.666727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.679542+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:32:07.679542+0000","last_peered":"2023-10-16T14:32:07.679542+0000","last_clean":"2023-10-16T14:32:07.679542+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:32:07.679542+0000","last_undegraded":"2023-10-16T14:32:07.679542+0000","last_fullsized":"2023-10-16T14:32:07.679542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044063+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:07.044063+0000","last_peered":"2023-10-16T14:33:07.044063+0000","last_clean":"2023-10-16T14:33:07.044063+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:07.044063+0000","last_undegraded":"2023-10-16T14:33:07.044063+0000","last_fullsized":"2023-10-16T14:33:07.044063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671147+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:32:06.671147+0000","last_peered":"2023-10-16T14:32:06.671147+0000","last_clean":"2023-10-16T14:32:06.671147+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:32:06.671147+0000","last_undegraded":"2023-10-16T14:32:06.671147+0000","last_fullsized":"2023-10-16T14:32:06.671147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673704+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:32:06.673704+0000","last_peered":"2023-10-16T14:32:06.673704+0000","last_clean":"2023-10-16T14:32:06.673704+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:32:06.673704+0000","last_undegraded":"2023-10-16T14:32:06.673704+0000","last_fullsized":"2023-10-16T14:32:06.673704+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044343+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:07.044343+0000","last_peered":"2023-10-16T14:33:07.044343+0000","last_clean":"2023-10-16T14:33:07.044343+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:07.044343+0000","last_undegraded":"2023-10-16T14:33:07.044343+0000","last_fullsized":"2023-10-16T14:33:07.044343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.676421+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:32:06.676421+0000","last_peered":"2023-10-16T14:32:06.676421+0000","last_clean":"2023-10-16T14:32:06.676421+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:32:06.676421+0000","last_undegraded":"2023-10-16T14:32:06.676421+0000","last_fullsized":"2023-10-16T14:32:06.676421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.920726+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:32:07.920726+0000","last_peered":"2023-10-16T14:32:07.920726+0000","last_clean":"2023-10-16T14:32:07.920726+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:32:07.920726+0000","last_undegraded":"2023-10-16T14:32:07.920726+0000","last_fullsized":"2023-10-16T14:32:07.920726+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.673166+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:32:06.673166+0000","last_peered":"2023-10-16T14:32:06.673166+0000","last_clean":"2023-10-16T14:32:06.673166+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:32:06.673166+0000","last_undegraded":"2023-10-16T14:32:06.673166+0000","last_fullsized":"2023-10-16T14:32:06.673166+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.666819+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:32:06.666819+0000","last_peered":"2023-10-16T14:32:06.666819+0000","last_clean":"2023-10-16T14:32:06.666819+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:32:06.666819+0000","last_undegraded":"2023-10-16T14:32:06.666819+0000","last_fullsized":"2023-10-16T14:32:06.666819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.672347+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:32:06.672347+0000","last_peered":"2023-10-16T14:32:06.672347+0000","last_clean":"2023-10-16T14:32:06.672347+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:32:06.672347+0000","last_undegraded":"2023-10-16T14:32:06.672347+0000","last_fullsized":"2023-10-16T14:32:06.672347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677665+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:32:06.677665+0000","last_peered":"2023-10-16T14:32:06.677665+0000","last_clean":"2023-10-16T14:32:06.677665+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:32:06.677665+0000","last_undegraded":"2023-10-16T14:32:06.677665+0000","last_fullsized":"2023-10-16T14:32:06.677665+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.674866+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:32:06.674866+0000","last_peered":"2023-10-16T14:32:06.674866+0000","last_clean":"2023-10-16T14:32:06.674866+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:32:06.674866+0000","last_undegraded":"2023-10-16T14:32:06.674866+0000","last_fullsized":"2023-10-16T14:32:06.674866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.689519+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:32:06.689519+0000","last_peered":"2023-10-16T14:32:06.689519+0000","last_clean":"2023-10-16T14:32:06.689519+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:32:06.689519+0000","last_undegraded":"2023-10-16T14:32:06.689519+0000","last_fullsized":"2023-10-16T14:32:06.689519+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671532+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:32:06.671532+0000","last_peered":"2023-10-16T14:32:06.671532+0000","last_clean":"2023-10-16T14:32:06.671532+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:32:06.671532+0000","last_undegraded":"2023-10-16T14:32:06.671532+0000","last_fullsized":"2023-10-16T14:32:06.671532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.688068+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:32:06.688068+0000","last_peered":"2023-10-16T14:32:06.688068+0000","last_clean":"2023-10-16T14:32:06.688068+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:32:06.688068+0000","last_undegraded":"2023-10-16T14:32:06.688068+0000","last_fullsized":"2023-10-16T14:32:06.688068+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687091+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:07.687091+0000","last_peered":"2023-10-16T14:32:07.687091+0000","last_clean":"2023-10-16T14:32:07.687091+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:07.687091+0000","last_undegraded":"2023-10-16T14:32:07.687091+0000","last_fullsized":"2023-10-16T14:32:07.687091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.671377+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:32:06.671377+0000","last_peered":"2023-10-16T14:32:06.671377+0000","last_clean":"2023-10-16T14:32:06.671377+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:32:06.671377+0000","last_undegraded":"2023-10-16T14:32:06.671377+0000","last_fullsized":"2023-10-16T14:32:06.671377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-10-16T14:32:06.677266+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:32:06.677266+0000","last_peered":"2023-10-16T14:32:06.677266+0000","last_clean":"2023-10-16T14:32:06.677266+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:32:06.677266+0000","last_undegraded":"2023-10-16T14:32:06.677266+0000","last_fullsized":"2023-10-16T14:32:06.677266+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":369,"num_read_kb":269,"num_write":164,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":237678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":487598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.56599999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.871,"15min":0.871},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.68400000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71699999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.73599999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.75}]},{"osd":9,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.256,"15min":1.256},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.82499999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78700000000000003,"15min":0.78400000000000003},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95499999999999996,"5min":1.214,"15min":1.214},"last":0.84799999999999998}]}]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1687552,"data_stored":1139512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58799999999999997}]},{"osd":1,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.375},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.90400000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.92400000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.68100000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57099999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.038,"15min":1.038},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.745}]}]},{"osd":9,"up_from":66,"seq":283467841576,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":491348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56699999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.502},{"interface":"front","average":{"1min":0.433,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.48499999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.754,"15min":0.77900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83899999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":1.262,"15min":1.262},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.73999999999999999,"5min":1.54,"15min":1.54},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84899999999999998,"5min":1.246,"15min":1.246},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":1.319,"15min":1.319},"last":0.442},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71599999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.81699999999999995,"15min":0.84199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.83499999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.82099999999999995,"15min":0.84999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.91700000000000004,"15min":0.95499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.876,"15min":0.91500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.452,"15min":1.452},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.96699999999999997,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.90600000000000003,"5min":1.474,"15min":1.474},"last":0.59899999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":1.637,"15min":1.637},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81100000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.747}]},{"osd":10,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.873,"15min":0.87},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.72899999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.76900000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":486841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.38},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56299999999999994}]},{"osd":3,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86699999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.056,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.76300000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.67200000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":1.02,"15min":1.02},"last":0.624},{"interface":"front","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77600000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70499999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.71599999999999997,"15min":0.68899999999999995},"min":{"1min":0.67800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.81999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.751},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86399999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1101576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53800000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.501},{"interface":"front","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73899999999999999,"5min":2.589,"15min":2.589},"last":0.755},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85499999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65400000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58099999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92500000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.60499999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.621},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.503,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.495}]},{"osd":1,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71799999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.625},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66900000000000004,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.139,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.66800000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.012,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0489999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54300000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.03,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.58299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.232,"15min":1.232},"last":0.70899999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.107},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72599999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0549999999999999,"5min":1.133,"15min":1.133},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85099999999999998,"15min":0.85699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0449999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.84699999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.012,"15min":0.93200000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":11.375999999999999,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90900000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0680000000000001,"5min":1.157,"15min":1.157},"last":0.873}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":556977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872}]},{"osd":3,"last update":"Mon Oct 16 14:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:32:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.41899999999999998},"min":{"1min":0.35599999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.32700000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":1.181,"15min":1.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.624,"15min":0.58299999999999996},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.83899999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.92400000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.96199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77000000000000002,"15min":0.76600000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.85899999999999999,"15min":0.81799999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.78200000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.97599999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.99299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":483212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60399999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.879,"5min":2.7349999999999999,"15min":2.7349999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.61699999999999999,"15min":0.57099999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94399999999999995,"5min":1.141,"15min":1.141},"last":0.88400000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.65900000000000003,"15min":0.59399999999999997},"min":{"1min":0.56399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":1.202,"15min":1.202},"last":0.74399999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.68000000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88600000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72399999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":1.101,"15min":1.101},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":1.006}]},{"osd":6,"last update":"Mon Oct 16 14:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.016,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":1.04},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.95899999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.84599999999999997},"min":{"1min":0.74399999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.073,"5min":1.274,"15min":1.274},"last":1.018},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.069,"5min":1.216,"15min":1.216},"last":1.0640000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.85899999999999999,"15min":0.88500000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.84799999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.96899999999999997,"15min":0.98699999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0529999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.97599999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":1.026},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.0820000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.075}]},{"osd":11,"last update":"Mon Oct 16 14:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.96399999999999997,"15min":0.96199999999999997},"min":{"1min":0.81899999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.095},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.079,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.0840000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1093632,"data_stored":559304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53700000000000003,"15min":0.5},"min":{"1min":0.40200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.435},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57899999999999996,"15min":0.50800000000000001},"min":{"1min":0.42999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77400000000000002,"5min":1.022,"15min":1.022},"last":0.59999999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63100000000000001,"15min":0.57699999999999996},"min":{"1min":0.55400000000000005,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.55400000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.38}]},{"osd":6,"last update":"Mon Oct 16 14:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.622},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.78200000000000003,"15min":0.67600000000000005},"min":{"1min":0.72299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.0190000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.73399999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99299999999999999,"5min":1.044,"15min":1.044},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.67500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.84099999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.746}]}]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":482349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.42599999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.433},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65600000000000003,"15min":0.60699999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.53400000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.72499999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.41699999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66200000000000003,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83299999999999996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.81200000000000006},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80500000000000005,"5min":1.214,"15min":1.214},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.111,"5min":1.327,"15min":1.327},"last":0.84199999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80200000000000005}]},{"osd":11,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.86299999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":486805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29199999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.63700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.308,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59099999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.55200000000000005}]},{"osd":3,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.85599999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.52300000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.621}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.64300000000000002,"15min":0.63200000000000001},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68600000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63500000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70999999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.81999999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.84399999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.89000000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1088666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.501},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39700000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95099999999999996,"5min":1.425,"15min":1.425},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.39400000000000002},"min":{"1min":0.379,"5min":0.221,"15min":0.221},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.435}]},{"osd":3,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63400000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55800000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:32:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.46800000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.94899999999999995,"5min":1.028,"15min":1.028},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.68000000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":1.196,"15min":1.196},"last":0.622},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.69399999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81599999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60499999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.58599999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.76600000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.752},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.59899999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":11864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":11598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:33:10.593 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-16T14:33:11.335 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:11.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:11 smithi178 ceph-mon[98495]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:11.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:11 smithi178 ceph-mon[98495]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:11 smithi186 ceph-mon[134892]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:11 smithi186 ceph-mon[134892]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:11.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:11 smithi187 ceph-mon[135297]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:11.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:11 smithi187 ceph-mon[135297]: pgmap v323: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:12.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 -- 172.21.15.178:0/160694252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 msgr2=0x7f746810ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 --2- 172.21.15.178:0/160694252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f746810ddb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f746000b0d0 tx=0x7f746000b3e0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 -- 172.21.15.178:0/160694252 shutdown_connections 2023-10-16T14:33:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 --2- 172.21.15.178:0/160694252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f746810ddb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 --2- 172.21.15.178:0/160694252 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468106f90 0x7f7468107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 --2- 172.21.15.178:0/160694252 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468100f80 0x7f7468101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 -- 172.21.15.178:0/160694252 >> 172.21.15.178:0/160694252 conn(0x7f74680780a0 msgr2=0x7f74680784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 -- 172.21.15.178:0/160694252 shutdown_connections 2023-10-16T14:33:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.971+0000 7f746c940700 1 -- 172.21.15.178:0/160694252 wait complete. 2023-10-16T14:33:12.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 Processor -- start 2023-10-16T14:33:12.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 -- start start 2023-10-16T14:33:12.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 0x7f74681992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:12.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:12.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468106f90 0x7f746819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:12.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819e280 con 0x7f7468101940 2023-10-16T14:33:12.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819e3f0 con 0x7f7468106f90 2023-10-16T14:33:12.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.972+0000 7f746c940700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819e5b0 con 0x7f7468100f80 2023-10-16T14:33:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f7465d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f7465d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34804/0 (socket says 172.21.15.178:34804) 2023-10-16T14:33:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 0x7f74681992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f7465d9b700 1 -- 172.21.15.178:0/2747325769 learned_addr learned my addr 172.21.15.178:0/2747325769 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:12.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f7466d9d700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468106f90 0x7f746819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 -- 172.21.15.178:0/2747325769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468106f90 msgr2=0x7f746819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468106f90 0x7f746819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 -- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 msgr2=0x7f7468199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:12.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 -- 172.21.15.178:0/2747325769 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009d00 con 0x7f7468100f80 2023-10-16T14:33:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f7465d9b700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.973+0000 7f746659c700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 0x7f74681992c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9f0 tx=0x7f745800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.974+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745800c9c0 con 0x7f7468100f80 2023-10-16T14:33:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.974+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746819e890 con 0x7f7468100f80 2023-10-16T14:33:12.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.974+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7458010430 con 0x7f7468100f80 2023-10-16T14:33:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.974+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458018680 con 0x7f7468100f80 2023-10-16T14:33:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.974+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74681a60f0 con 0x7f7468100f80 2023-10-16T14:33:12.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.975+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746804ebb0 con 0x7f7468100f80 2023-10-16T14:33:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.977+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f7458010860 con 0x7f7468100f80 2023-10-16T14:33:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.978+0000 7f74577fe700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 0x7f74500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:12.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.978+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7458014070 con 0x7f7468100f80 2023-10-16T14:33:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.978+0000 7f7465d9b700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 0x7f74500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.979+0000 7f7465d9b700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 0x7f74500a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f746819a8e0 tx=0x7f745c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:12.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:12.980+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7458096450 con 0x7f7468100f80 2023-10-16T14:33:13.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.181+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7468111050 con 0x7f74500a1510 2023-10-16T14:33:13.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.194+0000 7f74577fe700 1 -- 172.21.15.178:0/2747325769 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427206 (secure 0 0 0) 0x7f7468111050 con 0x7f74500a1510 2023-10-16T14:33:13.205 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:13.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 msgr2=0x7f74500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:13.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 0x7f74500a39d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f746819a8e0 tx=0x7f745c009260 comp rx=0 tx=0).stop 2023-10-16T14:33:13.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 msgr2=0x7f74681992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:13.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 0x7f74681992c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f745800e9f0 tx=0x7f745800ed00 comp rx=0 tx=0).stop 2023-10-16T14:33:13.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 shutdown_connections 2023-10-16T14:33:13.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f74500a1510 0x7f74500a39d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:13.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7468106f90 0x7f746819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:13.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7468100f80 0x7f74681992c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:13.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 --2- 172.21.15.178:0/2747325769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7468101940 0x7f7468199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:13.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 >> 172.21.15.178:0/2747325769 conn(0x7f74680780a0 msgr2=0x7f746810c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:13.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 shutdown_connections 2023-10-16T14:33:13.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:13.200+0000 7f746c940700 1 -- 172.21.15.178:0/2747325769 wait complete. 2023-10-16T14:33:13.211 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:33:13.727 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2023-10-16T14:33:12.611714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":510,"num_read_kb":383,"num_write":347,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":417,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338316,"kb_used_data":14064,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603508,"statfs":{"total":1151940427776,"available":1151593992192,"internally_reserved":0,"allocated":14401536,"data_stored":8016717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":20,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002429"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043646+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:07.043646+0000","last_peered":"2023-10-16T14:33:07.043646+0000","last_clean":"2023-10-16T14:33:07.043646+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043646+0000","last_undegraded":"2023-10-16T14:33:07.043646+0000","last_fullsized":"2023-10-16T14:33:07.043646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176157+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:07.176157+0000","last_peered":"2023-10-16T14:33:07.176157+0000","last_clean":"2023-10-16T14:33:07.176157+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:07.176157+0000","last_undegraded":"2023-10-16T14:33:07.176157+0000","last_fullsized":"2023-10-16T14:33:07.176157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175977+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:07.175977+0000","last_peered":"2023-10-16T14:33:07.175977+0000","last_clean":"2023-10-16T14:33:07.175977+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:07.175977+0000","last_undegraded":"2023-10-16T14:33:07.175977+0000","last_fullsized":"2023-10-16T14:33:07.175977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036974+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:07.036974+0000","last_peered":"2023-10-16T14:33:07.036974+0000","last_clean":"2023-10-16T14:33:07.036974+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:07.036974+0000","last_undegraded":"2023-10-16T14:33:07.036974+0000","last_fullsized":"2023-10-16T14:33:07.036974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039414+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:07.039414+0000","last_peered":"2023-10-16T14:33:07.039414+0000","last_clean":"2023-10-16T14:33:07.039414+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:07.039414+0000","last_undegraded":"2023-10-16T14:33:07.039414+0000","last_fullsized":"2023-10-16T14:33:07.039414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676353+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:32:07.676353+0000","last_peered":"2023-10-16T14:32:07.676353+0000","last_clean":"2023-10-16T14:32:07.676353+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:32:07.676353+0000","last_undegraded":"2023-10-16T14:32:07.676353+0000","last_fullsized":"2023-10-16T14:32:07.676353+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043499+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:07.043499+0000","last_peered":"2023-10-16T14:33:07.043499+0000","last_clean":"2023-10-16T14:33:07.043499+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:07.043499+0000","last_undegraded":"2023-10-16T14:33:07.043499+0000","last_fullsized":"2023-10-16T14:33:07.043499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039389+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:07.039389+0000","last_peered":"2023-10-16T14:33:07.039389+0000","last_clean":"2023-10-16T14:33:07.039389+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:07.039389+0000","last_undegraded":"2023-10-16T14:33:07.039389+0000","last_fullsized":"2023-10-16T14:33:07.039389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686611+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:32:07.686611+0000","last_peered":"2023-10-16T14:32:07.686611+0000","last_clean":"2023-10-16T14:32:07.686611+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:32:07.686611+0000","last_undegraded":"2023-10-16T14:32:07.686611+0000","last_fullsized":"2023-10-16T14:32:07.686611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676955+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:32:07.676955+0000","last_peered":"2023-10-16T14:32:07.676955+0000","last_clean":"2023-10-16T14:32:07.676955+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:32:07.676955+0000","last_undegraded":"2023-10-16T14:32:07.676955+0000","last_fullsized":"2023-10-16T14:32:07.676955+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.045066+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:07.045066+0000","last_peered":"2023-10-16T14:33:07.045066+0000","last_clean":"2023-10-16T14:33:07.045066+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:07.045066+0000","last_undegraded":"2023-10-16T14:33:07.045066+0000","last_fullsized":"2023-10-16T14:33:07.045066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036842+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:07.036842+0000","last_peered":"2023-10-16T14:33:07.036842+0000","last_clean":"2023-10-16T14:33:07.036842+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:07.036842+0000","last_undegraded":"2023-10-16T14:33:07.036842+0000","last_fullsized":"2023-10-16T14:33:07.036842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039435+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:07.039435+0000","last_peered":"2023-10-16T14:33:07.039435+0000","last_clean":"2023-10-16T14:33:07.039435+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:07.039435+0000","last_undegraded":"2023-10-16T14:33:07.039435+0000","last_fullsized":"2023-10-16T14:33:07.039435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175813+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:07.175813+0000","last_peered":"2023-10-16T14:33:07.175813+0000","last_clean":"2023-10-16T14:33:07.175813+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:07.175813+0000","last_undegraded":"2023-10-16T14:33:07.175813+0000","last_fullsized":"2023-10-16T14:33:07.175813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175819+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:07.175819+0000","last_peered":"2023-10-16T14:33:07.175819+0000","last_clean":"2023-10-16T14:33:07.175819+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:07.175819+0000","last_undegraded":"2023-10-16T14:33:07.175819+0000","last_fullsized":"2023-10-16T14:33:07.175819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686250+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:32:07.686250+0000","last_peered":"2023-10-16T14:32:07.686250+0000","last_clean":"2023-10-16T14:32:07.686250+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:32:07.686250+0000","last_undegraded":"2023-10-16T14:32:07.686250+0000","last_fullsized":"2023-10-16T14:32:07.686250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043521+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:07.043521+0000","last_peered":"2023-10-16T14:33:07.043521+0000","last_clean":"2023-10-16T14:33:07.043521+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:07.043521+0000","last_undegraded":"2023-10-16T14:33:07.043521+0000","last_fullsized":"2023-10-16T14:33:07.043521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043605+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:07.043605+0000","last_peered":"2023-10-16T14:33:07.043605+0000","last_clean":"2023-10-16T14:33:07.043605+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:07.043605+0000","last_undegraded":"2023-10-16T14:33:07.043605+0000","last_fullsized":"2023-10-16T14:33:07.043605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032587+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:07.032587+0000","last_peered":"2023-10-16T14:33:07.032587+0000","last_clean":"2023-10-16T14:33:07.032587+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:07.032587+0000","last_undegraded":"2023-10-16T14:33:07.032587+0000","last_fullsized":"2023-10-16T14:33:07.032587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032303+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:07.032303+0000","last_peered":"2023-10-16T14:33:07.032303+0000","last_clean":"2023-10-16T14:33:07.032303+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:07.032303+0000","last_undegraded":"2023-10-16T14:33:07.032303+0000","last_fullsized":"2023-10-16T14:33:07.032303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043652+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:07.043652+0000","last_peered":"2023-10-16T14:33:07.043652+0000","last_clean":"2023-10-16T14:33:07.043652+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:07.043652+0000","last_undegraded":"2023-10-16T14:33:07.043652+0000","last_fullsized":"2023-10-16T14:33:07.043652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040001+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:07.040001+0000","last_peered":"2023-10-16T14:33:07.040001+0000","last_clean":"2023-10-16T14:33:07.040001+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:07.040001+0000","last_undegraded":"2023-10-16T14:33:07.040001+0000","last_fullsized":"2023-10-16T14:33:07.040001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036273+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:07.036273+0000","last_peered":"2023-10-16T14:33:07.036273+0000","last_clean":"2023-10-16T14:33:07.036273+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:07.036273+0000","last_undegraded":"2023-10-16T14:33:07.036273+0000","last_fullsized":"2023-10-16T14:33:07.036273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040091+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:07.040091+0000","last_peered":"2023-10-16T14:33:07.040091+0000","last_clean":"2023-10-16T14:33:07.040091+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:07.040091+0000","last_undegraded":"2023-10-16T14:33:07.040091+0000","last_fullsized":"2023-10-16T14:33:07.040091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175937+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:07.175937+0000","last_peered":"2023-10-16T14:33:07.175937+0000","last_clean":"2023-10-16T14:33:07.175937+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:07.175937+0000","last_undegraded":"2023-10-16T14:33:07.175937+0000","last_fullsized":"2023-10-16T14:33:07.175937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036728+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:07.036728+0000","last_peered":"2023-10-16T14:33:07.036728+0000","last_clean":"2023-10-16T14:33:07.036728+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:07.036728+0000","last_undegraded":"2023-10-16T14:33:07.036728+0000","last_fullsized":"2023-10-16T14:33:07.036728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040455+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:07.040455+0000","last_peered":"2023-10-16T14:33:07.040455+0000","last_clean":"2023-10-16T14:33:07.040455+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:07.040455+0000","last_undegraded":"2023-10-16T14:33:07.040455+0000","last_fullsized":"2023-10-16T14:33:07.040455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686779+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:32:07.686779+0000","last_peered":"2023-10-16T14:32:07.686779+0000","last_clean":"2023-10-16T14:32:07.686779+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:32:07.686779+0000","last_undegraded":"2023-10-16T14:32:07.686779+0000","last_fullsized":"2023-10-16T14:32:07.686779+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.417921+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.417921+0000","last_peered":"2023-10-16T14:33:08.417921+0000","last_clean":"2023-10-16T14:33:08.417921+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.417921+0000","last_undegraded":"2023-10-16T14:33:08.417921+0000","last_fullsized":"2023-10-16T14:33:08.417921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040394+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:07.040394+0000","last_peered":"2023-10-16T14:33:07.040394+0000","last_clean":"2023-10-16T14:33:07.040394+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:07.040394+0000","last_undegraded":"2023-10-16T14:33:07.040394+0000","last_fullsized":"2023-10-16T14:33:07.040394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032469+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:07.032469+0000","last_peered":"2023-10-16T14:33:07.032469+0000","last_clean":"2023-10-16T14:33:07.032469+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:07.032469+0000","last_undegraded":"2023-10-16T14:33:07.032469+0000","last_fullsized":"2023-10-16T14:33:07.032469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":94,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.800167+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:33:08.800167+0000","last_peered":"2023-10-16T14:33:08.800167+0000","last_clean":"2023-10-16T14:33:08.800167+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:33:08.800167+0000","last_undegraded":"2023-10-16T14:33:08.800167+0000","last_fullsized":"2023-10-16T14:33:08.800167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":64,"num_read_kb":12,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039050+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:07.039050+0000","last_peered":"2023-10-16T14:33:07.039050+0000","last_clean":"2023-10-16T14:33:07.039050+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:07.039050+0000","last_undegraded":"2023-10-16T14:33:07.039050+0000","last_fullsized":"2023-10-16T14:33:07.039050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032291+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:07.032291+0000","last_peered":"2023-10-16T14:33:07.032291+0000","last_clean":"2023-10-16T14:33:07.032291+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:07.032291+0000","last_undegraded":"2023-10-16T14:33:07.032291+0000","last_fullsized":"2023-10-16T14:33:07.032291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:08.702886+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:32:08.702886+0000","last_peered":"2023-10-16T14:32:08.702886+0000","last_clean":"2023-10-16T14:32:08.702886+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:32:08.702886+0000","last_undegraded":"2023-10-16T14:32:08.702886+0000","last_fullsized":"2023-10-16T14:32:08.702886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040470+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:07.040470+0000","last_peered":"2023-10-16T14:33:07.040470+0000","last_clean":"2023-10-16T14:33:07.040470+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:07.040470+0000","last_undegraded":"2023-10-16T14:33:07.040470+0000","last_fullsized":"2023-10-16T14:33:07.040470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043543+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:07.043543+0000","last_peered":"2023-10-16T14:33:07.043543+0000","last_clean":"2023-10-16T14:33:07.043543+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:07.043543+0000","last_undegraded":"2023-10-16T14:33:07.043543+0000","last_fullsized":"2023-10-16T14:33:07.043543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039294+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:07.039294+0000","last_peered":"2023-10-16T14:33:07.039294+0000","last_clean":"2023-10-16T14:33:07.039294+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:07.039294+0000","last_undegraded":"2023-10-16T14:33:07.039294+0000","last_fullsized":"2023-10-16T14:33:07.039294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039337+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:07.039337+0000","last_peered":"2023-10-16T14:33:07.039337+0000","last_clean":"2023-10-16T14:33:07.039337+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:07.039337+0000","last_undegraded":"2023-10-16T14:33:07.039337+0000","last_fullsized":"2023-10-16T14:33:07.039337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176168+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:07.176168+0000","last_peered":"2023-10-16T14:33:07.176168+0000","last_clean":"2023-10-16T14:33:07.176168+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:07.176168+0000","last_undegraded":"2023-10-16T14:33:07.176168+0000","last_fullsized":"2023-10-16T14:33:07.176168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039620+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:07.039620+0000","last_peered":"2023-10-16T14:33:07.039620+0000","last_clean":"2023-10-16T14:33:07.039620+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:07.039620+0000","last_undegraded":"2023-10-16T14:33:07.039620+0000","last_fullsized":"2023-10-16T14:33:07.039620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042991+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:07.042991+0000","last_peered":"2023-10-16T14:33:07.042991+0000","last_clean":"2023-10-16T14:33:07.042991+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:07.042991+0000","last_undegraded":"2023-10-16T14:33:07.042991+0000","last_fullsized":"2023-10-16T14:33:07.042991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039499+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:07.039499+0000","last_peered":"2023-10-16T14:33:07.039499+0000","last_clean":"2023-10-16T14:33:07.039499+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:07.039499+0000","last_undegraded":"2023-10-16T14:33:07.039499+0000","last_fullsized":"2023-10-16T14:33:07.039499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":81,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:06.796227+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:33:06.796227+0000","last_peered":"2023-10-16T14:33:06.796227+0000","last_clean":"2023-10-16T14:33:06.796227+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:33:06.796227+0000","last_undegraded":"2023-10-16T14:33:06.796227+0000","last_fullsized":"2023-10-16T14:33:06.796227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686884+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:32:07.686884+0000","last_peered":"2023-10-16T14:32:07.686884+0000","last_clean":"2023-10-16T14:32:07.686884+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:32:07.686884+0000","last_undegraded":"2023-10-16T14:32:07.686884+0000","last_fullsized":"2023-10-16T14:32:07.686884+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043012+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:07.043012+0000","last_peered":"2023-10-16T14:33:07.043012+0000","last_clean":"2023-10-16T14:33:07.043012+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:07.043012+0000","last_undegraded":"2023-10-16T14:33:07.043012+0000","last_fullsized":"2023-10-16T14:33:07.043012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044405+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:07.044405+0000","last_peered":"2023-10-16T14:33:07.044405+0000","last_clean":"2023-10-16T14:33:07.044405+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:07.044405+0000","last_undegraded":"2023-10-16T14:33:07.044405+0000","last_fullsized":"2023-10-16T14:33:07.044405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176030+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:07.176030+0000","last_peered":"2023-10-16T14:33:07.176030+0000","last_clean":"2023-10-16T14:33:07.176030+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:07.176030+0000","last_undegraded":"2023-10-16T14:33:07.176030+0000","last_fullsized":"2023-10-16T14:33:07.176030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036808+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:07.036808+0000","last_peered":"2023-10-16T14:33:07.036808+0000","last_clean":"2023-10-16T14:33:07.036808+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:07.036808+0000","last_undegraded":"2023-10-16T14:33:07.036808+0000","last_fullsized":"2023-10-16T14:33:07.036808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039954+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:07.039954+0000","last_peered":"2023-10-16T14:33:07.039954+0000","last_clean":"2023-10-16T14:33:07.039954+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:07.039954+0000","last_undegraded":"2023-10-16T14:33:07.039954+0000","last_fullsized":"2023-10-16T14:33:07.039954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686866+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:32:07.686866+0000","last_peered":"2023-10-16T14:32:07.686866+0000","last_clean":"2023-10-16T14:32:07.686866+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:32:07.686866+0000","last_undegraded":"2023-10-16T14:32:07.686866+0000","last_fullsized":"2023-10-16T14:32:07.686866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2023-10-16T14:31:41.073517+0000","last_change":"2023-10-16T14:33:07.654159+0000","last_active":"2023-10-16T14:31:40.061406+0000","last_peered":"2023-10-16T14:31:40.061406+0000","last_clean":"2023-10-16T14:31:40.061406+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-16T14:31:41.073517+0000","last_undegraded":"2023-10-16T14:31:41.073517+0000","last_fullsized":"2023-10-16T14:31:41.073517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":178,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:33:06.796700+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:33:06.796700+0000","last_peered":"2023-10-16T14:33:06.796700+0000","last_clean":"2023-10-16T14:33:06.796700+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:33:06.796700+0000","last_undegraded":"2023-10-16T14:33:06.796700+0000","last_fullsized":"2023-10-16T14:33:06.796700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65228,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":124,"num_read_kb":126,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040314+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:07.040314+0000","last_peered":"2023-10-16T14:33:07.040314+0000","last_clean":"2023-10-16T14:33:07.040314+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:07.040314+0000","last_undegraded":"2023-10-16T14:33:07.040314+0000","last_fullsized":"2023-10-16T14:33:07.040314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040938+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:07.040938+0000","last_peered":"2023-10-16T14:33:07.040938+0000","last_clean":"2023-10-16T14:33:07.040938+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:07.040938+0000","last_undegraded":"2023-10-16T14:33:07.040938+0000","last_fullsized":"2023-10-16T14:33:07.040938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032436+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:07.032436+0000","last_peered":"2023-10-16T14:33:07.032436+0000","last_clean":"2023-10-16T14:33:07.032436+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:07.032436+0000","last_undegraded":"2023-10-16T14:33:07.032436+0000","last_fullsized":"2023-10-16T14:33:07.032436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040127+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:07.040127+0000","last_peered":"2023-10-16T14:33:07.040127+0000","last_clean":"2023-10-16T14:33:07.040127+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:07.040127+0000","last_undegraded":"2023-10-16T14:33:07.040127+0000","last_fullsized":"2023-10-16T14:33:07.040127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043392+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:07.043392+0000","last_peered":"2023-10-16T14:33:07.043392+0000","last_clean":"2023-10-16T14:33:07.043392+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:07.043392+0000","last_undegraded":"2023-10-16T14:33:07.043392+0000","last_fullsized":"2023-10-16T14:33:07.043392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040568+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:07.040568+0000","last_peered":"2023-10-16T14:33:07.040568+0000","last_clean":"2023-10-16T14:33:07.040568+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:07.040568+0000","last_undegraded":"2023-10-16T14:33:07.040568+0000","last_fullsized":"2023-10-16T14:33:07.040568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043937+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:07.043937+0000","last_peered":"2023-10-16T14:33:07.043937+0000","last_clean":"2023-10-16T14:33:07.043937+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:07.043937+0000","last_undegraded":"2023-10-16T14:33:07.043937+0000","last_fullsized":"2023-10-16T14:33:07.043937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043473+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:07.043473+0000","last_peered":"2023-10-16T14:33:07.043473+0000","last_clean":"2023-10-16T14:33:07.043473+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:07.043473+0000","last_undegraded":"2023-10-16T14:33:07.043473+0000","last_fullsized":"2023-10-16T14:33:07.043473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":86,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040104+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:33:08.040104+0000","last_peered":"2023-10-16T14:33:08.040104+0000","last_clean":"2023-10-16T14:33:08.040104+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:33:08.040104+0000","last_undegraded":"2023-10-16T14:33:08.040104+0000","last_fullsized":"2023-10-16T14:33:08.040104+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043458+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:07.043458+0000","last_peered":"2023-10-16T14:33:07.043458+0000","last_clean":"2023-10-16T14:33:07.043458+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:07.043458+0000","last_undegraded":"2023-10-16T14:33:07.043458+0000","last_fullsized":"2023-10-16T14:33:07.043458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039890+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:07.039890+0000","last_peered":"2023-10-16T14:33:07.039890+0000","last_clean":"2023-10-16T14:33:07.039890+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:07.039890+0000","last_undegraded":"2023-10-16T14:33:07.039890+0000","last_fullsized":"2023-10-16T14:33:07.039890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038993+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:07.038993+0000","last_peered":"2023-10-16T14:33:07.038993+0000","last_clean":"2023-10-16T14:33:07.038993+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:07.038993+0000","last_undegraded":"2023-10-16T14:33:07.038993+0000","last_fullsized":"2023-10-16T14:33:07.038993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040263+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:07.040263+0000","last_peered":"2023-10-16T14:33:07.040263+0000","last_clean":"2023-10-16T14:33:07.040263+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:07.040263+0000","last_undegraded":"2023-10-16T14:33:07.040263+0000","last_fullsized":"2023-10-16T14:33:07.040263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040066+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:07.040066+0000","last_peered":"2023-10-16T14:33:07.040066+0000","last_clean":"2023-10-16T14:33:07.040066+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:07.040066+0000","last_undegraded":"2023-10-16T14:33:07.040066+0000","last_fullsized":"2023-10-16T14:33:07.040066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043486+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:07.043486+0000","last_peered":"2023-10-16T14:33:07.043486+0000","last_clean":"2023-10-16T14:33:07.043486+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:07.043486+0000","last_undegraded":"2023-10-16T14:33:07.043486+0000","last_fullsized":"2023-10-16T14:33:07.043486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039233+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:07.039233+0000","last_peered":"2023-10-16T14:33:07.039233+0000","last_clean":"2023-10-16T14:33:07.039233+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:07.039233+0000","last_undegraded":"2023-10-16T14:33:07.039233+0000","last_fullsized":"2023-10-16T14:33:07.039233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043347+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:07.043347+0000","last_peered":"2023-10-16T14:33:07.043347+0000","last_clean":"2023-10-16T14:33:07.043347+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:07.043347+0000","last_undegraded":"2023-10-16T14:33:07.043347+0000","last_fullsized":"2023-10-16T14:33:07.043347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.676812+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:32:07.676812+0000","last_peered":"2023-10-16T14:32:07.676812+0000","last_clean":"2023-10-16T14:32:07.676812+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:32:07.676812+0000","last_undegraded":"2023-10-16T14:32:07.676812+0000","last_fullsized":"2023-10-16T14:32:07.676812+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039503+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:07.039503+0000","last_peered":"2023-10-16T14:33:07.039503+0000","last_clean":"2023-10-16T14:33:07.039503+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:07.039503+0000","last_undegraded":"2023-10-16T14:33:07.039503+0000","last_fullsized":"2023-10-16T14:33:07.039503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043623+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:07.043623+0000","last_peered":"2023-10-16T14:33:07.043623+0000","last_clean":"2023-10-16T14:33:07.043623+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:07.043623+0000","last_undegraded":"2023-10-16T14:33:07.043623+0000","last_fullsized":"2023-10-16T14:33:07.043623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.682383+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:32:07.682383+0000","last_peered":"2023-10-16T14:32:07.682383+0000","last_clean":"2023-10-16T14:32:07.682383+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:32:07.682383+0000","last_undegraded":"2023-10-16T14:32:07.682383+0000","last_fullsized":"2023-10-16T14:32:07.682383+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039757+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:07.039757+0000","last_peered":"2023-10-16T14:33:07.039757+0000","last_clean":"2023-10-16T14:33:07.039757+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:07.039757+0000","last_undegraded":"2023-10-16T14:33:07.039757+0000","last_fullsized":"2023-10-16T14:33:07.039757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039716+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:07.039716+0000","last_peered":"2023-10-16T14:33:07.039716+0000","last_clean":"2023-10-16T14:33:07.039716+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:07.039716+0000","last_undegraded":"2023-10-16T14:33:07.039716+0000","last_fullsized":"2023-10-16T14:33:07.039716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":87,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:09.800411+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:33:09.800411+0000","last_peered":"2023-10-16T14:33:09.800411+0000","last_clean":"2023-10-16T14:33:09.800411+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:33:09.800411+0000","last_undegraded":"2023-10-16T14:33:09.800411+0000","last_fullsized":"2023-10-16T14:33:09.800411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":62,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050385+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050385+0000","last_peered":"2023-10-16T14:33:08.050385+0000","last_clean":"2023-10-16T14:33:08.050385+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050385+0000","last_undegraded":"2023-10-16T14:33:08.050385+0000","last_fullsized":"2023-10-16T14:33:08.050385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039720+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:07.039720+0000","last_peered":"2023-10-16T14:33:07.039720+0000","last_clean":"2023-10-16T14:33:07.039720+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:07.039720+0000","last_undegraded":"2023-10-16T14:33:07.039720+0000","last_fullsized":"2023-10-16T14:33:07.039720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036662+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:07.036662+0000","last_peered":"2023-10-16T14:33:07.036662+0000","last_clean":"2023-10-16T14:33:07.036662+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:07.036662+0000","last_undegraded":"2023-10-16T14:33:07.036662+0000","last_fullsized":"2023-10-16T14:33:07.036662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043337+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:07.043337+0000","last_peered":"2023-10-16T14:33:07.043337+0000","last_clean":"2023-10-16T14:33:07.043337+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:07.043337+0000","last_undegraded":"2023-10-16T14:33:07.043337+0000","last_fullsized":"2023-10-16T14:33:07.043337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040242+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:07.040242+0000","last_peered":"2023-10-16T14:33:07.040242+0000","last_clean":"2023-10-16T14:33:07.040242+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:07.040242+0000","last_undegraded":"2023-10-16T14:33:07.040242+0000","last_fullsized":"2023-10-16T14:33:07.040242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039113+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:07.039113+0000","last_peered":"2023-10-16T14:33:07.039113+0000","last_clean":"2023-10-16T14:33:07.039113+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:07.039113+0000","last_undegraded":"2023-10-16T14:33:07.039113+0000","last_fullsized":"2023-10-16T14:33:07.039113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043414+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:07.043414+0000","last_peered":"2023-10-16T14:33:07.043414+0000","last_clean":"2023-10-16T14:33:07.043414+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:07.043414+0000","last_undegraded":"2023-10-16T14:33:07.043414+0000","last_fullsized":"2023-10-16T14:33:07.043414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043206+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:07.043206+0000","last_peered":"2023-10-16T14:33:07.043206+0000","last_clean":"2023-10-16T14:33:07.043206+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:07.043206+0000","last_undegraded":"2023-10-16T14:33:07.043206+0000","last_fullsized":"2023-10-16T14:33:07.043206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040357+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:07.040357+0000","last_peered":"2023-10-16T14:33:07.040357+0000","last_clean":"2023-10-16T14:33:07.040357+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:07.040357+0000","last_undegraded":"2023-10-16T14:33:07.040357+0000","last_fullsized":"2023-10-16T14:33:07.040357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039573+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:07.039573+0000","last_peered":"2023-10-16T14:33:07.039573+0000","last_clean":"2023-10-16T14:33:07.039573+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:07.039573+0000","last_undegraded":"2023-10-16T14:33:07.039573+0000","last_fullsized":"2023-10-16T14:33:07.039573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039295+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:07.039295+0000","last_peered":"2023-10-16T14:33:07.039295+0000","last_clean":"2023-10-16T14:33:07.039295+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:07.039295+0000","last_undegraded":"2023-10-16T14:33:07.039295+0000","last_fullsized":"2023-10-16T14:33:07.039295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043516+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:07.043516+0000","last_peered":"2023-10-16T14:33:07.043516+0000","last_clean":"2023-10-16T14:33:07.043516+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:07.043516+0000","last_undegraded":"2023-10-16T14:33:07.043516+0000","last_fullsized":"2023-10-16T14:33:07.043516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039866+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:07.039866+0000","last_peered":"2023-10-16T14:33:07.039866+0000","last_clean":"2023-10-16T14:33:07.039866+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:07.039866+0000","last_undegraded":"2023-10-16T14:33:07.039866+0000","last_fullsized":"2023-10-16T14:33:07.039866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032019+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:07.032019+0000","last_peered":"2023-10-16T14:33:07.032019+0000","last_clean":"2023-10-16T14:33:07.032019+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:07.032019+0000","last_undegraded":"2023-10-16T14:33:07.032019+0000","last_fullsized":"2023-10-16T14:33:07.032019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.037257+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:07.037257+0000","last_peered":"2023-10-16T14:33:07.037257+0000","last_clean":"2023-10-16T14:33:07.037257+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:07.037257+0000","last_undegraded":"2023-10-16T14:33:07.037257+0000","last_fullsized":"2023-10-16T14:33:07.037257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176243+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:07.176243+0000","last_peered":"2023-10-16T14:33:07.176243+0000","last_clean":"2023-10-16T14:33:07.176243+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:07.176243+0000","last_undegraded":"2023-10-16T14:33:07.176243+0000","last_fullsized":"2023-10-16T14:33:07.176243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036828+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:07.036828+0000","last_peered":"2023-10-16T14:33:07.036828+0000","last_clean":"2023-10-16T14:33:07.036828+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:07.036828+0000","last_undegraded":"2023-10-16T14:33:07.036828+0000","last_fullsized":"2023-10-16T14:33:07.036828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686357+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:32:07.686357+0000","last_peered":"2023-10-16T14:32:07.686357+0000","last_clean":"2023-10-16T14:32:07.686357+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:32:07.686357+0000","last_undegraded":"2023-10-16T14:32:07.686357+0000","last_fullsized":"2023-10-16T14:32:07.686357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038984+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:07.038984+0000","last_peered":"2023-10-16T14:33:07.038984+0000","last_clean":"2023-10-16T14:33:07.038984+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:07.038984+0000","last_undegraded":"2023-10-16T14:33:07.038984+0000","last_fullsized":"2023-10-16T14:33:07.038984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043627+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:07.043627+0000","last_peered":"2023-10-16T14:33:07.043627+0000","last_clean":"2023-10-16T14:33:07.043627+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:07.043627+0000","last_undegraded":"2023-10-16T14:33:07.043627+0000","last_fullsized":"2023-10-16T14:33:07.043627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039296+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:07.039296+0000","last_peered":"2023-10-16T14:33:07.039296+0000","last_clean":"2023-10-16T14:33:07.039296+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:07.039296+0000","last_undegraded":"2023-10-16T14:33:07.039296+0000","last_fullsized":"2023-10-16T14:33:07.039296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043106+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:07.043106+0000","last_peered":"2023-10-16T14:33:07.043106+0000","last_clean":"2023-10-16T14:33:07.043106+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043106+0000","last_undegraded":"2023-10-16T14:33:07.043106+0000","last_fullsized":"2023-10-16T14:33:07.043106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036610+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:07.036610+0000","last_peered":"2023-10-16T14:33:07.036610+0000","last_clean":"2023-10-16T14:33:07.036610+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:07.036610+0000","last_undegraded":"2023-10-16T14:33:07.036610+0000","last_fullsized":"2023-10-16T14:33:07.036610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032216+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:07.032216+0000","last_peered":"2023-10-16T14:33:07.032216+0000","last_clean":"2023-10-16T14:33:07.032216+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:07.032216+0000","last_undegraded":"2023-10-16T14:33:07.032216+0000","last_fullsized":"2023-10-16T14:33:07.032216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036738+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:07.036738+0000","last_peered":"2023-10-16T14:33:07.036738+0000","last_clean":"2023-10-16T14:33:07.036738+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:07.036738+0000","last_undegraded":"2023-10-16T14:33:07.036738+0000","last_fullsized":"2023-10-16T14:33:07.036738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039502+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:07.039502+0000","last_peered":"2023-10-16T14:33:07.039502+0000","last_clean":"2023-10-16T14:33:07.039502+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:07.039502+0000","last_undegraded":"2023-10-16T14:33:07.039502+0000","last_fullsized":"2023-10-16T14:33:07.039502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043684+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:07.043684+0000","last_peered":"2023-10-16T14:33:07.043684+0000","last_clean":"2023-10-16T14:33:07.043684+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:07.043684+0000","last_undegraded":"2023-10-16T14:33:07.043684+0000","last_fullsized":"2023-10-16T14:33:07.043684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.686854+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:32:07.686854+0000","last_peered":"2023-10-16T14:32:07.686854+0000","last_clean":"2023-10-16T14:32:07.686854+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:32:07.686854+0000","last_undegraded":"2023-10-16T14:32:07.686854+0000","last_fullsized":"2023-10-16T14:32:07.686854+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032456+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:07.032456+0000","last_peered":"2023-10-16T14:33:07.032456+0000","last_clean":"2023-10-16T14:33:07.032456+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:07.032456+0000","last_undegraded":"2023-10-16T14:33:07.032456+0000","last_fullsized":"2023-10-16T14:33:07.032456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044342+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:07.044342+0000","last_peered":"2023-10-16T14:33:07.044342+0000","last_clean":"2023-10-16T14:33:07.044342+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:07.044342+0000","last_undegraded":"2023-10-16T14:33:07.044342+0000","last_fullsized":"2023-10-16T14:33:07.044342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175857+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:07.175857+0000","last_peered":"2023-10-16T14:33:07.175857+0000","last_clean":"2023-10-16T14:33:07.175857+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:07.175857+0000","last_undegraded":"2023-10-16T14:33:07.175857+0000","last_fullsized":"2023-10-16T14:33:07.175857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040371+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:07.040371+0000","last_peered":"2023-10-16T14:33:07.040371+0000","last_clean":"2023-10-16T14:33:07.040371+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:07.040371+0000","last_undegraded":"2023-10-16T14:33:07.040371+0000","last_fullsized":"2023-10-16T14:33:07.040371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044063+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:07.044063+0000","last_peered":"2023-10-16T14:33:07.044063+0000","last_clean":"2023-10-16T14:33:07.044063+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:07.044063+0000","last_undegraded":"2023-10-16T14:33:07.044063+0000","last_fullsized":"2023-10-16T14:33:07.044063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039237+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:07.039237+0000","last_peered":"2023-10-16T14:33:07.039237+0000","last_clean":"2023-10-16T14:33:07.039237+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:07.039237+0000","last_undegraded":"2023-10-16T14:33:07.039237+0000","last_fullsized":"2023-10-16T14:33:07.039237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039819+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:07.039819+0000","last_peered":"2023-10-16T14:33:07.039819+0000","last_clean":"2023-10-16T14:33:07.039819+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:07.039819+0000","last_undegraded":"2023-10-16T14:33:07.039819+0000","last_fullsized":"2023-10-16T14:33:07.039819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044343+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:07.044343+0000","last_peered":"2023-10-16T14:33:07.044343+0000","last_clean":"2023-10-16T14:33:07.044343+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:07.044343+0000","last_undegraded":"2023-10-16T14:33:07.044343+0000","last_fullsized":"2023-10-16T14:33:07.044343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043155+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:07.043155+0000","last_peered":"2023-10-16T14:33:07.043155+0000","last_clean":"2023-10-16T14:33:07.043155+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:07.043155+0000","last_undegraded":"2023-10-16T14:33:07.043155+0000","last_fullsized":"2023-10-16T14:33:07.043155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032429+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:07.032429+0000","last_peered":"2023-10-16T14:33:07.032429+0000","last_clean":"2023-10-16T14:33:07.032429+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:07.032429+0000","last_undegraded":"2023-10-16T14:33:07.032429+0000","last_fullsized":"2023-10-16T14:33:07.032429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032099+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:07.032099+0000","last_peered":"2023-10-16T14:33:07.032099+0000","last_clean":"2023-10-16T14:33:07.032099+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:07.032099+0000","last_undegraded":"2023-10-16T14:33:07.032099+0000","last_fullsized":"2023-10-16T14:33:07.032099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176006+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:07.176006+0000","last_peered":"2023-10-16T14:33:07.176006+0000","last_clean":"2023-10-16T14:33:07.176006+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:07.176006+0000","last_undegraded":"2023-10-16T14:33:07.176006+0000","last_fullsized":"2023-10-16T14:33:07.176006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039610+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:07.039610+0000","last_peered":"2023-10-16T14:33:07.039610+0000","last_clean":"2023-10-16T14:33:07.039610+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:07.039610+0000","last_undegraded":"2023-10-16T14:33:07.039610+0000","last_fullsized":"2023-10-16T14:33:07.039610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043494+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:07.043494+0000","last_peered":"2023-10-16T14:33:07.043494+0000","last_clean":"2023-10-16T14:33:07.043494+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:07.043494+0000","last_undegraded":"2023-10-16T14:33:07.043494+0000","last_fullsized":"2023-10-16T14:33:07.043494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032712+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:07.032712+0000","last_peered":"2023-10-16T14:33:07.032712+0000","last_clean":"2023-10-16T14:33:07.032712+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:07.032712+0000","last_undegraded":"2023-10-16T14:33:07.032712+0000","last_fullsized":"2023-10-16T14:33:07.032712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040056+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:07.040056+0000","last_peered":"2023-10-16T14:33:07.040056+0000","last_clean":"2023-10-16T14:33:07.040056+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:07.040056+0000","last_undegraded":"2023-10-16T14:33:07.040056+0000","last_fullsized":"2023-10-16T14:33:07.040056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039696+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:07.039696+0000","last_peered":"2023-10-16T14:33:07.039696+0000","last_clean":"2023-10-16T14:33:07.039696+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:07.039696+0000","last_undegraded":"2023-10-16T14:33:07.039696+0000","last_fullsized":"2023-10-16T14:33:07.039696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039393+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:07.039393+0000","last_peered":"2023-10-16T14:33:07.039393+0000","last_clean":"2023-10-16T14:33:07.039393+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:07.039393+0000","last_undegraded":"2023-10-16T14:33:07.039393+0000","last_fullsized":"2023-10-16T14:33:07.039393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-10-16T14:32:07.687091+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:32:07.687091+0000","last_peered":"2023-10-16T14:32:07.687091+0000","last_clean":"2023-10-16T14:32:07.687091+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:32:07.687091+0000","last_undegraded":"2023-10-16T14:32:07.687091+0000","last_fullsized":"2023-10-16T14:32:07.687091+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039346+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:07.039346+0000","last_peered":"2023-10-16T14:33:07.039346+0000","last_clean":"2023-10-16T14:33:07.039346+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:07.039346+0000","last_undegraded":"2023-10-16T14:33:07.039346+0000","last_fullsized":"2023-10-16T14:33:07.039346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043752+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:07.043752+0000","last_peered":"2023-10-16T14:33:07.043752+0000","last_clean":"2023-10-16T14:33:07.043752+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:07.043752+0000","last_undegraded":"2023-10-16T14:33:07.043752+0000","last_fullsized":"2023-10-16T14:33:07.043752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":384,"num_read_kb":274,"num_write":164,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":237678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449120,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.872}]},{"osd":1,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.627},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.871,"15min":0.871},"last":0.47499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.495},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72599999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.83899999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.70399999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.256,"15min":1.256},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.91200000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78700000000000003,"15min":0.78400000000000003},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95499999999999996,"5min":1.214,"15min":1.214},"last":0.79800000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28664,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716488,"statfs":{"total":95995035648,"available":95965683712,"internally_reserved":0,"allocated":1687552,"data_stored":1139512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58799999999999997}]},{"osd":1,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.375},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42499999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.90400000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.54600000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.92400000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.68100000000000005}]},{"osd":9,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.93999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57099999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.038,"15min":1.038},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.745}]}]},{"osd":9,"up_from":66,"seq":283467841577,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.56200000000000006,"15min":0.58599999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.56699999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.433,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60899999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.34499999999999997}]},{"osd":1,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.754,"15min":0.77900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.83899999999999997,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.79400000000000004,"15min":0.82099999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":1.262,"15min":1.262},"last":0.41299999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.81999999999999995,"15min":0.85199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.73999999999999999,"5min":1.54,"15min":1.54},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84899999999999998,"5min":1.246,"15min":1.246},"last":0.69699999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.92700000000000005,"5min":1.319,"15min":1.319},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.753,"15min":0.78500000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.71599999999999997,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.45800000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.81699999999999995,"15min":0.84199999999999997},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.86399999999999999,"5min":1.345,"15min":1.345},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.81000000000000005,"15min":0.83499999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.82099999999999995,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.504}]},{"osd":5,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.82099999999999995,"15min":0.84999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.91700000000000004,"15min":0.95499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.82799999999999996,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.876,"15min":0.91500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.81399999999999995,"5min":1.452,"15min":1.452},"last":0.442},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.96699999999999997,"15min":1.002},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.90600000000000003,"5min":1.474,"15min":1.474},"last":0.48999999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.996,"5min":1.637,"15min":1.637},"last":0.748},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.89300000000000002,"15min":0.93000000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.81100000000000005,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.66100000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.873,"15min":0.87},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.54300000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.72799999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41399999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65100000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.374},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.437}]},{"osd":3,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86699999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.46500000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.875},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.056,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.71599999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":1.02,"15min":1.02},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.68999999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.77200000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.746,"15min":0.73799999999999999},"min":{"1min":0.52700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.02,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.052,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74399999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.71599999999999997,"15min":0.68899999999999995},"min":{"1min":0.67800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.78300000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81200000000000006}]}]},{"osd":7,"up_from":51,"seq":219043332145,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1617920,"data_stored":1101576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53800000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36799999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.501},{"interface":"front","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54400000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73899999999999999,"5min":2.589,"15min":2.589},"last":0.755},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64300000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.45200000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61899999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85499999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65400000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.58099999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:06 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.82199999999999995,"15min":0.83399999999999996},"min":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.95299999999999996,"5min":2.6230000000000002,"15min":2.6230000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.91300000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92500000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68999999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.60499999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.503,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68000000000000005}]},{"osd":1,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71799999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.625},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66900000000000004,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.139,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.012,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0489999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.03,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.58299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.232,"15min":1.232},"last":0.73399999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.107},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.746}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0549999999999999,"5min":1.133,"15min":1.133},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85099999999999998,"15min":0.85699999999999998},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.0449999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.80600000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.012,"15min":0.93200000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":11.375999999999999,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90900000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0680000000000001,"5min":1.157,"15min":1.157},"last":0.84899999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.874}]}]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1073152,"data_stored":556977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.56100000000000005,"15min":0.51800000000000002},"min":{"1min":0.38300000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.79800000000000004,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57299999999999995,"15min":0.52700000000000002},"min":{"1min":0.44400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.72599999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.872}]},{"osd":3,"last update":"Mon Oct 16 14:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:32:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.41899999999999998},"min":{"1min":0.35599999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.443,"15min":0.42699999999999999},"min":{"1min":0.32700000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61499999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":1.181,"15min":1.181},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.624,"15min":0.58299999999999996},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.83899999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.81999999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.92400000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.96199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77000000000000002,"15min":0.76600000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.85899999999999999,"15min":0.81799999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.78200000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.97599999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.99299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0249999999999999,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.66000000000000003,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94899999999999995,"5min":1.026,"15min":1.141},"last":1.069}]},{"osd":2,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.65900000000000003,"15min":0.59399999999999997},"min":{"1min":0.56399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":1.202,"15min":1.202},"last":0.82799999999999996}]},{"osd":3,"last update":"Mon Oct 16 14:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.97199999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.873},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88600000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.96099999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":1.101,"15min":1.101},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.64500000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.016,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.03}]},{"osd":7,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.90500000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.84599999999999997},"min":{"1min":0.74399999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.073,"5min":1.274,"15min":1.274},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.069,"5min":1.216,"15min":1.216},"last":0.84299999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.099,"5min":0.98899999999999999,"15min":0.99399999999999999},"min":{"1min":0.67800000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.9860000000000002,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":0.88600000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.0820000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.0109999999999999}]},{"osd":11,"last update":"Mon Oct 16 14:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.96399999999999997,"15min":0.96199999999999997},"min":{"1min":0.81899999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.079,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.71699999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":578696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53700000000000003,"15min":0.5},"min":{"1min":0.40200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":1.0640000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57899999999999996,"15min":0.50800000000000001},"min":{"1min":0.42999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77400000000000002,"5min":1.022,"15min":1.022},"last":0.81499999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63100000000000001,"15min":0.57699999999999996},"min":{"1min":0.55400000000000005,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.78200000000000003,"15min":0.67600000000000005},"min":{"1min":0.72299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.0190000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":1.1180000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.0529999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99299999999999999,"5min":1.044,"15min":1.044},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88200000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.006},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.0229999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.075}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.97799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.42599999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65600000000000003,"15min":0.60699999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.53400000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.85699999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.746},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.80400000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.77300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.503}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83299999999999996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.70399999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.79300000000000004,"15min":0.81200000000000006},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.80500000000000005,"5min":1.214,"15min":1.214},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.111,"5min":1.327,"15min":1.327},"last":0.72699999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79000000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.89000000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29199999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.63700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.308,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59099999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.45300000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78900000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.85599999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.65500000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.54400000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.82199999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.872},{"interface":"front","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80900000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.85799999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1108058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57199999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95099999999999996,"5min":1.425,"15min":1.425},"last":0.69499999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.39400000000000002},"min":{"1min":0.379,"5min":0.221,"15min":0.221},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.84599999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90900000000000003,"5min":1.028,"15min":1.028},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.55400000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":1.046,"15min":1.046},"last":0.64400000000000002}]},{"osd":7,"last update":"Mon Oct 16 14:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":1.196,"15min":1.196},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.86299999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81599999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:32:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68700000000000006,"15min":0.67800000000000005},"min":{"1min":0.58599999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.55900000000000005,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.629}]},{"osd":10,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.59899999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.90000000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":11864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":11598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:33:13.744 INFO:tasks.cephadm.ceph_manager.ceph:PG 4.12 is not active+clean 2023-10-16T14:33:13.745 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 90, 'state': 'peering', 'last_fresh': '2023-10-16T14:31:41.073517+0000', 'last_change': '2023-10-16T14:33:07.654159+0000', 'last_active': '2023-10-16T14:31:40.061406+0000', 'last_peered': '2023-10-16T14:31:40.061406+0000', 'last_clean': '2023-10-16T14:31:40.061406+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-16T14:31:41.073517+0000', 'last_undegraded': '2023-10-16T14:31:41.073517+0000', 'last_fullsized': '2023-10-16T14:31:41.073517+0000', 'mapping_epoch': 90, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 85, 'last_epoch_clean': 86, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-16T14:31:40.061406+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-16T14:31:40.061406+0000', 'last_clean_scrub_stamp': '2023-10-16T14:31:40.061406+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [10, 8, 11, 5], 'acting': [10, 8, 11, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 10, 'acting_primary': 10, 'purged_snaps': []} 2023-10-16T14:33:13.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph status --format=json 2023-10-16T14:33:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:13 smithi187 ceph-mon[135297]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:14.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:13 smithi178 ceph-mon[98495]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:13 smithi186 ceph-mon[134892]: pgmap v324: 129 pgs: 1 peering, 128 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:14.496 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:14.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:14 smithi187 ceph-mon[135297]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:14.954 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:14 smithi178 ceph-mon[98495]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:14 smithi186 ceph-mon[134892]: from='client.25066 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:15 smithi178 ceph-mon[98495]: pgmap v325: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:15 smithi178 ceph-mon[98495]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-16T14:33:15.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:15 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:33:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:15 smithi187 ceph-mon[135297]: pgmap v325: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:15 smithi187 ceph-mon[135297]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-16T14:33:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:15 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:33:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:15 smithi186 ceph-mon[134892]: pgmap v325: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 15 B/s, 0 objects/s recovering 2023-10-16T14:33:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:15 smithi186 ceph-mon[134892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2023-10-16T14:33:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:15 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:33:16.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 -- 172.21.15.178:0/305643229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 msgr2=0x7fabc0101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 --2- 172.21.15.178:0/305643229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc0101400 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabb00099c0 tx=0x7fabb0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 -- 172.21.15.178:0/305643229 shutdown_connections 2023-10-16T14:33:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 --2- 172.21.15.178:0/305643229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 0x7fabc010ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 --2- 172.21.15.178:0/305643229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 0x7fabc0107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 --2- 172.21.15.178:0/305643229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc0101400 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 -- 172.21.15.178:0/305643229 >> 172.21.15.178:0/305643229 conn(0x7fabc00780a0 msgr2=0x7fabc00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 -- 172.21.15.178:0/305643229 shutdown_connections 2023-10-16T14:33:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.178+0000 7fabc725b700 1 -- 172.21.15.178:0/305643229 wait complete. 2023-10-16T14:33:16.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.179+0000 7fabc725b700 1 Processor -- start 2023-10-16T14:33:16.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.179+0000 7fabc725b700 1 -- start start 2023-10-16T14:33:16.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:16.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 0x7fabc01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:16.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 0x7fabc019db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc019e170 con 0x7fabc0100f80 2023-10-16T14:33:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc019e2e0 con 0x7fabc0106f90 2023-10-16T14:33:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc725b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc019e4a0 con 0x7fabc0101940 2023-10-16T14:33:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc4ff7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc4ff7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:34814/0 (socket says 172.21.15.178:34814) 2023-10-16T14:33:16.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc4ff7700 1 -- 172.21.15.178:0/180880022 learned_addr learned my addr 172.21.15.178:0/180880022 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:16.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabb7fff700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 0x7fabc01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:16.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.180+0000 7fabc57f8700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 0x7fabc019db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:16.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 -- 172.21.15.178:0/180880022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 msgr2=0x7fabc01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:16.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 0x7fabc01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 -- 172.21.15.178:0/180880022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 msgr2=0x7fabc019db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:16.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 0x7fabc019db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 -- 172.21.15.178:0/180880022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb00095f0 con 0x7fabc0100f80 2023-10-16T14:33:16.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc4ff7700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00e9a0 tx=0x7fabbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:16.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc009790 con 0x7fabc0100f80 2023-10-16T14:33:16.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabbc010430 con 0x7fabc0100f80 2023-10-16T14:33:16.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc018680 con 0x7fabc0100f80 2023-10-16T14:33:16.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc019e780 con 0x7fabc0100f80 2023-10-16T14:33:16.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.181+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc01a5fe0 con 0x7fabc0100f80 2023-10-16T14:33:16.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.182+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc004ebb0 con 0x7fabc0100f80 2023-10-16T14:33:16.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.184+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fabbc0105a0 con 0x7fabc0100f80 2023-10-16T14:33:16.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.186+0000 7fabb5ffb700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 0x7faba80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:16.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.186+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fabbc014070 con 0x7fabc0100f80 2023-10-16T14:33:16.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.186+0000 7fabb7fff700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 0x7faba80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:16.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.187+0000 7fabb7fff700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 0x7faba80a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009990 tx=0x7fabb0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:16.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.190+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fabbc0964b0 con 0x7fabc0100f80 2023-10-16T14:33:16.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.468+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fabc019a510 con 0x7fabc0100f80 2023-10-16T14:33:16.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.469+0000 7fabb5ffb700 1 -- 172.21.15.178:0/180880022 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1798 (secure 0 0 0) 0x7fabbc099ad0 con 0x7fabc0100f80 2023-10-16T14:33:16.473 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:16.473 INFO:teuthology.orchestra.run.smithi178.stdout:{"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":433,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1697466648,"num_in_osds":12,"osd_in_since":1697466630,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":669840,"bytes_used":346513408,"bytes_avail":1151593914368,"bytes_total":1151940427776,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":15,"recovering_keys_per_sec":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"read_bytes_sec":3198,"read_op_per_sec":5,"write_op_per_sec":0},"fsmap":{"epoch":26,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24515},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24520},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24532},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24548},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24550},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24593}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.178:8443/"}},"servicemap":{"epoch":27,"modified":"2023-10-16T14:33:14.612689+0000","services":{"mds":{"daemons":{"summary":"","f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-16T14:33:16.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 msgr2=0x7faba80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:16.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 0x7faba80a3a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009990 tx=0x7fabb0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 msgr2=0x7fabc01991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:16.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fabbc00e9a0 tx=0x7fabbc00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 shutdown_connections 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faba80a1590 0x7faba80a3a50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fabc0106f90 0x7fabc019db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fabc0100f80 0x7fabc01991b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 --2- 172.21.15.178:0/180880022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabc0101940 0x7fabc01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 >> 172.21.15.178:0/180880022 conn(0x7fabc00780a0 msgr2=0x7fabc010c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 shutdown_connections 2023-10-16T14:33:16.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:16.471+0000 7fabc725b700 1 -- 172.21.15.178:0/180880022 wait complete. 2023-10-16T14:33:16.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:16 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/180880022' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:33:16.991 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2023-10-16T14:33:16.992 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-16T14:33:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:16 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/180880022' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:33:17.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:16 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/180880022' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-10-16T14:33:17.729 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:18.046 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:17 smithi178 ceph-mon[98495]: pgmap v326: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2023-10-16T14:33:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:17 smithi186 ceph-mon[134892]: pgmap v326: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2023-10-16T14:33:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:17 smithi187 ceph-mon[135297]: pgmap v326: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 12 B/s, 0 objects/s recovering 2023-10-16T14:33:19.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 -- 172.21.15.178:0/3510220488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 msgr2=0x7f25f010fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:19.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 --2- 172.21.15.178:0/3510220488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f010fd20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f25e800b0a0 tx=0x7f25e800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 -- 172.21.15.178:0/3510220488 shutdown_connections 2023-10-16T14:33:19.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 --2- 172.21.15.178:0/3510220488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f010fd20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 --2- 172.21.15.178:0/3510220488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 --2- 172.21.15.178:0/3510220488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 0x7f25f010d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 -- 172.21.15.178:0/3510220488 >> 172.21.15.178:0/3510220488 conn(0x7f25f00780a0 msgr2=0x7f25f00784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.297+0000 7f25f7a36700 1 -- 172.21.15.178:0/3510220488 shutdown_connections 2023-10-16T14:33:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.298+0000 7f25f7a36700 1 -- 172.21.15.178:0/3510220488 wait complete. 2023-10-16T14:33:19.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.298+0000 7f25f7a36700 1 Processor -- start 2023-10-16T14:33:19.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 -- start start 2023-10-16T14:33:19.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:19.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 0x7f25f01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:19.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:19.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e350 con 0x7f25f010d930 2023-10-16T14:33:19.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e4c0 con 0x7f25f00730e0 2023-10-16T14:33:19.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f7a36700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f019e680 con 0x7f25f0073a00 2023-10-16T14:33:19.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f57d2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:19.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f57d2700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32922/0 (socket says 172.21.15.178:32922) 2023-10-16T14:33:19.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f57d2700 1 -- 172.21.15.178:0/1596832124 learned_addr learned my addr 172.21.15.178:0/1596832124 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:19.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.299+0000 7f25f5fd3700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:19.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f4fd1700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 0x7f25f01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:19.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 -- 172.21.15.178:0/1596832124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 msgr2=0x7f25f01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:19.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 0x7f25f01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 -- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 msgr2=0x7f25f019dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:19.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f019dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 -- 172.21.15.178:0/1596832124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e8009d00 con 0x7f25f00730e0 2023-10-16T14:33:19.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f57d2700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9a0 tx=0x7f25ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:19.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec009790 con 0x7f25f00730e0 2023-10-16T14:33:19.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f019e960 con 0x7f25f00730e0 2023-10-16T14:33:19.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ec010430 con 0x7f25f00730e0 2023-10-16T14:33:19.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.300+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec018680 con 0x7f25f00730e0 2023-10-16T14:33:19.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.301+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f01a61c0 con 0x7f25f00730e0 2023-10-16T14:33:19.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.301+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f004ebb0 con 0x7f25f00730e0 2023-10-16T14:33:19.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.303+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f25ec0105a0 con 0x7f25f00730e0 2023-10-16T14:33:19.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.305+0000 7f25e67fc700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 0x7f25dc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:19.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.305+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25ec014070 con 0x7f25f00730e0 2023-10-16T14:33:19.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.305+0000 7f25f4fd1700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 0x7f25dc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:19.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.306+0000 7f25f4fd1700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 0x7f25dc0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25f019a9b0 tx=0x7f25e000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:19.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.307+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f25ec096770 con 0x7f25f00730e0 2023-10-16T14:33:19.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.501+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25f0113250 con 0x7f25dc0a1510 2023-10-16T14:33:19.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.514+0000 7f25e67fc700 1 -- 172.21.15.178:0/1596832124 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427855 (secure 0 0 0) 0x7f25f0113250 con 0x7f25dc0a1510 2023-10-16T14:33:19.524 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:19.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 msgr2=0x7f25dc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:19.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 0x7f25dc0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25f019a9b0 tx=0x7f25e000b410 comp rx=0 tx=0).stop 2023-10-16T14:33:19.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 msgr2=0x7f25f0199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:19.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f25ec00e9a0 tx=0x7f25ec00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 shutdown_connections 2023-10-16T14:33:19.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25dc0a1510 0x7f25dc0a39d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25f010d930 0x7f25f019dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25f00730e0 0x7f25f0199390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 --2- 172.21.15.178:0/1596832124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25f0073a00 0x7f25f01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:19.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.519+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 >> 172.21.15.178:0/1596832124 conn(0x7f25f00780a0 msgr2=0x7f25f00fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:19.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.520+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 shutdown_connections 2023-10-16T14:33:19.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:19.520+0000 7f25f7a36700 1 -- 172.21.15.178:0/1596832124 wait complete. 2023-10-16T14:33:19.530 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:33:19.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:19 smithi178 ceph-mon[98495]: pgmap v327: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-10-16T14:33:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:19 smithi186 ceph-mon[134892]: pgmap v327: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-10-16T14:33:20.126 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2023-10-16T14:33:18.613476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":555,"num_read_kb":408,"num_write":347,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338392,"kb_used_data":14132,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603432,"statfs":{"total":1151940427776,"available":1151593914368,"internally_reserved":0,"allocated":14471168,"data_stored":8059621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.581990"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043646+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:07.043646+0000","last_peered":"2023-10-16T14:33:07.043646+0000","last_clean":"2023-10-16T14:33:07.043646+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043646+0000","last_undegraded":"2023-10-16T14:33:07.043646+0000","last_fullsized":"2023-10-16T14:33:07.043646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176157+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:07.176157+0000","last_peered":"2023-10-16T14:33:07.176157+0000","last_clean":"2023-10-16T14:33:07.176157+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:07.176157+0000","last_undegraded":"2023-10-16T14:33:07.176157+0000","last_fullsized":"2023-10-16T14:33:07.176157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175977+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:07.175977+0000","last_peered":"2023-10-16T14:33:07.175977+0000","last_clean":"2023-10-16T14:33:07.175977+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:07.175977+0000","last_undegraded":"2023-10-16T14:33:07.175977+0000","last_fullsized":"2023-10-16T14:33:07.175977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036974+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:07.036974+0000","last_peered":"2023-10-16T14:33:07.036974+0000","last_clean":"2023-10-16T14:33:07.036974+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:07.036974+0000","last_undegraded":"2023-10-16T14:33:07.036974+0000","last_fullsized":"2023-10-16T14:33:07.036974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039414+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:07.039414+0000","last_peered":"2023-10-16T14:33:07.039414+0000","last_clean":"2023-10-16T14:33:07.039414+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:07.039414+0000","last_undegraded":"2023-10-16T14:33:07.039414+0000","last_fullsized":"2023-10-16T14:33:07.039414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042411+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:33:07.042411+0000","last_peered":"2023-10-16T14:33:07.042411+0000","last_clean":"2023-10-16T14:33:07.042411+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:33:07.042411+0000","last_undegraded":"2023-10-16T14:33:07.042411+0000","last_fullsized":"2023-10-16T14:33:07.042411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043499+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:07.043499+0000","last_peered":"2023-10-16T14:33:07.043499+0000","last_clean":"2023-10-16T14:33:07.043499+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:07.043499+0000","last_undegraded":"2023-10-16T14:33:07.043499+0000","last_fullsized":"2023-10-16T14:33:07.043499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039389+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:07.039389+0000","last_peered":"2023-10-16T14:33:07.039389+0000","last_clean":"2023-10-16T14:33:07.039389+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:07.039389+0000","last_undegraded":"2023-10-16T14:33:07.039389+0000","last_fullsized":"2023-10-16T14:33:07.039389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175871+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:33:07.175871+0000","last_peered":"2023-10-16T14:33:07.175871+0000","last_clean":"2023-10-16T14:33:07.175871+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:33:07.175871+0000","last_undegraded":"2023-10-16T14:33:07.175871+0000","last_fullsized":"2023-10-16T14:33:07.175871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042938+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:33:07.042938+0000","last_peered":"2023-10-16T14:33:07.042938+0000","last_clean":"2023-10-16T14:33:07.042938+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:33:07.042938+0000","last_undegraded":"2023-10-16T14:33:07.042938+0000","last_fullsized":"2023-10-16T14:33:07.042938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.045066+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:07.045066+0000","last_peered":"2023-10-16T14:33:07.045066+0000","last_clean":"2023-10-16T14:33:07.045066+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:07.045066+0000","last_undegraded":"2023-10-16T14:33:07.045066+0000","last_fullsized":"2023-10-16T14:33:07.045066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036842+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:07.036842+0000","last_peered":"2023-10-16T14:33:07.036842+0000","last_clean":"2023-10-16T14:33:07.036842+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:07.036842+0000","last_undegraded":"2023-10-16T14:33:07.036842+0000","last_fullsized":"2023-10-16T14:33:07.036842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039435+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:07.039435+0000","last_peered":"2023-10-16T14:33:07.039435+0000","last_clean":"2023-10-16T14:33:07.039435+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:07.039435+0000","last_undegraded":"2023-10-16T14:33:07.039435+0000","last_fullsized":"2023-10-16T14:33:07.039435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175813+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:07.175813+0000","last_peered":"2023-10-16T14:33:07.175813+0000","last_clean":"2023-10-16T14:33:07.175813+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:07.175813+0000","last_undegraded":"2023-10-16T14:33:07.175813+0000","last_fullsized":"2023-10-16T14:33:07.175813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175819+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:07.175819+0000","last_peered":"2023-10-16T14:33:07.175819+0000","last_clean":"2023-10-16T14:33:07.175819+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:07.175819+0000","last_undegraded":"2023-10-16T14:33:07.175819+0000","last_fullsized":"2023-10-16T14:33:07.175819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175325+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:33:07.175325+0000","last_peered":"2023-10-16T14:33:07.175325+0000","last_clean":"2023-10-16T14:33:07.175325+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:33:07.175325+0000","last_undegraded":"2023-10-16T14:33:07.175325+0000","last_fullsized":"2023-10-16T14:33:07.175325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043521+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:07.043521+0000","last_peered":"2023-10-16T14:33:07.043521+0000","last_clean":"2023-10-16T14:33:07.043521+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:07.043521+0000","last_undegraded":"2023-10-16T14:33:07.043521+0000","last_fullsized":"2023-10-16T14:33:07.043521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043605+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:07.043605+0000","last_peered":"2023-10-16T14:33:07.043605+0000","last_clean":"2023-10-16T14:33:07.043605+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:07.043605+0000","last_undegraded":"2023-10-16T14:33:07.043605+0000","last_fullsized":"2023-10-16T14:33:07.043605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032587+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:07.032587+0000","last_peered":"2023-10-16T14:33:07.032587+0000","last_clean":"2023-10-16T14:33:07.032587+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:07.032587+0000","last_undegraded":"2023-10-16T14:33:07.032587+0000","last_fullsized":"2023-10-16T14:33:07.032587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032303+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:07.032303+0000","last_peered":"2023-10-16T14:33:07.032303+0000","last_clean":"2023-10-16T14:33:07.032303+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:07.032303+0000","last_undegraded":"2023-10-16T14:33:07.032303+0000","last_fullsized":"2023-10-16T14:33:07.032303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043652+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:07.043652+0000","last_peered":"2023-10-16T14:33:07.043652+0000","last_clean":"2023-10-16T14:33:07.043652+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:07.043652+0000","last_undegraded":"2023-10-16T14:33:07.043652+0000","last_fullsized":"2023-10-16T14:33:07.043652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040001+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:07.040001+0000","last_peered":"2023-10-16T14:33:07.040001+0000","last_clean":"2023-10-16T14:33:07.040001+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:07.040001+0000","last_undegraded":"2023-10-16T14:33:07.040001+0000","last_fullsized":"2023-10-16T14:33:07.040001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036273+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:07.036273+0000","last_peered":"2023-10-16T14:33:07.036273+0000","last_clean":"2023-10-16T14:33:07.036273+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:07.036273+0000","last_undegraded":"2023-10-16T14:33:07.036273+0000","last_fullsized":"2023-10-16T14:33:07.036273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040091+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:07.040091+0000","last_peered":"2023-10-16T14:33:07.040091+0000","last_clean":"2023-10-16T14:33:07.040091+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:07.040091+0000","last_undegraded":"2023-10-16T14:33:07.040091+0000","last_fullsized":"2023-10-16T14:33:07.040091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175937+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:07.175937+0000","last_peered":"2023-10-16T14:33:07.175937+0000","last_clean":"2023-10-16T14:33:07.175937+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:07.175937+0000","last_undegraded":"2023-10-16T14:33:07.175937+0000","last_fullsized":"2023-10-16T14:33:07.175937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036728+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:07.036728+0000","last_peered":"2023-10-16T14:33:07.036728+0000","last_clean":"2023-10-16T14:33:07.036728+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:07.036728+0000","last_undegraded":"2023-10-16T14:33:07.036728+0000","last_fullsized":"2023-10-16T14:33:07.036728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040455+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:07.040455+0000","last_peered":"2023-10-16T14:33:07.040455+0000","last_clean":"2023-10-16T14:33:07.040455+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:07.040455+0000","last_undegraded":"2023-10-16T14:33:07.040455+0000","last_fullsized":"2023-10-16T14:33:07.040455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175668+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:33:07.175668+0000","last_peered":"2023-10-16T14:33:07.175668+0000","last_clean":"2023-10-16T14:33:07.175668+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:33:07.175668+0000","last_undegraded":"2023-10-16T14:33:07.175668+0000","last_fullsized":"2023-10-16T14:33:07.175668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.417921+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.417921+0000","last_peered":"2023-10-16T14:33:08.417921+0000","last_clean":"2023-10-16T14:33:08.417921+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.417921+0000","last_undegraded":"2023-10-16T14:33:08.417921+0000","last_fullsized":"2023-10-16T14:33:08.417921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040394+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:07.040394+0000","last_peered":"2023-10-16T14:33:07.040394+0000","last_clean":"2023-10-16T14:33:07.040394+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:07.040394+0000","last_undegraded":"2023-10-16T14:33:07.040394+0000","last_fullsized":"2023-10-16T14:33:07.040394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032469+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:07.032469+0000","last_peered":"2023-10-16T14:33:07.032469+0000","last_clean":"2023-10-16T14:33:07.032469+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:07.032469+0000","last_undegraded":"2023-10-16T14:33:07.032469+0000","last_fullsized":"2023-10-16T14:33:07.032469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:13.808400+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:33:13.808400+0000","last_peered":"2023-10-16T14:33:13.808400+0000","last_clean":"2023-10-16T14:33:13.808400+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:33:13.808400+0000","last_undegraded":"2023-10-16T14:33:13.808400+0000","last_fullsized":"2023-10-16T14:33:13.808400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":12,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039050+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:07.039050+0000","last_peered":"2023-10-16T14:33:07.039050+0000","last_clean":"2023-10-16T14:33:07.039050+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:07.039050+0000","last_undegraded":"2023-10-16T14:33:07.039050+0000","last_fullsized":"2023-10-16T14:33:07.039050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032291+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:07.032291+0000","last_peered":"2023-10-16T14:33:07.032291+0000","last_clean":"2023-10-16T14:33:07.032291+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:07.032291+0000","last_undegraded":"2023-10-16T14:33:07.032291+0000","last_fullsized":"2023-10-16T14:33:07.032291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042313+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:33:07.042313+0000","last_peered":"2023-10-16T14:33:07.042313+0000","last_clean":"2023-10-16T14:33:07.042313+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:33:07.042313+0000","last_undegraded":"2023-10-16T14:33:07.042313+0000","last_fullsized":"2023-10-16T14:33:07.042313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040470+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:07.040470+0000","last_peered":"2023-10-16T14:33:07.040470+0000","last_clean":"2023-10-16T14:33:07.040470+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:07.040470+0000","last_undegraded":"2023-10-16T14:33:07.040470+0000","last_fullsized":"2023-10-16T14:33:07.040470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043543+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:07.043543+0000","last_peered":"2023-10-16T14:33:07.043543+0000","last_clean":"2023-10-16T14:33:07.043543+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:07.043543+0000","last_undegraded":"2023-10-16T14:33:07.043543+0000","last_fullsized":"2023-10-16T14:33:07.043543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039294+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:07.039294+0000","last_peered":"2023-10-16T14:33:07.039294+0000","last_clean":"2023-10-16T14:33:07.039294+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:07.039294+0000","last_undegraded":"2023-10-16T14:33:07.039294+0000","last_fullsized":"2023-10-16T14:33:07.039294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039337+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:07.039337+0000","last_peered":"2023-10-16T14:33:07.039337+0000","last_clean":"2023-10-16T14:33:07.039337+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:07.039337+0000","last_undegraded":"2023-10-16T14:33:07.039337+0000","last_fullsized":"2023-10-16T14:33:07.039337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176168+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:07.176168+0000","last_peered":"2023-10-16T14:33:07.176168+0000","last_clean":"2023-10-16T14:33:07.176168+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:07.176168+0000","last_undegraded":"2023-10-16T14:33:07.176168+0000","last_fullsized":"2023-10-16T14:33:07.176168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039620+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:07.039620+0000","last_peered":"2023-10-16T14:33:07.039620+0000","last_clean":"2023-10-16T14:33:07.039620+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:07.039620+0000","last_undegraded":"2023-10-16T14:33:07.039620+0000","last_fullsized":"2023-10-16T14:33:07.039620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042991+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:07.042991+0000","last_peered":"2023-10-16T14:33:07.042991+0000","last_clean":"2023-10-16T14:33:07.042991+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:07.042991+0000","last_undegraded":"2023-10-16T14:33:07.042991+0000","last_fullsized":"2023-10-16T14:33:07.042991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039499+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:07.039499+0000","last_peered":"2023-10-16T14:33:07.039499+0000","last_clean":"2023-10-16T14:33:07.039499+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:07.039499+0000","last_undegraded":"2023-10-16T14:33:07.039499+0000","last_fullsized":"2023-10-16T14:33:07.039499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:15.810493+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:33:15.810493+0000","last_peered":"2023-10-16T14:33:15.810493+0000","last_clean":"2023-10-16T14:33:15.810493+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:33:15.810493+0000","last_undegraded":"2023-10-16T14:33:15.810493+0000","last_fullsized":"2023-10-16T14:33:15.810493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175847+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:33:07.175847+0000","last_peered":"2023-10-16T14:33:07.175847+0000","last_clean":"2023-10-16T14:33:07.175847+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:33:07.175847+0000","last_undegraded":"2023-10-16T14:33:07.175847+0000","last_fullsized":"2023-10-16T14:33:07.175847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043012+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:07.043012+0000","last_peered":"2023-10-16T14:33:07.043012+0000","last_clean":"2023-10-16T14:33:07.043012+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:07.043012+0000","last_undegraded":"2023-10-16T14:33:07.043012+0000","last_fullsized":"2023-10-16T14:33:07.043012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044405+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:07.044405+0000","last_peered":"2023-10-16T14:33:07.044405+0000","last_clean":"2023-10-16T14:33:07.044405+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:07.044405+0000","last_undegraded":"2023-10-16T14:33:07.044405+0000","last_fullsized":"2023-10-16T14:33:07.044405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176030+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:07.176030+0000","last_peered":"2023-10-16T14:33:07.176030+0000","last_clean":"2023-10-16T14:33:07.176030+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:07.176030+0000","last_undegraded":"2023-10-16T14:33:07.176030+0000","last_fullsized":"2023-10-16T14:33:07.176030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036808+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:07.036808+0000","last_peered":"2023-10-16T14:33:07.036808+0000","last_clean":"2023-10-16T14:33:07.036808+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:07.036808+0000","last_undegraded":"2023-10-16T14:33:07.036808+0000","last_fullsized":"2023-10-16T14:33:07.036808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039954+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:07.039954+0000","last_peered":"2023-10-16T14:33:07.039954+0000","last_clean":"2023-10-16T14:33:07.039954+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:07.039954+0000","last_undegraded":"2023-10-16T14:33:07.039954+0000","last_fullsized":"2023-10-16T14:33:07.039954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175772+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:33:07.175772+0000","last_peered":"2023-10-16T14:33:07.175772+0000","last_clean":"2023-10-16T14:33:07.175772+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:33:07.175772+0000","last_undegraded":"2023-10-16T14:33:07.175772+0000","last_fullsized":"2023-10-16T14:33:07.175772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.045902+0000","last_change":"2023-10-16T14:33:08.045902+0000","last_active":"2023-10-16T14:33:08.045902+0000","last_peered":"2023-10-16T14:33:08.045902+0000","last_clean":"2023-10-16T14:33:08.045902+0000","last_became_active":"2023-10-16T14:33:08.045481+0000","last_became_peered":"2023-10-16T14:33:08.045481+0000","last_unstale":"2023-10-16T14:33:08.045902+0000","last_undegraded":"2023-10-16T14:33:08.045902+0000","last_fullsized":"2023-10-16T14:33:08.045902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:18.091442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":199,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:16.811912+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:33:16.811912+0000","last_peered":"2023-10-16T14:33:16.811912+0000","last_clean":"2023-10-16T14:33:16.811912+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:33:16.811912+0000","last_undegraded":"2023-10-16T14:33:16.811912+0000","last_fullsized":"2023-10-16T14:33:16.811912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65228,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":136,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040314+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:07.040314+0000","last_peered":"2023-10-16T14:33:07.040314+0000","last_clean":"2023-10-16T14:33:07.040314+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:07.040314+0000","last_undegraded":"2023-10-16T14:33:07.040314+0000","last_fullsized":"2023-10-16T14:33:07.040314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040938+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:07.040938+0000","last_peered":"2023-10-16T14:33:07.040938+0000","last_clean":"2023-10-16T14:33:07.040938+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:07.040938+0000","last_undegraded":"2023-10-16T14:33:07.040938+0000","last_fullsized":"2023-10-16T14:33:07.040938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032436+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:07.032436+0000","last_peered":"2023-10-16T14:33:07.032436+0000","last_clean":"2023-10-16T14:33:07.032436+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:07.032436+0000","last_undegraded":"2023-10-16T14:33:07.032436+0000","last_fullsized":"2023-10-16T14:33:07.032436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040127+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:07.040127+0000","last_peered":"2023-10-16T14:33:07.040127+0000","last_clean":"2023-10-16T14:33:07.040127+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:07.040127+0000","last_undegraded":"2023-10-16T14:33:07.040127+0000","last_fullsized":"2023-10-16T14:33:07.040127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043392+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:07.043392+0000","last_peered":"2023-10-16T14:33:07.043392+0000","last_clean":"2023-10-16T14:33:07.043392+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:07.043392+0000","last_undegraded":"2023-10-16T14:33:07.043392+0000","last_fullsized":"2023-10-16T14:33:07.043392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040568+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:07.040568+0000","last_peered":"2023-10-16T14:33:07.040568+0000","last_clean":"2023-10-16T14:33:07.040568+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:07.040568+0000","last_undegraded":"2023-10-16T14:33:07.040568+0000","last_fullsized":"2023-10-16T14:33:07.040568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043937+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:07.043937+0000","last_peered":"2023-10-16T14:33:07.043937+0000","last_clean":"2023-10-16T14:33:07.043937+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:07.043937+0000","last_undegraded":"2023-10-16T14:33:07.043937+0000","last_fullsized":"2023-10-16T14:33:07.043937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043473+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:07.043473+0000","last_peered":"2023-10-16T14:33:07.043473+0000","last_clean":"2023-10-16T14:33:07.043473+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:07.043473+0000","last_undegraded":"2023-10-16T14:33:07.043473+0000","last_fullsized":"2023-10-16T14:33:07.043473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":91,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:12.805793+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:33:12.805793+0000","last_peered":"2023-10-16T14:33:12.805793+0000","last_clean":"2023-10-16T14:33:12.805793+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:33:12.805793+0000","last_undegraded":"2023-10-16T14:33:12.805793+0000","last_fullsized":"2023-10-16T14:33:12.805793+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043458+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:07.043458+0000","last_peered":"2023-10-16T14:33:07.043458+0000","last_clean":"2023-10-16T14:33:07.043458+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:07.043458+0000","last_undegraded":"2023-10-16T14:33:07.043458+0000","last_fullsized":"2023-10-16T14:33:07.043458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039890+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:07.039890+0000","last_peered":"2023-10-16T14:33:07.039890+0000","last_clean":"2023-10-16T14:33:07.039890+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:07.039890+0000","last_undegraded":"2023-10-16T14:33:07.039890+0000","last_fullsized":"2023-10-16T14:33:07.039890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038993+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:07.038993+0000","last_peered":"2023-10-16T14:33:07.038993+0000","last_clean":"2023-10-16T14:33:07.038993+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:07.038993+0000","last_undegraded":"2023-10-16T14:33:07.038993+0000","last_fullsized":"2023-10-16T14:33:07.038993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040263+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:07.040263+0000","last_peered":"2023-10-16T14:33:07.040263+0000","last_clean":"2023-10-16T14:33:07.040263+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:07.040263+0000","last_undegraded":"2023-10-16T14:33:07.040263+0000","last_fullsized":"2023-10-16T14:33:07.040263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040066+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:07.040066+0000","last_peered":"2023-10-16T14:33:07.040066+0000","last_clean":"2023-10-16T14:33:07.040066+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:07.040066+0000","last_undegraded":"2023-10-16T14:33:07.040066+0000","last_fullsized":"2023-10-16T14:33:07.040066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043486+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:07.043486+0000","last_peered":"2023-10-16T14:33:07.043486+0000","last_clean":"2023-10-16T14:33:07.043486+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:07.043486+0000","last_undegraded":"2023-10-16T14:33:07.043486+0000","last_fullsized":"2023-10-16T14:33:07.043486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039233+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:07.039233+0000","last_peered":"2023-10-16T14:33:07.039233+0000","last_clean":"2023-10-16T14:33:07.039233+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:07.039233+0000","last_undegraded":"2023-10-16T14:33:07.039233+0000","last_fullsized":"2023-10-16T14:33:07.039233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043347+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:07.043347+0000","last_peered":"2023-10-16T14:33:07.043347+0000","last_clean":"2023-10-16T14:33:07.043347+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:07.043347+0000","last_undegraded":"2023-10-16T14:33:07.043347+0000","last_fullsized":"2023-10-16T14:33:07.043347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042620+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:33:07.042620+0000","last_peered":"2023-10-16T14:33:07.042620+0000","last_clean":"2023-10-16T14:33:07.042620+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:33:07.042620+0000","last_undegraded":"2023-10-16T14:33:07.042620+0000","last_fullsized":"2023-10-16T14:33:07.042620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039503+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:07.039503+0000","last_peered":"2023-10-16T14:33:07.039503+0000","last_clean":"2023-10-16T14:33:07.039503+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:07.039503+0000","last_undegraded":"2023-10-16T14:33:07.039503+0000","last_fullsized":"2023-10-16T14:33:07.039503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043623+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:07.043623+0000","last_peered":"2023-10-16T14:33:07.043623+0000","last_clean":"2023-10-16T14:33:07.043623+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:07.043623+0000","last_undegraded":"2023-10-16T14:33:07.043623+0000","last_fullsized":"2023-10-16T14:33:07.043623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042306+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:33:07.042306+0000","last_peered":"2023-10-16T14:33:07.042306+0000","last_clean":"2023-10-16T14:33:07.042306+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:33:07.042306+0000","last_undegraded":"2023-10-16T14:33:07.042306+0000","last_fullsized":"2023-10-16T14:33:07.042306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039757+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:07.039757+0000","last_peered":"2023-10-16T14:33:07.039757+0000","last_clean":"2023-10-16T14:33:07.039757+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:07.039757+0000","last_undegraded":"2023-10-16T14:33:07.039757+0000","last_fullsized":"2023-10-16T14:33:07.039757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039716+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:07.039716+0000","last_peered":"2023-10-16T14:33:07.039716+0000","last_clean":"2023-10-16T14:33:07.039716+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:07.039716+0000","last_undegraded":"2023-10-16T14:33:07.039716+0000","last_fullsized":"2023-10-16T14:33:07.039716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":92,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:14.808197+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:33:14.808197+0000","last_peered":"2023-10-16T14:33:14.808197+0000","last_clean":"2023-10-16T14:33:14.808197+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:33:14.808197+0000","last_undegraded":"2023-10-16T14:33:14.808197+0000","last_fullsized":"2023-10-16T14:33:14.808197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050385+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050385+0000","last_peered":"2023-10-16T14:33:08.050385+0000","last_clean":"2023-10-16T14:33:08.050385+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050385+0000","last_undegraded":"2023-10-16T14:33:08.050385+0000","last_fullsized":"2023-10-16T14:33:08.050385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039720+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:07.039720+0000","last_peered":"2023-10-16T14:33:07.039720+0000","last_clean":"2023-10-16T14:33:07.039720+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:07.039720+0000","last_undegraded":"2023-10-16T14:33:07.039720+0000","last_fullsized":"2023-10-16T14:33:07.039720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036662+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:07.036662+0000","last_peered":"2023-10-16T14:33:07.036662+0000","last_clean":"2023-10-16T14:33:07.036662+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:07.036662+0000","last_undegraded":"2023-10-16T14:33:07.036662+0000","last_fullsized":"2023-10-16T14:33:07.036662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":316,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043337+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:07.043337+0000","last_peered":"2023-10-16T14:33:07.043337+0000","last_clean":"2023-10-16T14:33:07.043337+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:07.043337+0000","last_undegraded":"2023-10-16T14:33:07.043337+0000","last_fullsized":"2023-10-16T14:33:07.043337+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040242+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:07.040242+0000","last_peered":"2023-10-16T14:33:07.040242+0000","last_clean":"2023-10-16T14:33:07.040242+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:07.040242+0000","last_undegraded":"2023-10-16T14:33:07.040242+0000","last_fullsized":"2023-10-16T14:33:07.040242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039113+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:07.039113+0000","last_peered":"2023-10-16T14:33:07.039113+0000","last_clean":"2023-10-16T14:33:07.039113+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:07.039113+0000","last_undegraded":"2023-10-16T14:33:07.039113+0000","last_fullsized":"2023-10-16T14:33:07.039113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043414+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:07.043414+0000","last_peered":"2023-10-16T14:33:07.043414+0000","last_clean":"2023-10-16T14:33:07.043414+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:07.043414+0000","last_undegraded":"2023-10-16T14:33:07.043414+0000","last_fullsized":"2023-10-16T14:33:07.043414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043206+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:07.043206+0000","last_peered":"2023-10-16T14:33:07.043206+0000","last_clean":"2023-10-16T14:33:07.043206+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:07.043206+0000","last_undegraded":"2023-10-16T14:33:07.043206+0000","last_fullsized":"2023-10-16T14:33:07.043206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040357+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:07.040357+0000","last_peered":"2023-10-16T14:33:07.040357+0000","last_clean":"2023-10-16T14:33:07.040357+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:07.040357+0000","last_undegraded":"2023-10-16T14:33:07.040357+0000","last_fullsized":"2023-10-16T14:33:07.040357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039573+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:07.039573+0000","last_peered":"2023-10-16T14:33:07.039573+0000","last_clean":"2023-10-16T14:33:07.039573+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:07.039573+0000","last_undegraded":"2023-10-16T14:33:07.039573+0000","last_fullsized":"2023-10-16T14:33:07.039573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039295+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:07.039295+0000","last_peered":"2023-10-16T14:33:07.039295+0000","last_clean":"2023-10-16T14:33:07.039295+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:07.039295+0000","last_undegraded":"2023-10-16T14:33:07.039295+0000","last_fullsized":"2023-10-16T14:33:07.039295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043516+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:07.043516+0000","last_peered":"2023-10-16T14:33:07.043516+0000","last_clean":"2023-10-16T14:33:07.043516+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:07.043516+0000","last_undegraded":"2023-10-16T14:33:07.043516+0000","last_fullsized":"2023-10-16T14:33:07.043516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039866+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:07.039866+0000","last_peered":"2023-10-16T14:33:07.039866+0000","last_clean":"2023-10-16T14:33:07.039866+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:07.039866+0000","last_undegraded":"2023-10-16T14:33:07.039866+0000","last_fullsized":"2023-10-16T14:33:07.039866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032019+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:07.032019+0000","last_peered":"2023-10-16T14:33:07.032019+0000","last_clean":"2023-10-16T14:33:07.032019+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:07.032019+0000","last_undegraded":"2023-10-16T14:33:07.032019+0000","last_fullsized":"2023-10-16T14:33:07.032019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.037257+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:07.037257+0000","last_peered":"2023-10-16T14:33:07.037257+0000","last_clean":"2023-10-16T14:33:07.037257+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:07.037257+0000","last_undegraded":"2023-10-16T14:33:07.037257+0000","last_fullsized":"2023-10-16T14:33:07.037257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176243+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:07.176243+0000","last_peered":"2023-10-16T14:33:07.176243+0000","last_clean":"2023-10-16T14:33:07.176243+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:07.176243+0000","last_undegraded":"2023-10-16T14:33:07.176243+0000","last_fullsized":"2023-10-16T14:33:07.176243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036828+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:07.036828+0000","last_peered":"2023-10-16T14:33:07.036828+0000","last_clean":"2023-10-16T14:33:07.036828+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:07.036828+0000","last_undegraded":"2023-10-16T14:33:07.036828+0000","last_fullsized":"2023-10-16T14:33:07.036828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175327+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:33:07.175327+0000","last_peered":"2023-10-16T14:33:07.175327+0000","last_clean":"2023-10-16T14:33:07.175327+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:33:07.175327+0000","last_undegraded":"2023-10-16T14:33:07.175327+0000","last_fullsized":"2023-10-16T14:33:07.175327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038984+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:07.038984+0000","last_peered":"2023-10-16T14:33:07.038984+0000","last_clean":"2023-10-16T14:33:07.038984+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:07.038984+0000","last_undegraded":"2023-10-16T14:33:07.038984+0000","last_fullsized":"2023-10-16T14:33:07.038984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043627+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:07.043627+0000","last_peered":"2023-10-16T14:33:07.043627+0000","last_clean":"2023-10-16T14:33:07.043627+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:07.043627+0000","last_undegraded":"2023-10-16T14:33:07.043627+0000","last_fullsized":"2023-10-16T14:33:07.043627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039296+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:07.039296+0000","last_peered":"2023-10-16T14:33:07.039296+0000","last_clean":"2023-10-16T14:33:07.039296+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:07.039296+0000","last_undegraded":"2023-10-16T14:33:07.039296+0000","last_fullsized":"2023-10-16T14:33:07.039296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043106+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:07.043106+0000","last_peered":"2023-10-16T14:33:07.043106+0000","last_clean":"2023-10-16T14:33:07.043106+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043106+0000","last_undegraded":"2023-10-16T14:33:07.043106+0000","last_fullsized":"2023-10-16T14:33:07.043106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036610+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:07.036610+0000","last_peered":"2023-10-16T14:33:07.036610+0000","last_clean":"2023-10-16T14:33:07.036610+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:07.036610+0000","last_undegraded":"2023-10-16T14:33:07.036610+0000","last_fullsized":"2023-10-16T14:33:07.036610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032216+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:07.032216+0000","last_peered":"2023-10-16T14:33:07.032216+0000","last_clean":"2023-10-16T14:33:07.032216+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:07.032216+0000","last_undegraded":"2023-10-16T14:33:07.032216+0000","last_fullsized":"2023-10-16T14:33:07.032216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036738+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:07.036738+0000","last_peered":"2023-10-16T14:33:07.036738+0000","last_clean":"2023-10-16T14:33:07.036738+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:07.036738+0000","last_undegraded":"2023-10-16T14:33:07.036738+0000","last_fullsized":"2023-10-16T14:33:07.036738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039502+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:07.039502+0000","last_peered":"2023-10-16T14:33:07.039502+0000","last_clean":"2023-10-16T14:33:07.039502+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:07.039502+0000","last_undegraded":"2023-10-16T14:33:07.039502+0000","last_fullsized":"2023-10-16T14:33:07.039502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043684+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:07.043684+0000","last_peered":"2023-10-16T14:33:07.043684+0000","last_clean":"2023-10-16T14:33:07.043684+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:07.043684+0000","last_undegraded":"2023-10-16T14:33:07.043684+0000","last_fullsized":"2023-10-16T14:33:07.043684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175367+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:33:07.175367+0000","last_peered":"2023-10-16T14:33:07.175367+0000","last_clean":"2023-10-16T14:33:07.175367+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:33:07.175367+0000","last_undegraded":"2023-10-16T14:33:07.175367+0000","last_fullsized":"2023-10-16T14:33:07.175367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032456+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:07.032456+0000","last_peered":"2023-10-16T14:33:07.032456+0000","last_clean":"2023-10-16T14:33:07.032456+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:07.032456+0000","last_undegraded":"2023-10-16T14:33:07.032456+0000","last_fullsized":"2023-10-16T14:33:07.032456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044342+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:07.044342+0000","last_peered":"2023-10-16T14:33:07.044342+0000","last_clean":"2023-10-16T14:33:07.044342+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:07.044342+0000","last_undegraded":"2023-10-16T14:33:07.044342+0000","last_fullsized":"2023-10-16T14:33:07.044342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175857+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:07.175857+0000","last_peered":"2023-10-16T14:33:07.175857+0000","last_clean":"2023-10-16T14:33:07.175857+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:07.175857+0000","last_undegraded":"2023-10-16T14:33:07.175857+0000","last_fullsized":"2023-10-16T14:33:07.175857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040371+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:07.040371+0000","last_peered":"2023-10-16T14:33:07.040371+0000","last_clean":"2023-10-16T14:33:07.040371+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:07.040371+0000","last_undegraded":"2023-10-16T14:33:07.040371+0000","last_fullsized":"2023-10-16T14:33:07.040371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044063+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:07.044063+0000","last_peered":"2023-10-16T14:33:07.044063+0000","last_clean":"2023-10-16T14:33:07.044063+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:07.044063+0000","last_undegraded":"2023-10-16T14:33:07.044063+0000","last_fullsized":"2023-10-16T14:33:07.044063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039237+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:07.039237+0000","last_peered":"2023-10-16T14:33:07.039237+0000","last_clean":"2023-10-16T14:33:07.039237+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:07.039237+0000","last_undegraded":"2023-10-16T14:33:07.039237+0000","last_fullsized":"2023-10-16T14:33:07.039237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039819+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:07.039819+0000","last_peered":"2023-10-16T14:33:07.039819+0000","last_clean":"2023-10-16T14:33:07.039819+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:07.039819+0000","last_undegraded":"2023-10-16T14:33:07.039819+0000","last_fullsized":"2023-10-16T14:33:07.039819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044343+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:07.044343+0000","last_peered":"2023-10-16T14:33:07.044343+0000","last_clean":"2023-10-16T14:33:07.044343+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:07.044343+0000","last_undegraded":"2023-10-16T14:33:07.044343+0000","last_fullsized":"2023-10-16T14:33:07.044343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043155+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:07.043155+0000","last_peered":"2023-10-16T14:33:07.043155+0000","last_clean":"2023-10-16T14:33:07.043155+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:07.043155+0000","last_undegraded":"2023-10-16T14:33:07.043155+0000","last_fullsized":"2023-10-16T14:33:07.043155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032429+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:07.032429+0000","last_peered":"2023-10-16T14:33:07.032429+0000","last_clean":"2023-10-16T14:33:07.032429+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:07.032429+0000","last_undegraded":"2023-10-16T14:33:07.032429+0000","last_fullsized":"2023-10-16T14:33:07.032429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032099+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:07.032099+0000","last_peered":"2023-10-16T14:33:07.032099+0000","last_clean":"2023-10-16T14:33:07.032099+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:07.032099+0000","last_undegraded":"2023-10-16T14:33:07.032099+0000","last_fullsized":"2023-10-16T14:33:07.032099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176006+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:07.176006+0000","last_peered":"2023-10-16T14:33:07.176006+0000","last_clean":"2023-10-16T14:33:07.176006+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:07.176006+0000","last_undegraded":"2023-10-16T14:33:07.176006+0000","last_fullsized":"2023-10-16T14:33:07.176006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039610+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:07.039610+0000","last_peered":"2023-10-16T14:33:07.039610+0000","last_clean":"2023-10-16T14:33:07.039610+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:07.039610+0000","last_undegraded":"2023-10-16T14:33:07.039610+0000","last_fullsized":"2023-10-16T14:33:07.039610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043494+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:07.043494+0000","last_peered":"2023-10-16T14:33:07.043494+0000","last_clean":"2023-10-16T14:33:07.043494+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:07.043494+0000","last_undegraded":"2023-10-16T14:33:07.043494+0000","last_fullsized":"2023-10-16T14:33:07.043494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032712+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:07.032712+0000","last_peered":"2023-10-16T14:33:07.032712+0000","last_clean":"2023-10-16T14:33:07.032712+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:07.032712+0000","last_undegraded":"2023-10-16T14:33:07.032712+0000","last_fullsized":"2023-10-16T14:33:07.032712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040056+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:07.040056+0000","last_peered":"2023-10-16T14:33:07.040056+0000","last_clean":"2023-10-16T14:33:07.040056+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:07.040056+0000","last_undegraded":"2023-10-16T14:33:07.040056+0000","last_fullsized":"2023-10-16T14:33:07.040056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039696+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:07.039696+0000","last_peered":"2023-10-16T14:33:07.039696+0000","last_clean":"2023-10-16T14:33:07.039696+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:07.039696+0000","last_undegraded":"2023-10-16T14:33:07.039696+0000","last_fullsized":"2023-10-16T14:33:07.039696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039393+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:07.039393+0000","last_peered":"2023-10-16T14:33:07.039393+0000","last_clean":"2023-10-16T14:33:07.039393+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:07.039393+0000","last_undegraded":"2023-10-16T14:33:07.039393+0000","last_fullsized":"2023-10-16T14:33:07.039393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175600+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:33:07.175600+0000","last_peered":"2023-10-16T14:33:07.175600+0000","last_clean":"2023-10-16T14:33:07.175600+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:33:07.175600+0000","last_undegraded":"2023-10-16T14:33:07.175600+0000","last_fullsized":"2023-10-16T14:33:07.175600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039346+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:07.039346+0000","last_peered":"2023-10-16T14:33:07.039346+0000","last_clean":"2023-10-16T14:33:07.039346+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:07.039346+0000","last_undegraded":"2023-10-16T14:33:07.039346+0000","last_fullsized":"2023-10-16T14:33:07.039346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043752+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:07.043752+0000","last_peered":"2023-10-16T14:33:07.043752+0000","last_clean":"2023-10-16T14:33:07.043752+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:07.043752+0000","last_undegraded":"2023-10-16T14:33:07.043752+0000","last_fullsized":"2023-10-16T14:33:07.043752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":429,"num_read_kb":299,"num_write":164,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":237678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.61899999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.871,"15min":0.871},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.85199999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65100000000000002}]},{"osd":7,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.79200000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.77500000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.256,"15min":1.256},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.54500000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78700000000000003,"15min":0.78400000000000003},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95499999999999996,"5min":1.214,"15min":1.214},"last":0.80800000000000005}]}]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1724416,"data_stored":1163000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.629},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.69799999999999995}]},{"osd":1,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.52100000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.42899999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.68300000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72999999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84899999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57099999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.038,"15min":1.038},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.83299999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73199999999999998,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.622}]},{"osd":2,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.745,"15min":0.76700000000000002},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.77400000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72499999999999998,"15min":0.77600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79700000000000004,"15min":0.83499999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.397,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.751}]},{"osd":5,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.84399999999999997},"min":{"1min":0.51200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89500000000000002,"15min":0.94699999999999995},"min":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.83899999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83999999999999997,"15min":0.90300000000000002},"min":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":0.99199999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.2559999999999998,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.78600000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.94199999999999995},"min":{"1min":0.60499999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88400000000000001,"15min":0.92700000000000005},"min":{"1min":0.624,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.88900000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.873,"15min":0.87},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.82199999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.85099999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103215,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.71199999999999997}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.74199999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51500000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86699999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73199999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.056,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.622}]},{"osd":5,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.56299999999999994}]},{"osd":6,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":1.02,"15min":1.02},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81699999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.96399999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.76700000000000002},"min":{"1min":0.55400000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.873}]},{"osd":10,"last update":"Mon Oct 16 14:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.71599999999999997,"15min":0.68899999999999995},"min":{"1min":0.67800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.90800000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85499999999999998}]}]},{"osd":7,"up_from":51,"seq":219043332147,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1111284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53800000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.36699999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73899999999999999,"5min":2.589,"15min":2.589},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.58099999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.505},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85499999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.65300000000000002}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.745}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92500000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77900000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.79600000000000004}]}]},{"osd":6,"up_from":46,"seq":197568495671,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.441},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.503,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.41999999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71799999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.625},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66900000000000004,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.139,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.012,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0489999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.03,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.58299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.232,"15min":1.232},"last":0.75700000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.107},"last":0.745},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.76900000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0700000000000001,"5min":1.133,"15min":1.133},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0029999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81200000000000006}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.012,"15min":0.93200000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":11.375999999999999,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90900000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0680000000000001,"5min":1.157,"15min":1.157},"last":0.83999999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.871},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.85999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":566685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.34200000000000003},"max":{"1min":0.83299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.49399999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.81399999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.42999999999999999},"min":{"1min":0.40500000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.55900000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":1.181,"15min":1.181},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.624,"15min":0.58299999999999996},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.68000000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85499999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.50900000000000001},"min":{"1min":0.41499999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.55300000000000005,"15min":0.52900000000000003},"min":{"1min":0.36699999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.90900000000000003}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.92400000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77000000000000002,"15min":0.76600000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.85899999999999999,"15min":0.81799999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.83899999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.878},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.94699999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.96799999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0249999999999999,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.66000000000000003,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94899999999999995,"5min":1.026,"15min":1.141},"last":0.628}]},{"osd":2,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.65900000000000003,"15min":0.59399999999999997},"min":{"1min":0.56399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":1.202,"15min":1.202},"last":0.81299999999999994}]},{"osd":3,"last update":"Mon Oct 16 14:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75800000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88600000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.64500000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":1.101,"15min":1.101},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Mon Oct 16 14:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.016,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73699999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.748}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.84599999999999997},"min":{"1min":0.74399999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.073,"5min":1.274,"15min":1.274},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.069,"5min":1.216,"15min":1.216},"last":0.78200000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.099,"5min":0.98899999999999999,"15min":0.99399999999999999},"min":{"1min":0.67800000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.9860000000000002,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":0.79200000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.0820000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.875}]},{"osd":11,"last update":"Mon Oct 16 14:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.96399999999999997,"15min":0.96199999999999997},"min":{"1min":0.81899999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.079,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80600000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":578696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53700000000000003,"15min":0.5},"min":{"1min":0.40200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":1.0640000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57899999999999996,"15min":0.50800000000000001},"min":{"1min":0.42999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77400000000000002,"5min":1.022,"15min":1.022},"last":0.81499999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63100000000000001,"15min":0.57699999999999996},"min":{"1min":0.55400000000000005,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.78200000000000003,"15min":0.67600000000000005},"min":{"1min":0.72299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.0190000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":1.1180000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":1.0529999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99299999999999999,"5min":1.044,"15min":1.044},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.88200000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:32:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.68500000000000005,"15min":0.67600000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":1.006},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.751,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":1.0229999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.075}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.97799999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.42599999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65600000000000003,"15min":0.60699999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.53400000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.92500000000000004}]},{"osd":2,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.627},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83899999999999997,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68300000000000005,"15min":0.67400000000000004},"min":{"1min":0.46500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.90800000000000003,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.84999999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.80900000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83299999999999996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.96199999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1},{"interface":"front","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.79800000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.0229999999999999}]}]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29199999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.63700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.308,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59099999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.441}]},{"osd":2,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.51000000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.85599999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46600000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.54400000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60099999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.71799999999999997}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81399999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96399999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.75},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.73999999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70599999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.85099999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757308,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1108058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.40999999999999998}]},{"osd":1,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95099999999999996,"5min":1.425,"15min":1.425},"last":0.61299999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.39400000000000002},"min":{"1min":0.379,"5min":0.221,"15min":0.221},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.436},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.38100000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.54400000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90900000000000003,"5min":1.028,"15min":1.028},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.55400000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":1.046,"15min":1.046},"last":0.76600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":1.196,"15min":1.196},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.73699999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81599999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.66500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.629,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.68700000000000006}]},{"osd":10,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.752},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.59899999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.82299999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.79000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":11864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":11598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:33:20.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:19 smithi187 ceph-mon[135297]: pgmap v327: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-10-16T14:33:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:20 smithi178 ceph-mon[98495]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:20 smithi186 ceph-mon[134892]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:21.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:20 smithi187 ceph-mon[135297]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:22.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:21 smithi178 ceph-mon[98495]: pgmap v328: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:21 smithi186 ceph-mon[134892]: pgmap v328: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:22.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:21 smithi187 ceph-mon[135297]: pgmap v328: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:23.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph pg dump --format=json 2023-10-16T14:33:23.888 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:24.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:23 smithi178 ceph-mon[98495]: pgmap v329: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:23 smithi186 ceph-mon[134892]: pgmap v329: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:23 smithi187 ceph-mon[135297]: pgmap v329: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:25.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.502+0000 7f875374a700 1 -- 172.21.15.178:0/685185688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 msgr2=0x7f874c107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:25.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.502+0000 7f875374a700 1 --2- 172.21.15.178:0/685185688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c107370 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f87480099c0 tx=0x7f8748009cd0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 -- 172.21.15.178:0/685185688 shutdown_connections 2023-10-16T14:33:25.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 --2- 172.21.15.178:0/685185688 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c101940 0x7f874c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 --2- 172.21.15.178:0/685185688 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c107370 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 --2- 172.21.15.178:0/685185688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c100f80 0x7f874c101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 -- 172.21.15.178:0/685185688 >> 172.21.15.178:0/685185688 conn(0x7f874c0780a0 msgr2=0x7f874c0784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:25.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 -- 172.21.15.178:0/685185688 shutdown_connections 2023-10-16T14:33:25.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.503+0000 7f875374a700 1 -- 172.21.15.178:0/685185688 wait complete. 2023-10-16T14:33:25.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 Processor -- start 2023-10-16T14:33:25.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 -- start start 2023-10-16T14:33:25.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c100f80 0x7f874c0731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:25.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c101940 0x7f874c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:25.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:25.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a5450 con 0x7f874c106f90 2023-10-16T14:33:25.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a55c0 con 0x7f874c100f80 2023-10-16T14:33:25.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f875374a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a5780 con 0x7f874c101940 2023-10-16T14:33:25.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.504+0000 7f8751ce7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:25.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:48316/0 (socket says 172.21.15.178:48316) 2023-10-16T14:33:25.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 -- 172.21.15.178:0/71843791 learned_addr learned my addr 172.21.15.178:0/71843791 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:25.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f87514e6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c100f80 0x7f874c0731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:25.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8750ce5700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c101940 0x7f874c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:25.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 -- 172.21.15.178:0/71843791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c101940 msgr2=0x7f874c0736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:25.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c101940 0x7f874c0736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 -- 172.21.15.178:0/71843791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c100f80 msgr2=0x7f874c0731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:25.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c100f80 0x7f874c0731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 -- 172.21.15.178:0/71843791 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87480095f0 con 0x7f874c106f90 2023-10-16T14:33:25.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f8751ce7700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f874400efc0 tx=0x7f874400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:25.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744018070 con 0x7f874c106f90 2023-10-16T14:33:25.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1a5a00 con 0x7f874c106f90 2023-10-16T14:33:25.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f874400f040 con 0x7f874c106f90 2023-10-16T14:33:25.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.505+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400cc10 con 0x7f874c106f90 2023-10-16T14:33:25.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.506+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a5eb0 con 0x7f874c106f90 2023-10-16T14:33:25.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.507+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c04ebb0 con 0x7f874c106f90 2023-10-16T14:33:25.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.509+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f874400cd90 con 0x7f874c106f90 2023-10-16T14:33:25.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.510+0000 7f87427fc700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 0x7f87380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:25.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.510+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f87440cd520 con 0x7f874c106f90 2023-10-16T14:33:25.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.510+0000 7f87514e6700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 0x7f87380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:25.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.511+0000 7f87514e6700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 0x7f87380a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8748009990 tx=0x7f8748005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:25.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.512+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f87440966e0 con 0x7f874c106f90 2023-10-16T14:33:25.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.708+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f874c1110f0 con 0x7f87380a1510 2023-10-16T14:33:25.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.722+0000 7f87427fc700 1 -- 172.21.15.178:0/71843791 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427620 (secure 0 0 0) 0x7f874c1110f0 con 0x7f87380a1510 2023-10-16T14:33:25.731 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:25.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 msgr2=0x7f87380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 0x7f87380a39d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8748009990 tx=0x7f8748005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 msgr2=0x7f874c073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f874400efc0 tx=0x7f874400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 shutdown_connections 2023-10-16T14:33:25.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87380a1510 0x7f87380a39d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f874c106f90 0x7f874c073c30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f874c100f80 0x7f874c0731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 --2- 172.21.15.178:0/71843791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f874c101940 0x7f874c0736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 >> 172.21.15.178:0/71843791 conn(0x7f874c0780a0 msgr2=0x7f874c0ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:25.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 shutdown_connections 2023-10-16T14:33:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:25.727+0000 7f875374a700 1 -- 172.21.15.178:0/71843791 wait complete. 2023-10-16T14:33:25.737 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:33:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:25 smithi178 ceph-mon[98495]: pgmap v330: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:25 smithi186 ceph-mon[134892]: pgmap v330: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:26.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:25 smithi187 ceph-mon[135297]: pgmap v330: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:26.282 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2023-10-16T14:33:24.615316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669840,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":585,"num_read_kb":423,"num_write":347,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338392,"kb_used_data":14132,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603432,"statfs":{"total":1151940427776,"available":1151593914368,"internally_reserved":0,"allocated":14471168,"data_stored":8059621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003601"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043646+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:07.043646+0000","last_peered":"2023-10-16T14:33:07.043646+0000","last_clean":"2023-10-16T14:33:07.043646+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043646+0000","last_undegraded":"2023-10-16T14:33:07.043646+0000","last_fullsized":"2023-10-16T14:33:07.043646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176157+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:07.176157+0000","last_peered":"2023-10-16T14:33:07.176157+0000","last_clean":"2023-10-16T14:33:07.176157+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:07.176157+0000","last_undegraded":"2023-10-16T14:33:07.176157+0000","last_fullsized":"2023-10-16T14:33:07.176157+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175977+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:07.175977+0000","last_peered":"2023-10-16T14:33:07.175977+0000","last_clean":"2023-10-16T14:33:07.175977+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:07.175977+0000","last_undegraded":"2023-10-16T14:33:07.175977+0000","last_fullsized":"2023-10-16T14:33:07.175977+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036974+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:07.036974+0000","last_peered":"2023-10-16T14:33:07.036974+0000","last_clean":"2023-10-16T14:33:07.036974+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:07.036974+0000","last_undegraded":"2023-10-16T14:33:07.036974+0000","last_fullsized":"2023-10-16T14:33:07.036974+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039414+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:07.039414+0000","last_peered":"2023-10-16T14:33:07.039414+0000","last_clean":"2023-10-16T14:33:07.039414+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:07.039414+0000","last_undegraded":"2023-10-16T14:33:07.039414+0000","last_fullsized":"2023-10-16T14:33:07.039414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042411+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:33:07.042411+0000","last_peered":"2023-10-16T14:33:07.042411+0000","last_clean":"2023-10-16T14:33:07.042411+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:33:07.042411+0000","last_undegraded":"2023-10-16T14:33:07.042411+0000","last_fullsized":"2023-10-16T14:33:07.042411+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043499+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:07.043499+0000","last_peered":"2023-10-16T14:33:07.043499+0000","last_clean":"2023-10-16T14:33:07.043499+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:07.043499+0000","last_undegraded":"2023-10-16T14:33:07.043499+0000","last_fullsized":"2023-10-16T14:33:07.043499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039389+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:07.039389+0000","last_peered":"2023-10-16T14:33:07.039389+0000","last_clean":"2023-10-16T14:33:07.039389+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:07.039389+0000","last_undegraded":"2023-10-16T14:33:07.039389+0000","last_fullsized":"2023-10-16T14:33:07.039389+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175871+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:33:07.175871+0000","last_peered":"2023-10-16T14:33:07.175871+0000","last_clean":"2023-10-16T14:33:07.175871+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:33:07.175871+0000","last_undegraded":"2023-10-16T14:33:07.175871+0000","last_fullsized":"2023-10-16T14:33:07.175871+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042938+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:33:07.042938+0000","last_peered":"2023-10-16T14:33:07.042938+0000","last_clean":"2023-10-16T14:33:07.042938+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:33:07.042938+0000","last_undegraded":"2023-10-16T14:33:07.042938+0000","last_fullsized":"2023-10-16T14:33:07.042938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.045066+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:07.045066+0000","last_peered":"2023-10-16T14:33:07.045066+0000","last_clean":"2023-10-16T14:33:07.045066+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:07.045066+0000","last_undegraded":"2023-10-16T14:33:07.045066+0000","last_fullsized":"2023-10-16T14:33:07.045066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036842+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:07.036842+0000","last_peered":"2023-10-16T14:33:07.036842+0000","last_clean":"2023-10-16T14:33:07.036842+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:07.036842+0000","last_undegraded":"2023-10-16T14:33:07.036842+0000","last_fullsized":"2023-10-16T14:33:07.036842+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039435+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:07.039435+0000","last_peered":"2023-10-16T14:33:07.039435+0000","last_clean":"2023-10-16T14:33:07.039435+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:07.039435+0000","last_undegraded":"2023-10-16T14:33:07.039435+0000","last_fullsized":"2023-10-16T14:33:07.039435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175813+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:07.175813+0000","last_peered":"2023-10-16T14:33:07.175813+0000","last_clean":"2023-10-16T14:33:07.175813+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:07.175813+0000","last_undegraded":"2023-10-16T14:33:07.175813+0000","last_fullsized":"2023-10-16T14:33:07.175813+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175819+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:07.175819+0000","last_peered":"2023-10-16T14:33:07.175819+0000","last_clean":"2023-10-16T14:33:07.175819+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:07.175819+0000","last_undegraded":"2023-10-16T14:33:07.175819+0000","last_fullsized":"2023-10-16T14:33:07.175819+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175325+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:33:07.175325+0000","last_peered":"2023-10-16T14:33:07.175325+0000","last_clean":"2023-10-16T14:33:07.175325+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:33:07.175325+0000","last_undegraded":"2023-10-16T14:33:07.175325+0000","last_fullsized":"2023-10-16T14:33:07.175325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043521+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:07.043521+0000","last_peered":"2023-10-16T14:33:07.043521+0000","last_clean":"2023-10-16T14:33:07.043521+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:07.043521+0000","last_undegraded":"2023-10-16T14:33:07.043521+0000","last_fullsized":"2023-10-16T14:33:07.043521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043605+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:07.043605+0000","last_peered":"2023-10-16T14:33:07.043605+0000","last_clean":"2023-10-16T14:33:07.043605+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:07.043605+0000","last_undegraded":"2023-10-16T14:33:07.043605+0000","last_fullsized":"2023-10-16T14:33:07.043605+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032587+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:07.032587+0000","last_peered":"2023-10-16T14:33:07.032587+0000","last_clean":"2023-10-16T14:33:07.032587+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:07.032587+0000","last_undegraded":"2023-10-16T14:33:07.032587+0000","last_fullsized":"2023-10-16T14:33:07.032587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032303+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:07.032303+0000","last_peered":"2023-10-16T14:33:07.032303+0000","last_clean":"2023-10-16T14:33:07.032303+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:07.032303+0000","last_undegraded":"2023-10-16T14:33:07.032303+0000","last_fullsized":"2023-10-16T14:33:07.032303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043652+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:07.043652+0000","last_peered":"2023-10-16T14:33:07.043652+0000","last_clean":"2023-10-16T14:33:07.043652+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:07.043652+0000","last_undegraded":"2023-10-16T14:33:07.043652+0000","last_fullsized":"2023-10-16T14:33:07.043652+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040001+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:07.040001+0000","last_peered":"2023-10-16T14:33:07.040001+0000","last_clean":"2023-10-16T14:33:07.040001+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:07.040001+0000","last_undegraded":"2023-10-16T14:33:07.040001+0000","last_fullsized":"2023-10-16T14:33:07.040001+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036273+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:07.036273+0000","last_peered":"2023-10-16T14:33:07.036273+0000","last_clean":"2023-10-16T14:33:07.036273+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:07.036273+0000","last_undegraded":"2023-10-16T14:33:07.036273+0000","last_fullsized":"2023-10-16T14:33:07.036273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040091+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:07.040091+0000","last_peered":"2023-10-16T14:33:07.040091+0000","last_clean":"2023-10-16T14:33:07.040091+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:07.040091+0000","last_undegraded":"2023-10-16T14:33:07.040091+0000","last_fullsized":"2023-10-16T14:33:07.040091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175937+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:07.175937+0000","last_peered":"2023-10-16T14:33:07.175937+0000","last_clean":"2023-10-16T14:33:07.175937+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:07.175937+0000","last_undegraded":"2023-10-16T14:33:07.175937+0000","last_fullsized":"2023-10-16T14:33:07.175937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036728+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:07.036728+0000","last_peered":"2023-10-16T14:33:07.036728+0000","last_clean":"2023-10-16T14:33:07.036728+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:07.036728+0000","last_undegraded":"2023-10-16T14:33:07.036728+0000","last_fullsized":"2023-10-16T14:33:07.036728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040455+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:07.040455+0000","last_peered":"2023-10-16T14:33:07.040455+0000","last_clean":"2023-10-16T14:33:07.040455+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:07.040455+0000","last_undegraded":"2023-10-16T14:33:07.040455+0000","last_fullsized":"2023-10-16T14:33:07.040455+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175668+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:33:07.175668+0000","last_peered":"2023-10-16T14:33:07.175668+0000","last_clean":"2023-10-16T14:33:07.175668+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:33:07.175668+0000","last_undegraded":"2023-10-16T14:33:07.175668+0000","last_fullsized":"2023-10-16T14:33:07.175668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.417921+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.417921+0000","last_peered":"2023-10-16T14:33:08.417921+0000","last_clean":"2023-10-16T14:33:08.417921+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.417921+0000","last_undegraded":"2023-10-16T14:33:08.417921+0000","last_fullsized":"2023-10-16T14:33:08.417921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040394+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:07.040394+0000","last_peered":"2023-10-16T14:33:07.040394+0000","last_clean":"2023-10-16T14:33:07.040394+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:07.040394+0000","last_undegraded":"2023-10-16T14:33:07.040394+0000","last_fullsized":"2023-10-16T14:33:07.040394+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032469+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:07.032469+0000","last_peered":"2023-10-16T14:33:07.032469+0000","last_clean":"2023-10-16T14:33:07.032469+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:07.032469+0000","last_undegraded":"2023-10-16T14:33:07.032469+0000","last_fullsized":"2023-10-16T14:33:07.032469+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":104,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:18.816045+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:33:18.816045+0000","last_peered":"2023-10-16T14:33:18.816045+0000","last_clean":"2023-10-16T14:33:18.816045+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:33:18.816045+0000","last_undegraded":"2023-10-16T14:33:18.816045+0000","last_fullsized":"2023-10-16T14:33:18.816045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":12,"num_write":6,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039050+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:07.039050+0000","last_peered":"2023-10-16T14:33:07.039050+0000","last_clean":"2023-10-16T14:33:07.039050+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:07.039050+0000","last_undegraded":"2023-10-16T14:33:07.039050+0000","last_fullsized":"2023-10-16T14:33:07.039050+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032291+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:07.032291+0000","last_peered":"2023-10-16T14:33:07.032291+0000","last_clean":"2023-10-16T14:33:07.032291+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:07.032291+0000","last_undegraded":"2023-10-16T14:33:07.032291+0000","last_fullsized":"2023-10-16T14:33:07.032291+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042313+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:33:07.042313+0000","last_peered":"2023-10-16T14:33:07.042313+0000","last_clean":"2023-10-16T14:33:07.042313+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:33:07.042313+0000","last_undegraded":"2023-10-16T14:33:07.042313+0000","last_fullsized":"2023-10-16T14:33:07.042313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040470+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:07.040470+0000","last_peered":"2023-10-16T14:33:07.040470+0000","last_clean":"2023-10-16T14:33:07.040470+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:07.040470+0000","last_undegraded":"2023-10-16T14:33:07.040470+0000","last_fullsized":"2023-10-16T14:33:07.040470+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043543+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:07.043543+0000","last_peered":"2023-10-16T14:33:07.043543+0000","last_clean":"2023-10-16T14:33:07.043543+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:07.043543+0000","last_undegraded":"2023-10-16T14:33:07.043543+0000","last_fullsized":"2023-10-16T14:33:07.043543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039294+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:07.039294+0000","last_peered":"2023-10-16T14:33:07.039294+0000","last_clean":"2023-10-16T14:33:07.039294+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:07.039294+0000","last_undegraded":"2023-10-16T14:33:07.039294+0000","last_fullsized":"2023-10-16T14:33:07.039294+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039337+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:07.039337+0000","last_peered":"2023-10-16T14:33:07.039337+0000","last_clean":"2023-10-16T14:33:07.039337+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:07.039337+0000","last_undegraded":"2023-10-16T14:33:07.039337+0000","last_fullsized":"2023-10-16T14:33:07.039337+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176168+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:07.176168+0000","last_peered":"2023-10-16T14:33:07.176168+0000","last_clean":"2023-10-16T14:33:07.176168+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:07.176168+0000","last_undegraded":"2023-10-16T14:33:07.176168+0000","last_fullsized":"2023-10-16T14:33:07.176168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039620+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:07.039620+0000","last_peered":"2023-10-16T14:33:07.039620+0000","last_clean":"2023-10-16T14:33:07.039620+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:07.039620+0000","last_undegraded":"2023-10-16T14:33:07.039620+0000","last_fullsized":"2023-10-16T14:33:07.039620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042991+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:07.042991+0000","last_peered":"2023-10-16T14:33:07.042991+0000","last_clean":"2023-10-16T14:33:07.042991+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:07.042991+0000","last_undegraded":"2023-10-16T14:33:07.042991+0000","last_fullsized":"2023-10-16T14:33:07.042991+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039499+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:07.039499+0000","last_peered":"2023-10-16T14:33:07.039499+0000","last_clean":"2023-10-16T14:33:07.039499+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:07.039499+0000","last_undegraded":"2023-10-16T14:33:07.039499+0000","last_fullsized":"2023-10-16T14:33:07.039499+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:20.818561+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:33:20.818561+0000","last_peered":"2023-10-16T14:33:20.818561+0000","last_clean":"2023-10-16T14:33:20.818561+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:33:20.818561+0000","last_undegraded":"2023-10-16T14:33:20.818561+0000","last_fullsized":"2023-10-16T14:33:20.818561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175847+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:33:07.175847+0000","last_peered":"2023-10-16T14:33:07.175847+0000","last_clean":"2023-10-16T14:33:07.175847+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:33:07.175847+0000","last_undegraded":"2023-10-16T14:33:07.175847+0000","last_fullsized":"2023-10-16T14:33:07.175847+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043012+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:07.043012+0000","last_peered":"2023-10-16T14:33:07.043012+0000","last_clean":"2023-10-16T14:33:07.043012+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:07.043012+0000","last_undegraded":"2023-10-16T14:33:07.043012+0000","last_fullsized":"2023-10-16T14:33:07.043012+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044405+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:07.044405+0000","last_peered":"2023-10-16T14:33:07.044405+0000","last_clean":"2023-10-16T14:33:07.044405+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:07.044405+0000","last_undegraded":"2023-10-16T14:33:07.044405+0000","last_fullsized":"2023-10-16T14:33:07.044405+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176030+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:07.176030+0000","last_peered":"2023-10-16T14:33:07.176030+0000","last_clean":"2023-10-16T14:33:07.176030+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:07.176030+0000","last_undegraded":"2023-10-16T14:33:07.176030+0000","last_fullsized":"2023-10-16T14:33:07.176030+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036808+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:07.036808+0000","last_peered":"2023-10-16T14:33:07.036808+0000","last_clean":"2023-10-16T14:33:07.036808+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:07.036808+0000","last_undegraded":"2023-10-16T14:33:07.036808+0000","last_fullsized":"2023-10-16T14:33:07.036808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039954+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:07.039954+0000","last_peered":"2023-10-16T14:33:07.039954+0000","last_clean":"2023-10-16T14:33:07.039954+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:07.039954+0000","last_undegraded":"2023-10-16T14:33:07.039954+0000","last_fullsized":"2023-10-16T14:33:07.039954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175772+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:33:07.175772+0000","last_peered":"2023-10-16T14:33:07.175772+0000","last_clean":"2023-10-16T14:33:07.175772+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:33:07.175772+0000","last_undegraded":"2023-10-16T14:33:07.175772+0000","last_fullsized":"2023-10-16T14:33:07.175772+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.045902+0000","last_change":"2023-10-16T14:33:08.045902+0000","last_active":"2023-10-16T14:33:08.045902+0000","last_peered":"2023-10-16T14:33:08.045902+0000","last_clean":"2023-10-16T14:33:08.045902+0000","last_became_active":"2023-10-16T14:33:08.045481+0000","last_became_peered":"2023-10-16T14:33:08.045481+0000","last_unstale":"2023-10-16T14:33:08.045902+0000","last_undegraded":"2023-10-16T14:33:08.045902+0000","last_fullsized":"2023-10-16T14:33:08.045902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:18.091442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'34","reported_seq":209,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:21.820274+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:33:21.820274+0000","last_peered":"2023-10-16T14:33:21.820274+0000","last_clean":"2023-10-16T14:33:21.820274+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:33:21.820274+0000","last_undegraded":"2023-10-16T14:33:21.820274+0000","last_fullsized":"2023-10-16T14:33:21.820274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65228,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":154,"num_read_kb":141,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040314+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:07.040314+0000","last_peered":"2023-10-16T14:33:07.040314+0000","last_clean":"2023-10-16T14:33:07.040314+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:07.040314+0000","last_undegraded":"2023-10-16T14:33:07.040314+0000","last_fullsized":"2023-10-16T14:33:07.040314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040938+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:07.040938+0000","last_peered":"2023-10-16T14:33:07.040938+0000","last_clean":"2023-10-16T14:33:07.040938+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:07.040938+0000","last_undegraded":"2023-10-16T14:33:07.040938+0000","last_fullsized":"2023-10-16T14:33:07.040938+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032436+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:07.032436+0000","last_peered":"2023-10-16T14:33:07.032436+0000","last_clean":"2023-10-16T14:33:07.032436+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:07.032436+0000","last_undegraded":"2023-10-16T14:33:07.032436+0000","last_fullsized":"2023-10-16T14:33:07.032436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040127+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:07.040127+0000","last_peered":"2023-10-16T14:33:07.040127+0000","last_clean":"2023-10-16T14:33:07.040127+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:07.040127+0000","last_undegraded":"2023-10-16T14:33:07.040127+0000","last_fullsized":"2023-10-16T14:33:07.040127+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043392+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:07.043392+0000","last_peered":"2023-10-16T14:33:07.043392+0000","last_clean":"2023-10-16T14:33:07.043392+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:07.043392+0000","last_undegraded":"2023-10-16T14:33:07.043392+0000","last_fullsized":"2023-10-16T14:33:07.043392+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040568+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:07.040568+0000","last_peered":"2023-10-16T14:33:07.040568+0000","last_clean":"2023-10-16T14:33:07.040568+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:07.040568+0000","last_undegraded":"2023-10-16T14:33:07.040568+0000","last_fullsized":"2023-10-16T14:33:07.040568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043937+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:07.043937+0000","last_peered":"2023-10-16T14:33:07.043937+0000","last_clean":"2023-10-16T14:33:07.043937+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:07.043937+0000","last_undegraded":"2023-10-16T14:33:07.043937+0000","last_fullsized":"2023-10-16T14:33:07.043937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043473+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:07.043473+0000","last_peered":"2023-10-16T14:33:07.043473+0000","last_clean":"2023-10-16T14:33:07.043473+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:07.043473+0000","last_undegraded":"2023-10-16T14:33:07.043473+0000","last_fullsized":"2023-10-16T14:33:07.043473+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:17.813767+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:33:17.813767+0000","last_peered":"2023-10-16T14:33:17.813767+0000","last_clean":"2023-10-16T14:33:17.813767+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:33:17.813767+0000","last_undegraded":"2023-10-16T14:33:17.813767+0000","last_fullsized":"2023-10-16T14:33:17.813767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043458+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:07.043458+0000","last_peered":"2023-10-16T14:33:07.043458+0000","last_clean":"2023-10-16T14:33:07.043458+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:07.043458+0000","last_undegraded":"2023-10-16T14:33:07.043458+0000","last_fullsized":"2023-10-16T14:33:07.043458+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039890+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:07.039890+0000","last_peered":"2023-10-16T14:33:07.039890+0000","last_clean":"2023-10-16T14:33:07.039890+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:07.039890+0000","last_undegraded":"2023-10-16T14:33:07.039890+0000","last_fullsized":"2023-10-16T14:33:07.039890+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038993+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:07.038993+0000","last_peered":"2023-10-16T14:33:07.038993+0000","last_clean":"2023-10-16T14:33:07.038993+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:07.038993+0000","last_undegraded":"2023-10-16T14:33:07.038993+0000","last_fullsized":"2023-10-16T14:33:07.038993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040263+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:07.040263+0000","last_peered":"2023-10-16T14:33:07.040263+0000","last_clean":"2023-10-16T14:33:07.040263+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:07.040263+0000","last_undegraded":"2023-10-16T14:33:07.040263+0000","last_fullsized":"2023-10-16T14:33:07.040263+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040066+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:07.040066+0000","last_peered":"2023-10-16T14:33:07.040066+0000","last_clean":"2023-10-16T14:33:07.040066+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:07.040066+0000","last_undegraded":"2023-10-16T14:33:07.040066+0000","last_fullsized":"2023-10-16T14:33:07.040066+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043486+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:07.043486+0000","last_peered":"2023-10-16T14:33:07.043486+0000","last_clean":"2023-10-16T14:33:07.043486+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:07.043486+0000","last_undegraded":"2023-10-16T14:33:07.043486+0000","last_fullsized":"2023-10-16T14:33:07.043486+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039233+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:07.039233+0000","last_peered":"2023-10-16T14:33:07.039233+0000","last_clean":"2023-10-16T14:33:07.039233+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:07.039233+0000","last_undegraded":"2023-10-16T14:33:07.039233+0000","last_fullsized":"2023-10-16T14:33:07.039233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043347+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:07.043347+0000","last_peered":"2023-10-16T14:33:07.043347+0000","last_clean":"2023-10-16T14:33:07.043347+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:07.043347+0000","last_undegraded":"2023-10-16T14:33:07.043347+0000","last_fullsized":"2023-10-16T14:33:07.043347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042620+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:33:07.042620+0000","last_peered":"2023-10-16T14:33:07.042620+0000","last_clean":"2023-10-16T14:33:07.042620+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:33:07.042620+0000","last_undegraded":"2023-10-16T14:33:07.042620+0000","last_fullsized":"2023-10-16T14:33:07.042620+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039503+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:07.039503+0000","last_peered":"2023-10-16T14:33:07.039503+0000","last_clean":"2023-10-16T14:33:07.039503+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:07.039503+0000","last_undegraded":"2023-10-16T14:33:07.039503+0000","last_fullsized":"2023-10-16T14:33:07.039503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043623+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:07.043623+0000","last_peered":"2023-10-16T14:33:07.043623+0000","last_clean":"2023-10-16T14:33:07.043623+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:07.043623+0000","last_undegraded":"2023-10-16T14:33:07.043623+0000","last_fullsized":"2023-10-16T14:33:07.043623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.042306+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:33:07.042306+0000","last_peered":"2023-10-16T14:33:07.042306+0000","last_clean":"2023-10-16T14:33:07.042306+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:33:07.042306+0000","last_undegraded":"2023-10-16T14:33:07.042306+0000","last_fullsized":"2023-10-16T14:33:07.042306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039757+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:07.039757+0000","last_peered":"2023-10-16T14:33:07.039757+0000","last_clean":"2023-10-16T14:33:07.039757+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:07.039757+0000","last_undegraded":"2023-10-16T14:33:07.039757+0000","last_fullsized":"2023-10-16T14:33:07.039757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039716+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:07.039716+0000","last_peered":"2023-10-16T14:33:07.039716+0000","last_clean":"2023-10-16T14:33:07.039716+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:07.039716+0000","last_undegraded":"2023-10-16T14:33:07.039716+0000","last_fullsized":"2023-10-16T14:33:07.039716+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:19.816483+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:33:19.816483+0000","last_peered":"2023-10-16T14:33:19.816483+0000","last_clean":"2023-10-16T14:33:19.816483+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:33:19.816483+0000","last_undegraded":"2023-10-16T14:33:19.816483+0000","last_fullsized":"2023-10-16T14:33:19.816483+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050385+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050385+0000","last_peered":"2023-10-16T14:33:08.050385+0000","last_clean":"2023-10-16T14:33:08.050385+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050385+0000","last_undegraded":"2023-10-16T14:33:08.050385+0000","last_fullsized":"2023-10-16T14:33:08.050385+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039720+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:07.039720+0000","last_peered":"2023-10-16T14:33:07.039720+0000","last_clean":"2023-10-16T14:33:07.039720+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:07.039720+0000","last_undegraded":"2023-10-16T14:33:07.039720+0000","last_fullsized":"2023-10-16T14:33:07.039720+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036662+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:07.036662+0000","last_peered":"2023-10-16T14:33:07.036662+0000","last_clean":"2023-10-16T14:33:07.036662+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:07.036662+0000","last_undegraded":"2023-10-16T14:33:07.036662+0000","last_fullsized":"2023-10-16T14:33:07.036662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040909+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:08.040909+0000","last_peered":"2023-10-16T14:33:08.040909+0000","last_clean":"2023-10-16T14:33:08.040909+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:08.040909+0000","last_undegraded":"2023-10-16T14:33:08.040909+0000","last_fullsized":"2023-10-16T14:33:08.040909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040242+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:07.040242+0000","last_peered":"2023-10-16T14:33:07.040242+0000","last_clean":"2023-10-16T14:33:07.040242+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:07.040242+0000","last_undegraded":"2023-10-16T14:33:07.040242+0000","last_fullsized":"2023-10-16T14:33:07.040242+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039113+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:07.039113+0000","last_peered":"2023-10-16T14:33:07.039113+0000","last_clean":"2023-10-16T14:33:07.039113+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:07.039113+0000","last_undegraded":"2023-10-16T14:33:07.039113+0000","last_fullsized":"2023-10-16T14:33:07.039113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043414+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:07.043414+0000","last_peered":"2023-10-16T14:33:07.043414+0000","last_clean":"2023-10-16T14:33:07.043414+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:07.043414+0000","last_undegraded":"2023-10-16T14:33:07.043414+0000","last_fullsized":"2023-10-16T14:33:07.043414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043206+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:07.043206+0000","last_peered":"2023-10-16T14:33:07.043206+0000","last_clean":"2023-10-16T14:33:07.043206+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:07.043206+0000","last_undegraded":"2023-10-16T14:33:07.043206+0000","last_fullsized":"2023-10-16T14:33:07.043206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040357+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:07.040357+0000","last_peered":"2023-10-16T14:33:07.040357+0000","last_clean":"2023-10-16T14:33:07.040357+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:07.040357+0000","last_undegraded":"2023-10-16T14:33:07.040357+0000","last_fullsized":"2023-10-16T14:33:07.040357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039573+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:07.039573+0000","last_peered":"2023-10-16T14:33:07.039573+0000","last_clean":"2023-10-16T14:33:07.039573+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:07.039573+0000","last_undegraded":"2023-10-16T14:33:07.039573+0000","last_fullsized":"2023-10-16T14:33:07.039573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039295+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:07.039295+0000","last_peered":"2023-10-16T14:33:07.039295+0000","last_clean":"2023-10-16T14:33:07.039295+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:07.039295+0000","last_undegraded":"2023-10-16T14:33:07.039295+0000","last_fullsized":"2023-10-16T14:33:07.039295+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043516+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:07.043516+0000","last_peered":"2023-10-16T14:33:07.043516+0000","last_clean":"2023-10-16T14:33:07.043516+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:07.043516+0000","last_undegraded":"2023-10-16T14:33:07.043516+0000","last_fullsized":"2023-10-16T14:33:07.043516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039866+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:07.039866+0000","last_peered":"2023-10-16T14:33:07.039866+0000","last_clean":"2023-10-16T14:33:07.039866+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:07.039866+0000","last_undegraded":"2023-10-16T14:33:07.039866+0000","last_fullsized":"2023-10-16T14:33:07.039866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032019+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:07.032019+0000","last_peered":"2023-10-16T14:33:07.032019+0000","last_clean":"2023-10-16T14:33:07.032019+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:07.032019+0000","last_undegraded":"2023-10-16T14:33:07.032019+0000","last_fullsized":"2023-10-16T14:33:07.032019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.037257+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:07.037257+0000","last_peered":"2023-10-16T14:33:07.037257+0000","last_clean":"2023-10-16T14:33:07.037257+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:07.037257+0000","last_undegraded":"2023-10-16T14:33:07.037257+0000","last_fullsized":"2023-10-16T14:33:07.037257+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176243+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:07.176243+0000","last_peered":"2023-10-16T14:33:07.176243+0000","last_clean":"2023-10-16T14:33:07.176243+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:07.176243+0000","last_undegraded":"2023-10-16T14:33:07.176243+0000","last_fullsized":"2023-10-16T14:33:07.176243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036828+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:07.036828+0000","last_peered":"2023-10-16T14:33:07.036828+0000","last_clean":"2023-10-16T14:33:07.036828+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:07.036828+0000","last_undegraded":"2023-10-16T14:33:07.036828+0000","last_fullsized":"2023-10-16T14:33:07.036828+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175327+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:33:07.175327+0000","last_peered":"2023-10-16T14:33:07.175327+0000","last_clean":"2023-10-16T14:33:07.175327+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:33:07.175327+0000","last_undegraded":"2023-10-16T14:33:07.175327+0000","last_fullsized":"2023-10-16T14:33:07.175327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.038984+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:07.038984+0000","last_peered":"2023-10-16T14:33:07.038984+0000","last_clean":"2023-10-16T14:33:07.038984+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:07.038984+0000","last_undegraded":"2023-10-16T14:33:07.038984+0000","last_fullsized":"2023-10-16T14:33:07.038984+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043627+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:07.043627+0000","last_peered":"2023-10-16T14:33:07.043627+0000","last_clean":"2023-10-16T14:33:07.043627+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:07.043627+0000","last_undegraded":"2023-10-16T14:33:07.043627+0000","last_fullsized":"2023-10-16T14:33:07.043627+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039296+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:07.039296+0000","last_peered":"2023-10-16T14:33:07.039296+0000","last_clean":"2023-10-16T14:33:07.039296+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:07.039296+0000","last_undegraded":"2023-10-16T14:33:07.039296+0000","last_fullsized":"2023-10-16T14:33:07.039296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043106+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:07.043106+0000","last_peered":"2023-10-16T14:33:07.043106+0000","last_clean":"2023-10-16T14:33:07.043106+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:07.043106+0000","last_undegraded":"2023-10-16T14:33:07.043106+0000","last_fullsized":"2023-10-16T14:33:07.043106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036610+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:07.036610+0000","last_peered":"2023-10-16T14:33:07.036610+0000","last_clean":"2023-10-16T14:33:07.036610+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:07.036610+0000","last_undegraded":"2023-10-16T14:33:07.036610+0000","last_fullsized":"2023-10-16T14:33:07.036610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032216+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:07.032216+0000","last_peered":"2023-10-16T14:33:07.032216+0000","last_clean":"2023-10-16T14:33:07.032216+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:07.032216+0000","last_undegraded":"2023-10-16T14:33:07.032216+0000","last_fullsized":"2023-10-16T14:33:07.032216+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040181+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:07.040181+0000","last_peered":"2023-10-16T14:33:07.040181+0000","last_clean":"2023-10-16T14:33:07.040181+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:07.040181+0000","last_undegraded":"2023-10-16T14:33:07.040181+0000","last_fullsized":"2023-10-16T14:33:07.040181+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.036738+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:07.036738+0000","last_peered":"2023-10-16T14:33:07.036738+0000","last_clean":"2023-10-16T14:33:07.036738+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:07.036738+0000","last_undegraded":"2023-10-16T14:33:07.036738+0000","last_fullsized":"2023-10-16T14:33:07.036738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039502+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:07.039502+0000","last_peered":"2023-10-16T14:33:07.039502+0000","last_clean":"2023-10-16T14:33:07.039502+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:07.039502+0000","last_undegraded":"2023-10-16T14:33:07.039502+0000","last_fullsized":"2023-10-16T14:33:07.039502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043684+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:07.043684+0000","last_peered":"2023-10-16T14:33:07.043684+0000","last_clean":"2023-10-16T14:33:07.043684+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:07.043684+0000","last_undegraded":"2023-10-16T14:33:07.043684+0000","last_fullsized":"2023-10-16T14:33:07.043684+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175367+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:33:07.175367+0000","last_peered":"2023-10-16T14:33:07.175367+0000","last_clean":"2023-10-16T14:33:07.175367+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:33:07.175367+0000","last_undegraded":"2023-10-16T14:33:07.175367+0000","last_fullsized":"2023-10-16T14:33:07.175367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032456+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:07.032456+0000","last_peered":"2023-10-16T14:33:07.032456+0000","last_clean":"2023-10-16T14:33:07.032456+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:07.032456+0000","last_undegraded":"2023-10-16T14:33:07.032456+0000","last_fullsized":"2023-10-16T14:33:07.032456+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044342+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:07.044342+0000","last_peered":"2023-10-16T14:33:07.044342+0000","last_clean":"2023-10-16T14:33:07.044342+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:07.044342+0000","last_undegraded":"2023-10-16T14:33:07.044342+0000","last_fullsized":"2023-10-16T14:33:07.044342+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175857+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:07.175857+0000","last_peered":"2023-10-16T14:33:07.175857+0000","last_clean":"2023-10-16T14:33:07.175857+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:07.175857+0000","last_undegraded":"2023-10-16T14:33:07.175857+0000","last_fullsized":"2023-10-16T14:33:07.175857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040371+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:07.040371+0000","last_peered":"2023-10-16T14:33:07.040371+0000","last_clean":"2023-10-16T14:33:07.040371+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:07.040371+0000","last_undegraded":"2023-10-16T14:33:07.040371+0000","last_fullsized":"2023-10-16T14:33:07.040371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044063+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:07.044063+0000","last_peered":"2023-10-16T14:33:07.044063+0000","last_clean":"2023-10-16T14:33:07.044063+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:07.044063+0000","last_undegraded":"2023-10-16T14:33:07.044063+0000","last_fullsized":"2023-10-16T14:33:07.044063+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039237+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:07.039237+0000","last_peered":"2023-10-16T14:33:07.039237+0000","last_clean":"2023-10-16T14:33:07.039237+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:07.039237+0000","last_undegraded":"2023-10-16T14:33:07.039237+0000","last_fullsized":"2023-10-16T14:33:07.039237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039819+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:07.039819+0000","last_peered":"2023-10-16T14:33:07.039819+0000","last_clean":"2023-10-16T14:33:07.039819+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:07.039819+0000","last_undegraded":"2023-10-16T14:33:07.039819+0000","last_fullsized":"2023-10-16T14:33:07.039819+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.044343+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:07.044343+0000","last_peered":"2023-10-16T14:33:07.044343+0000","last_clean":"2023-10-16T14:33:07.044343+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:07.044343+0000","last_undegraded":"2023-10-16T14:33:07.044343+0000","last_fullsized":"2023-10-16T14:33:07.044343+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043155+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:07.043155+0000","last_peered":"2023-10-16T14:33:07.043155+0000","last_clean":"2023-10-16T14:33:07.043155+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:07.043155+0000","last_undegraded":"2023-10-16T14:33:07.043155+0000","last_fullsized":"2023-10-16T14:33:07.043155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032429+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:07.032429+0000","last_peered":"2023-10-16T14:33:07.032429+0000","last_clean":"2023-10-16T14:33:07.032429+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:07.032429+0000","last_undegraded":"2023-10-16T14:33:07.032429+0000","last_fullsized":"2023-10-16T14:33:07.032429+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032099+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:07.032099+0000","last_peered":"2023-10-16T14:33:07.032099+0000","last_clean":"2023-10-16T14:33:07.032099+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:07.032099+0000","last_undegraded":"2023-10-16T14:33:07.032099+0000","last_fullsized":"2023-10-16T14:33:07.032099+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.176006+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:07.176006+0000","last_peered":"2023-10-16T14:33:07.176006+0000","last_clean":"2023-10-16T14:33:07.176006+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:07.176006+0000","last_undegraded":"2023-10-16T14:33:07.176006+0000","last_fullsized":"2023-10-16T14:33:07.176006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039610+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:07.039610+0000","last_peered":"2023-10-16T14:33:07.039610+0000","last_clean":"2023-10-16T14:33:07.039610+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:07.039610+0000","last_undegraded":"2023-10-16T14:33:07.039610+0000","last_fullsized":"2023-10-16T14:33:07.039610+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043494+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:07.043494+0000","last_peered":"2023-10-16T14:33:07.043494+0000","last_clean":"2023-10-16T14:33:07.043494+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:07.043494+0000","last_undegraded":"2023-10-16T14:33:07.043494+0000","last_fullsized":"2023-10-16T14:33:07.043494+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.032712+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:07.032712+0000","last_peered":"2023-10-16T14:33:07.032712+0000","last_clean":"2023-10-16T14:33:07.032712+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:07.032712+0000","last_undegraded":"2023-10-16T14:33:07.032712+0000","last_fullsized":"2023-10-16T14:33:07.032712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.040056+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:07.040056+0000","last_peered":"2023-10-16T14:33:07.040056+0000","last_clean":"2023-10-16T14:33:07.040056+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:07.040056+0000","last_undegraded":"2023-10-16T14:33:07.040056+0000","last_fullsized":"2023-10-16T14:33:07.040056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039696+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:07.039696+0000","last_peered":"2023-10-16T14:33:07.039696+0000","last_clean":"2023-10-16T14:33:07.039696+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:07.039696+0000","last_undegraded":"2023-10-16T14:33:07.039696+0000","last_fullsized":"2023-10-16T14:33:07.039696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039393+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:07.039393+0000","last_peered":"2023-10-16T14:33:07.039393+0000","last_clean":"2023-10-16T14:33:07.039393+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:07.039393+0000","last_undegraded":"2023-10-16T14:33:07.039393+0000","last_fullsized":"2023-10-16T14:33:07.039393+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.175600+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:33:07.175600+0000","last_peered":"2023-10-16T14:33:07.175600+0000","last_clean":"2023-10-16T14:33:07.175600+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:33:07.175600+0000","last_undegraded":"2023-10-16T14:33:07.175600+0000","last_fullsized":"2023-10-16T14:33:07.175600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.039346+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:07.039346+0000","last_peered":"2023-10-16T14:33:07.039346+0000","last_clean":"2023-10-16T14:33:07.039346+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:07.039346+0000","last_undegraded":"2023-10-16T14:33:07.039346+0000","last_fullsized":"2023-10-16T14:33:07.039346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043752+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:07.043752+0000","last_peered":"2023-10-16T14:33:07.043752+0000","last_clean":"2023-10-16T14:33:07.043752+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:07.043752+0000","last_undegraded":"2023-10-16T14:33:07.043752+0000","last_fullsized":"2023-10-16T14:33:07.043752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79226,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":459,"num_read_kb":314,"num_write":164,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":237678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449122,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":4.1749999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.497},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.44400000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.42999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76800000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.33300000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77100000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48899999999999999,"15min":0.48999999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.79700000000000004,"5min":0.871,"15min":0.871},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.61799999999999999},"min":{"1min":0.45700000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.628,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60799999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80700000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46200000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69399999999999995,"15min":0.69999999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.62}]},{"osd":7,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71499999999999997,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71699999999999997}]},{"osd":8,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98899999999999999,"5min":1.256,"15min":1.256},"last":0.751},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.92500000000000004,"5min":0.999,"15min":0.999},"last":0.65100000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:32:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.97099999999999997,"5min":1.046,"15min":1.046},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78700000000000003,"15min":0.78400000000000003},"min":{"1min":0.60799999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.95499999999999996,"5min":1.214,"15min":1.214},"last":0.79900000000000004}]}]},{"osd":10,"up_from":71,"seq":304942678054,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1724416,"data_stored":1163000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.79000000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.45900000000000002,"15min":0.45700000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.41599999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.878,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.621},{"interface":"front","average":{"1min":0.628,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.82299999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.67500000000000004,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.85499999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.70499999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.35799999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.44400000000000001}]},{"osd":5,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67900000000000005,"15min":0.67400000000000004},"min":{"1min":0.41599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.746,"15min":0.752},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.016},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.91800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.81000000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.79300000000000004},"min":{"1min":0.51500000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.92000000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.57699999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.97699999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.53500000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.034,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.879}]},{"osd":11,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.57099999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.008,"5min":1.038,"15min":1.038},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.73999999999999999}]}]},{"osd":9,"up_from":66,"seq":283467841579,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.58299999999999996},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56799999999999995,"15min":0.59999999999999998},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73199999999999998,"15min":0.77200000000000002},"min":{"1min":0.39800000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.41299999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.319,"5min":2.319,"15min":2.319},"last":0.58799999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.79300000000000004,"15min":0.84299999999999997},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.745,"15min":0.76700000000000002},"min":{"1min":0.47199999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":0.53900000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.38300000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":2.7090000000000001,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.72499999999999998,"15min":0.77600000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.746}]},{"osd":4,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.79700000000000004,"15min":0.83499999999999996},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78900000000000003,"15min":0.82799999999999996},"min":{"1min":0.504,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.397,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.68999999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.80300000000000005,"15min":0.84399999999999997},"min":{"1min":0.51200000000000001,"5min":0.44,"15min":0.44},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.89500000000000002,"15min":0.94699999999999995},"min":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.83299999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.83999999999999997,"15min":0.90300000000000002},"min":{"1min":0.442,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.93600000000000005,"15min":0.99199999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.2559999999999998,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.67200000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.89600000000000002,"15min":0.94199999999999995},"min":{"1min":0.60499999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88400000000000001,"15min":0.92700000000000005},"min":{"1min":0.624,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.6379999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.82099999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.872,"15min":0.879},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.873,"15min":0.87},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.5840000000000001,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.79900000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.6779999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.77600000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103216,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.45500000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45000000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.505}]},{"osd":2,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.32700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81200000000000006,"5min":0.87,"15min":0.87},"last":0.38},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.35699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.46200000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.625},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66400000000000003,"15min":0.67400000000000004},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86699999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.056,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.58799999999999997},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63500000000000001,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.92600000000000005,"5min":1.087,"15min":1.087},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":1.02,"15min":1.02},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.72999999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.753}]},{"osd":7,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.73899999999999999,"15min":0.747},"min":{"1min":0.56399999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.749,"15min":0.747},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.72699999999999998}]},{"osd":9,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.76700000000000002},"min":{"1min":0.55400000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.76700000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:32:32 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.71599999999999997,"15min":0.68899999999999995},"min":{"1min":0.67800000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.74099999999999999,"15min":0.72299999999999998},"min":{"1min":0.56100000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.81499999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.83099999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1634304,"data_stored":1111284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.53800000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56499999999999995,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.625}]},{"osd":2,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.56799999999999995,"15min":0.54900000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.622,"5min":0.57199999999999995,"15min":0.56000000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58999999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60999999999999999,"15min":0.58799999999999997},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.73899999999999999,"5min":2.589,"15min":2.589},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.72599999999999998}]},{"osd":4,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.69999999999999996,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.753,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.68200000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.90400000000000003,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.755},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.64800000000000002,"15min":0.63100000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.878}]},{"osd":6,"last update":"Mon Oct 16 14:32:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.69799999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.85499999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68999999999999995,"15min":0.68100000000000005},"min":{"1min":0.47799999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.98899999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70099999999999996,"15min":0.68300000000000005},"min":{"1min":0.61499999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.95299999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.97299999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.76600000000000001,"15min":0.77400000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76600000000000001,"15min":0.77500000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92500000000000004,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.92600000000000005}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.89500000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495672,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.503,"15min":0.495},"min":{"1min":0.36199999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54200000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.63500000000000001,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71799999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.625},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66900000000000004,"15min":0.63500000000000001},"min":{"1min":0.46800000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84899999999999998,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.41899999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.44,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.139,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72899999999999998,"15min":0.72099999999999997},"min":{"1min":0.53500000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.012,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69999999999999996},"min":{"1min":0.50800000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0489999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Mon Oct 16 14:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.52100000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.03,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.77100000000000002,"15min":0.76100000000000001},"min":{"1min":0.58299999999999996,"5min":0.503,"15min":0.503},"max":{"1min":1.0900000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76900000000000002,"15min":0.75700000000000001},"min":{"1min":0.63200000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.997,"5min":1.232,"15min":1.232},"last":0.76000000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78400000000000003,"15min":0.79000000000000004},"min":{"1min":0.57999999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.107},"last":0.626},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.60899999999999999,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.52300000000000002}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0700000000000001,"5min":1.133,"15min":1.133},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0029999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.68300000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.012,"15min":0.93200000000000005},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":11.375999999999999,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.90600000000000003,"15min":0.90900000000000003},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.0680000000000001,"5min":1.157,"15min":1.157},"last":0.74299999999999999}]},{"osd":11,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":11.394,"5min":11.394,"15min":11.394},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":566685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.34200000000000003},"max":{"1min":0.83299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.68000000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.58399999999999996,"15min":0.53700000000000003},"min":{"1min":0.38800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.84799999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.56699999999999995,"15min":0.53700000000000003},"min":{"1min":0.45000000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.77200000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.42399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79500000000000004,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.496},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.61199999999999999,"15min":0.56299999999999994},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.83599999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.629}]},{"osd":4,"last update":"Mon Oct 16 14:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.42999999999999999},"min":{"1min":0.40500000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.70299999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:32:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.48199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.92900000000000005,"5min":1.181,"15min":1.181},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.624,"15min":0.58299999999999996},"min":{"1min":0.498,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.80900000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.69599999999999995,"15min":0.67000000000000004},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66200000000000003,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.875}]},{"osd":7,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.82199999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.497,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72199999999999998,"15min":0.69599999999999995},"min":{"1min":0.54300000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.83899999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.77000000000000002,"15min":0.76600000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1200000000000001,"5min":0.85899999999999999,"15min":0.81799999999999995},"min":{"1min":0.56999999999999995,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":6.9989999999999997,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.72099999999999997}]},{"osd":10,"last update":"Mon Oct 16 14:32:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.94499999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0249999999999999,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.66000000000000003,"15min":0.58499999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94899999999999995,"5min":1.026,"15min":1.141},"last":1.0149999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.34999999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.78200000000000003,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.65900000000000003,"15min":0.59399999999999997},"min":{"1min":0.56399999999999995,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.99299999999999999,"5min":1.202,"15min":1.202},"last":0.66500000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:32:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.88400000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:32:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.48399999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88600000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.69199999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:32:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69999999999999996,"15min":0.65300000000000002},"min":{"1min":0.66300000000000003,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92800000000000005,"5min":1.101,"15min":1.101},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64900000000000002,"15min":0.64600000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.90000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.63300000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.62,"15min":0.59199999999999997},"min":{"1min":0.42799999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.749,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.016,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.92200000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86899999999999999}]},{"osd":8,"last update":"Mon Oct 16 14:32:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85399999999999998,"15min":0.84599999999999997},"min":{"1min":0.74399999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.073,"5min":1.274,"15min":1.274},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.069,"5min":1.216,"15min":1.216},"last":0.78400000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83999999999999997,"15min":0.878},"min":{"1min":0.50800000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.099,"5min":0.98899999999999999,"15min":0.99399999999999999},"min":{"1min":0.67800000000000005,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.9860000000000002,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":1.0509999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85699999999999998,"15min":0.871},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.88500000000000001,"5min":1.25,"15min":1.25},"last":0.751},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.71299999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.0820000000000001,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.083}]},{"osd":11,"last update":"Mon Oct 16 14:32:52 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.96399999999999997,"15min":0.96199999999999997},"min":{"1min":0.81899999999999995,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.1080000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.115},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82099999999999995,"15min":0.81899999999999995},"min":{"1min":0.65500000000000003,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.079,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":578696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53700000000000003,"15min":0.5},"min":{"1min":0.40200000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.49399999999999999},"min":{"1min":0.38900000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71899999999999997,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59099999999999997}]},{"osd":1,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.57899999999999996,"15min":0.50800000000000001},"min":{"1min":0.42999999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77400000000000002,"5min":1.022,"15min":1.022},"last":0.622}]},{"osd":3,"last update":"Mon Oct 16 14:32:56 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.63100000000000001,"15min":0.57699999999999996},"min":{"1min":0.55400000000000005,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89900000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.60199999999999998,"15min":0.56499999999999995},"min":{"1min":0.44,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90600000000000003,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"min":{"1min":0.36599999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.755},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.47099999999999997,"15min":0.44400000000000001},"min":{"1min":0.34200000000000003,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:33:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.626,"15min":0.61499999999999999},"min":{"1min":0.44600000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.80600000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.78200000000000003,"15min":0.67600000000000005},"min":{"1min":0.72299999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.0190000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.96899999999999997}]},{"osd":7,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.94499999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:32:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.629,"15min":0.58099999999999996},"min":{"1min":0.47899999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99299999999999999,"5min":1.044,"15min":1.044},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.59699999999999998,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.82099999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70599999999999996,"15min":0.68300000000000005},"min":{"1min":0.63200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.747},"min":{"1min":0.74199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":1.018}]},{"osd":10,"last update":"Mon Oct 16 14:32:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.873},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.80100000000000005,"15min":0.78300000000000003},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.042}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.93000000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.42599999999999999},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.5249999999999999,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.48099999999999998,"15min":0.45500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.65600000000000003,"15min":0.60699999999999998},"min":{"1min":0.495,"5min":0.375,"15min":0.375},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.65200000000000002,"15min":0.59099999999999997},"min":{"1min":0.53400000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":3.0670000000000002,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.84799999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.2519999999999998,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.69499999999999995,"15min":0.63600000000000001},"min":{"1min":0.442,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.8069999999999999,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Oct 16 14:33:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":0.79700000000000004,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.443},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.93000000000000005}]},{"osd":5,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67000000000000004,"15min":0.64400000000000002},"min":{"1min":0.53500000000000003,"5min":0.435,"15min":0.435},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68000000000000005,"15min":0.64600000000000002},"min":{"1min":0.46800000000000003,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96299999999999997,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:33:06 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.67700000000000005,"15min":0.624},"min":{"1min":0.39600000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.71099999999999997,"15min":0.65500000000000003},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":3.2200000000000002,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.88500000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.83299999999999996,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.71899999999999997}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.98199999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.81200000000000006}]},{"osd":11,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.034}]}]},{"osd":4,"up_from":33,"seq":141733920834,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40300000000000002,"15min":0.38500000000000001},"min":{"1min":0.29199999999999998,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.63700000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.441},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41599999999999998,"15min":0.40000000000000002},"min":{"1min":0.308,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.59099999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.48099999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.42499999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81799999999999995,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.44500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73099999999999998,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61299999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:32:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.56200000000000006},"min":{"1min":0.377,"5min":0.309,"15min":0.309},"max":{"1min":0.85599999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.55600000000000005},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.45900000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.58899999999999997},"min":{"1min":0.51400000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.84199999999999997,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89800000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65300000000000002,"15min":0.63500000000000001},"min":{"1min":0.54400000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.72699999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58299999999999996,"15min":0.55900000000000005},"min":{"1min":0.34999999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.82599999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.626,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.93500000000000005,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.84899999999999998}]},{"osd":8,"last update":"Mon Oct 16 14:32:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.49399999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82299999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72499999999999998,"15min":0.72399999999999998},"min":{"1min":0.54600000000000004,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96699999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67100000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96399999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.81100000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:32:29 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80000000000000004,"15min":0.80500000000000005},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.88200000000000001}]}]},{"osd":5,"up_from":38,"seq":163208757309,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1108058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68999999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.79400000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.95099999999999996,"5min":1.425,"15min":1.425},"last":0.69999999999999996}]},{"osd":2,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.41499999999999998,"15min":0.39400000000000002},"min":{"1min":0.379,"5min":0.221,"15min":0.221},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.40600000000000003},"min":{"1min":0.31900000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.65700000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54300000000000004}]},{"osd":3,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56999999999999995,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.502,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.78100000000000003,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82899999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:32:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.57299999999999995},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.59799999999999998,"15min":0.55600000000000005},"min":{"1min":0.53000000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71199999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:33:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.39100000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90900000000000003,"5min":1.028,"15min":1.028},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68700000000000006,"15min":0.66700000000000004},"min":{"1min":0.55400000000000005,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88600000000000001,"5min":1.046,"15min":1.046},"last":0.88600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:32:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55600000000000005,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95599999999999996,"5min":1.196,"15min":1.196},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.64600000000000002,"15min":0.63800000000000001},"min":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.872,"5min":0.878,"15min":0.878},"last":0.91900000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:32:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.64100000000000001,"15min":0.63100000000000001},"min":{"1min":0.38800000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59099999999999997,"15min":0.57099999999999995},"min":{"1min":0.47599999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81599999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.60999999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.66500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.629,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.98499999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:32:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.70699999999999996,"15min":0.68999999999999995},"min":{"1min":0.54600000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.72099999999999997,"15min":0.70899999999999996},"min":{"1min":0.59899999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.94699999999999995}]},{"osd":11,"last update":"Mon Oct 16 14:33:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.749},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.871}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":76966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":11864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":11598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:33:26.291 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-10-16T14:33:26.292 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T14:33:26.292 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-10-16T14:33:26.292 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph health --format=json 2023-10-16T14:33:27.026 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:27.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:26 smithi178 ceph-mon[98495]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:26 smithi186 ceph-mon[134892]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:27.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:26 smithi187 ceph-mon[135297]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:33:28.028 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:27 smithi178 ceph-mon[98495]: pgmap v331: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:27 smithi186 ceph-mon[134892]: pgmap v331: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:28.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:27 smithi187 ceph-mon[135297]: pgmap v331: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:28.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 -- 172.21.15.178:0/1959988477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101110 msgr2=0x7f91bc1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:28.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 --2- 172.21.15.178:0/1959988477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101110 0x7f91bc1014f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009990 tx=0x7f91b8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 -- 172.21.15.178:0/1959988477 shutdown_connections 2023-10-16T14:33:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 --2- 172.21.15.178:0/1959988477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc1061f0 0x7f91bc10ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 --2- 172.21.15.178:0/1959988477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101110 0x7f91bc1014f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 --2- 172.21.15.178:0/1959988477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc101ac0 0x7f91bc105b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 -- 172.21.15.178:0/1959988477 >> 172.21.15.178:0/1959988477 conn(0x7f91bc0fc9c0 msgr2=0x7f91bc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:28.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 -- 172.21.15.178:0/1959988477 shutdown_connections 2023-10-16T14:33:28.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.694+0000 7f91c3367700 1 -- 172.21.15.178:0/1959988477 wait complete. 2023-10-16T14:33:28.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 Processor -- start 2023-10-16T14:33:28.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 -- start start 2023-10-16T14:33:28.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:28.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 0x7f91bc199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:28.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 0x7f91bc19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:28.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc19e1c0 con 0x7f91bc101ac0 2023-10-16T14:33:28.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc19e330 con 0x7f91bc1061f0 2023-10-16T14:33:28.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.695+0000 7f91c3367700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc19e4f0 con 0x7f91bc101110 2023-10-16T14:33:28.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:28.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c0902700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 0x7f91bc199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:28.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:34846/0 (socket says 172.21.15.178:34846) 2023-10-16T14:33:28.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 -- 172.21.15.178:0/275963557 learned_addr learned my addr 172.21.15.178:0/275963557 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:28.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1904700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 0x7f91bc19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 -- 172.21.15.178:0/275963557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 msgr2=0x7f91bc19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 0x7f91bc19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 -- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 msgr2=0x7f91bc199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 0x7f91bc199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 -- 172.21.15.178:0/275963557 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b80095f0 con 0x7f91bc101110 2023-10-16T14:33:28.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1904700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 0x7f91bc19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:28.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c0902700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 0x7f91bc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:28.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.696+0000 7f91c1103700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005910 tx=0x7f91b8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.699+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b801d070 con 0x7f91bc101110 2023-10-16T14:33:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.699+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc19e770 con 0x7f91bc101110 2023-10-16T14:33:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.699+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91b8022470 con 0x7f91bc101110 2023-10-16T14:33:28.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.699+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b8017650 con 0x7f91bc101110 2023-10-16T14:33:28.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.700+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc1a6030 con 0x7f91bc101110 2023-10-16T14:33:28.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.701+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc04ebb0 con 0x7f91bc101110 2023-10-16T14:33:28.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.702+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f91b800fad0 con 0x7f91bc101110 2023-10-16T14:33:28.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.703+0000 7f91b27fc700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 0x7f91a80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:28.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.704+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f91b80ce730 con 0x7f91bc101110 2023-10-16T14:33:28.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.704+0000 7f91c0902700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 0x7f91a80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:28.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.704+0000 7f91c0902700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 0x7f91a80a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91bc19a8c0 tx=0x7f91ac0073d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:28.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.708+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f91b80978f0 con 0x7f91bc101110 2023-10-16T14:33:28.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.947+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f91bc19a600 con 0x7f91bc101110 2023-10-16T14:33:28.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.948+0000 7f91b27fc700 1 -- 172.21.15.178:0/275963557 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f91b809af10 con 0x7f91bc101110 2023-10-16T14:33:28.953 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:28.953 INFO:teuthology.orchestra.run.smithi178.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T14:33:28.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 msgr2=0x7f91a80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:28.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 0x7f91a80a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f91bc19a8c0 tx=0x7f91ac0073d0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 msgr2=0x7f91bc1992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:28.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f91b8005910 tx=0x7f91b8004730 comp rx=0 tx=0).stop 2023-10-16T14:33:28.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 shutdown_connections 2023-10-16T14:33:28.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f91a80a1590 0x7f91a80a3a50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f91bc1061f0 0x7f91bc19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91bc101110 0x7f91bc1992f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 --2- 172.21.15.178:0/275963557 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91bc101ac0 0x7f91bc199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:28.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 >> 172.21.15.178:0/275963557 conn(0x7f91bc0fc9c0 msgr2=0x7f91bc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:28.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 shutdown_connections 2023-10-16T14:33:28.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:28.951+0000 7f91c3367700 1 -- 172.21.15.178:0/275963557 wait complete. 2023-10-16T14:33:29.527 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-10-16T14:33:29.528 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:33:29.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 -- 172.21.15.178:0/1749403412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8100f20 msgr2=0x7fa5d8101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 --2- 172.21.15.178:0/1749403412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8100f20 0x7fa5d8101340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0098f0 tx=0x7fa5cc009c00 comp rx=0 tx=0).stop 2023-10-16T14:33:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 -- 172.21.15.178:0/1749403412 shutdown_connections 2023-10-16T14:33:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 --2- 172.21.15.178:0/1749403412 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102a30 0x7fa5d8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 --2- 172.21.15.178:0/1749403412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8100f20 0x7fa5d8101340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 --2- 172.21.15.178:0/1749403412 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8102070 0x7fa5d81024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 -- 172.21.15.178:0/1749403412 >> 172.21.15.178:0/1749403412 conn(0x7fa5d80fc450 msgr2=0x7fa5d80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:29.755 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 -- 172.21.15.178:0/1749403412 shutdown_connections 2023-10-16T14:33:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.749+0000 7fa5defff700 1 -- 172.21.15.178:0/1749403412 wait complete. 2023-10-16T14:33:29.756 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 Processor -- start 2023-10-16T14:33:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 -- start start 2023-10-16T14:33:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:29.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102070 0x7fa5d80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8102a30 0x7fa5d80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.750+0000 7fa5defff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d8073f80 con 0x7fa5d8100f20 2023-10-16T14:33:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5defff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d80740f0 con 0x7fa5d8102070 2023-10-16T14:33:29.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5defff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d81a0b80 con 0x7fa5d8102a30 2023-10-16T14:33:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59500/0 (socket says 172.21.15.178:59500) 2023-10-16T14:33:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 -- 172.21.15.178:0/1039625194 learned_addr learned my addr 172.21.15.178:0/1039625194 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5d7fff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102070 0x7fa5d80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:29.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dd59c700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8102a30 0x7fa5d80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 -- 172.21.15.178:0/1039625194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8102a30 msgr2=0x7fa5d80737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8102a30 0x7fa5d80737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 -- 172.21.15.178:0/1039625194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102070 msgr2=0x7fa5d80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:29.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102070 0x7fa5d80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 -- 172.21.15.178:0/1039625194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc009520 con 0x7fa5d8100f20 2023-10-16T14:33:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.751+0000 7fa5dcd9b700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009fd0 tx=0x7fa5cc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.752+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc01c070 con 0x7fa5d8100f20 2023-10-16T14:33:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.752+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5cc00f7d0 con 0x7fa5d8100f20 2023-10-16T14:33:29.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.752+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004ab0 con 0x7fa5d8100f20 2023-10-16T14:33:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.752+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d81a0e00 con 0x7fa5d8100f20 2023-10-16T14:33:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.752+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d81a11d0 con 0x7fa5d8100f20 2023-10-16T14:33:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.753+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d810c780 con 0x7fa5d8100f20 2023-10-16T14:33:29.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.755+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fa5cc020070 con 0x7fa5d8100f20 2023-10-16T14:33:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.756+0000 7fa5d5ffb700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 0x7fa5c80a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.756+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa5cc0ce270 con 0x7fa5d8100f20 2023-10-16T14:33:29.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.756+0000 7fa5d7fff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 0x7fa5c80a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.757+0000 7fa5d7fff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 0x7fa5c80a3b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005fd0 tx=0x7fa5c4009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:29.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.758+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa5cc097430 con 0x7fa5d8100f20 2023-10-16T14:33:29.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:29 smithi178 ceph-mon[98495]: pgmap v332: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:29.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:29 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/275963557' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-16T14:33:29.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.965+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5d8107dc0 con 0x7fa5d8100f20 2023-10-16T14:33:29.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.966+0000 7fa5d5ffb700 1 -- 172.21.15.178:0/1039625194 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fa5cc09aa50 con 0x7fa5d8100f20 2023-10-16T14:33:29.970 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:29.971 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:33:29.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 msgr2=0x7fa5c80a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:29.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 0x7fa5c80a3b00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4005fd0 tx=0x7fa5c4009310 comp rx=0 tx=0).stop 2023-10-16T14:33:29.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 msgr2=0x7fa5d8074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:29.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009fd0 tx=0x7fa5cc00b790 comp rx=0 tx=0).stop 2023-10-16T14:33:29.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 shutdown_connections 2023-10-16T14:33:29.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa5c80a1640 0x7fa5c80a3b00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5d8102a30 0x7fa5d80737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5d8100f20 0x7fa5d8074c60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 --2- 172.21.15.178:0/1039625194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa5d8102070 0x7fa5d80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:29.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 >> 172.21.15.178:0/1039625194 conn(0x7fa5d80fc450 msgr2=0x7fa5d80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:29.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 shutdown_connections 2023-10-16T14:33:29.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:29.969+0000 7fa5defff700 1 -- 172.21.15.178:0/1039625194 wait complete. 2023-10-16T14:33:29.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:33:29.993 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:33:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:29 smithi186 ceph-mon[134892]: pgmap v332: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:29 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/275963557' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-16T14:33:30.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:29 smithi187 ceph-mon[135297]: pgmap v332: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:29 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/275963557' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-10-16T14:33:30.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 -- 172.21.15.178:0/1750811011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac100e80 msgr2=0x7f48ac1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 --2- 172.21.15.178:0/1750811011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac100e80 0x7f48ac1012a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f489c009990 tx=0x7f489c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 -- 172.21.15.178:0/1750811011 shutdown_connections 2023-10-16T14:33:30.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 --2- 172.21.15.178:0/1750811011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102a20 0x7f48ac109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 --2- 172.21.15.178:0/1750811011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac100e80 0x7f48ac1012a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 --2- 172.21.15.178:0/1750811011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac102060 0x7f48ac1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 -- 172.21.15.178:0/1750811011 >> 172.21.15.178:0/1750811011 conn(0x7f48ac0fc400 msgr2=0x7f48ac0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:30.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 -- 172.21.15.178:0/1750811011 shutdown_connections 2023-10-16T14:33:30.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.190+0000 7f48b25ff700 1 -- 172.21.15.178:0/1750811011 wait complete. 2023-10-16T14:33:30.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.191+0000 7f48b25ff700 1 Processor -- start 2023-10-16T14:33:30.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.191+0000 7f48b25ff700 1 -- start start 2023-10-16T14:33:30.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac100e80 0x7f48ac198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102060 0x7f48ac1993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1a09e0 con 0x7f48ac102a20 2023-10-16T14:33:30.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1a0b20 con 0x7f48ac102060 2023-10-16T14:33:30.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b25ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac1a0cb0 con 0x7f48ac100e80 2023-10-16T14:33:30.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48abfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac100e80 0x7f48ac198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59528/0 (socket says 172.21.15.178:59528) 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 -- 172.21.15.178:0/3249601556 learned_addr learned my addr 172.21.15.178:0/3249601556 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48ab7fe700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102060 0x7f48ac1993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 -- 172.21.15.178:0/3249601556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac100e80 msgr2=0x7f48ac198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac100e80 0x7f48ac198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 -- 172.21.15.178:0/3249601556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102060 msgr2=0x7f48ac1993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102060 0x7f48ac1993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.192+0000 7f48b0b9c700 1 -- 172.21.15.178:0/3249601556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0095f0 con 0x7f48ac102a20 2023-10-16T14:33:30.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48b0b9c700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48a400ef30 tx=0x7f48a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:30.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4018070 con 0x7f48ac102a20 2023-10-16T14:33:30.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48a400f040 con 0x7f48ac102a20 2023-10-16T14:33:30.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400cd00 con 0x7f48ac102a20 2023-10-16T14:33:30.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac1a0f30 con 0x7f48ac102a20 2023-10-16T14:33:30.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.193+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac1a1360 con 0x7f48ac102a20 2023-10-16T14:33:30.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.194+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac10c740 con 0x7f48ac102a20 2023-10-16T14:33:30.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.196+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f48a4003a00 con 0x7f48ac102a20 2023-10-16T14:33:30.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.197+0000 7f48a97fa700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 0x7f48940a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.197+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f48a40ccce0 con 0x7f48ac102a20 2023-10-16T14:33:30.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.198+0000 7f48abfff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 0x7f48940a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.198+0000 7f48abfff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 0x7f48940a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b2e0 tx=0x7f489c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:30.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.201+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f48a4095ea0 con 0x7f48ac102a20 2023-10-16T14:33:30.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.404+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48ac1a1550 con 0x7f48ac102a20 2023-10-16T14:33:30.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.405+0000 7f48a97fa700 1 -- 172.21.15.178:0/3249601556 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48a40994c0 con 0x7f48ac102a20 2023-10-16T14:33:30.409 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:33:30.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 msgr2=0x7f48940a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 0x7f48940a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f489c00b2e0 tx=0x7f489c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 msgr2=0x7f48ac1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f48a400ef30 tx=0x7f48a400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 shutdown_connections 2023-10-16T14:33:30.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f48940a1510 0x7f48940a39d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48ac102a20 0x7f48ac1a0410 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48ac100e80 0x7f48ac198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 --2- 172.21.15.178:0/3249601556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f48ac102060 0x7f48ac1993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 >> 172.21.15.178:0/3249601556 conn(0x7f48ac0fc400 msgr2=0x7f48ac0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:30.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 shutdown_connections 2023-10-16T14:33:30.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.407+0000 7f48b25ff700 1 -- 172.21.15.178:0/3249601556 wait complete. 2023-10-16T14:33:30.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:33:30.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.625+0000 7fe997dff700 1 -- 172.21.15.178:0/496556694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102080 msgr2=0x7fe990102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.625+0000 7fe997dff700 1 --2- 172.21.15.178:0/496556694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102080 0x7fe990102500 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe980009990 tx=0x7fe980009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 -- 172.21.15.178:0/496556694 shutdown_connections 2023-10-16T14:33:30.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 --2- 172.21.15.178:0/496556694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102a40 0x7fe990109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 --2- 172.21.15.178:0/496556694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 0x7fe9901012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 --2- 172.21.15.178:0/496556694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102080 0x7fe990102500 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 -- 172.21.15.178:0/496556694 >> 172.21.15.178:0/496556694 conn(0x7fe9900fc420 msgr2=0x7fe9900fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:30.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 -- 172.21.15.178:0/496556694 shutdown_connections 2023-10-16T14:33:30.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.626+0000 7fe997dff700 1 -- 172.21.15.178:0/496556694 wait complete. 2023-10-16T14:33:30.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 Processor -- start 2023-10-16T14:33:30.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 -- start start 2023-10-16T14:33:30.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 0x7fe990198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102080 0x7fe990199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901a09d0 con 0x7fe990102a40 2023-10-16T14:33:30.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901a0b10 con 0x7fe990102080 2023-10-16T14:33:30.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe997dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9901a0ca0 con 0x7fe990100ea0 2023-10-16T14:33:30.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe99639c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe99639c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59544/0 (socket says 172.21.15.178:59544) 2023-10-16T14:33:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe995b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 0x7fe990198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe99639c700 1 -- 172.21.15.178:0/2532948400 learned_addr learned my addr 172.21.15.178:0/2532948400 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:30.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.627+0000 7fe99539a700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102080 0x7fe990199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 -- 172.21.15.178:0/2532948400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 msgr2=0x7fe990198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 0x7fe990198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 -- 172.21.15.178:0/2532948400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102080 msgr2=0x7fe990199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102080 0x7fe990199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 -- 172.21.15.178:0/2532948400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9800095f0 con 0x7fe990102a40 2023-10-16T14:33:30.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe99639c700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe988007ee0 tx=0x7fe98800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:30.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988017070 con 0x7fe990102a40 2023-10-16T14:33:30.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98800f040 con 0x7fe990102a40 2023-10-16T14:33:30.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9901a0f80 con 0x7fe990102a40 2023-10-16T14:33:30.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988005580 con 0x7fe990102a40 2023-10-16T14:33:30.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.628+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901a13b0 con 0x7fe990102a40 2023-10-16T14:33:30.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.629+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99010c760 con 0x7fe990102a40 2023-10-16T14:33:30.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.631+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fe98800db20 con 0x7fe990102a40 2023-10-16T14:33:30.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.632+0000 7fe986ffd700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 0x7fe97c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:30.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.633+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9880cc9c0 con 0x7fe990102a40 2023-10-16T14:33:30.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.633+0000 7fe995b9b700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 0x7fe97c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:30.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.633+0000 7fe995b9b700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 0x7fe97c0a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe990101bf0 tx=0x7fe98c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:30.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.637+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe988095af0 con 0x7fe990102a40 2023-10-16T14:33:30.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.837+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9901a15a0 con 0x7fe990102a40 2023-10-16T14:33:30.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.837+0000 7fe986ffd700 1 -- 172.21.15.178:0/2532948400 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe988099110 con 0x7fe990102a40 2023-10-16T14:33:30.841 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:33:30.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.839+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 msgr2=0x7fe97c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.839+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 0x7fe97c0a39d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe990101bf0 tx=0x7fe98c009260 comp rx=0 tx=0).stop 2023-10-16T14:33:30.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 msgr2=0x7fe9901a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:30.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe988007ee0 tx=0x7fe98800d3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 shutdown_connections 2023-10-16T14:33:30.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe97c0a1510 0x7fe97c0a39d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe990102a40 0x7fe9901a0400 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe990100ea0 0x7fe990198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 --2- 172.21.15.178:0/2532948400 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe990102080 0x7fe990199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:30.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 >> 172.21.15.178:0/2532948400 conn(0x7fe9900fc420 msgr2=0x7fe9900fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:30.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 shutdown_connections 2023-10-16T14:33:30.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:30.840+0000 7fe997dff700 1 -- 172.21.15.178:0/2532948400 wait complete. 2023-10-16T14:33:30.863 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:33:31.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:30 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1039625194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:31.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:30 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3249601556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:31.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 -- 172.21.15.178:0/158453556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102060 msgr2=0x7fa6341024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 --2- 172.21.15.178:0/158453556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102060 0x7fa6341024e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009990 tx=0x7fa61c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 -- 172.21.15.178:0/158453556 shutdown_connections 2023-10-16T14:33:31.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 --2- 172.21.15.178:0/158453556 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634102a20 0x7fa634109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 --2- 172.21.15.178:0/158453556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634100e80 0x7fa6341012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 --2- 172.21.15.178:0/158453556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102060 0x7fa6341024e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 -- 172.21.15.178:0/158453556 >> 172.21.15.178:0/158453556 conn(0x7fa6340fc400 msgr2=0x7fa6340fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:31.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 -- 172.21.15.178:0/158453556 shutdown_connections 2023-10-16T14:33:31.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.060+0000 7fa6395ff700 1 -- 172.21.15.178:0/158453556 wait complete. 2023-10-16T14:33:31.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.061+0000 7fa6395ff700 1 Processor -- start 2023-10-16T14:33:31.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.061+0000 7fa6395ff700 1 -- start start 2023-10-16T14:33:31.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.061+0000 7fa6395ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634100e80 0x7fa634196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6395ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 0x7fa634197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6395ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6395ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63419e760 con 0x7fa634102a20 2023-10-16T14:33:31.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6395ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63419e8d0 con 0x7fa634100e80 2023-10-16T14:33:31.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6395ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63419ea90 con 0x7fa634102060 2023-10-16T14:33:31.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa62bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 0x7fa634197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59562/0 (socket says 172.21.15.178:59562) 2023-10-16T14:33:31.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 -- 172.21.15.178:0/3031086322 learned_addr learned my addr 172.21.15.178:0/3031086322 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:31.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa632ffd700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634100e80 0x7fa634196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 -- 172.21.15.178:0/3031086322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 msgr2=0x7fa634197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 0x7fa634197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 -- 172.21.15.178:0/3031086322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634100e80 msgr2=0x7fa634196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634100e80 0x7fa634196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 -- 172.21.15.178:0/3031086322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0095f0 con 0x7fa634102a20 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa62bfff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 0x7fa634197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.062+0000 7fa6337fe700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00ef30 tx=0x7fa62c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.063+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c018070 con 0x7fa634102a20 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.063+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63419ed10 con 0x7fa634102a20 2023-10-16T14:33:31.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.063+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa62c00f040 con 0x7fa634102a20 2023-10-16T14:33:31.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.063+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00cd00 con 0x7fa634102a20 2023-10-16T14:33:31.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.063+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63419f1d0 con 0x7fa634102a20 2023-10-16T14:33:31.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.064+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa634107e80 con 0x7fa634102a20 2023-10-16T14:33:31.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.066+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fa62c003a00 con 0x7fa634102a20 2023-10-16T14:33:31.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.067+0000 7fa630ff9700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 0x7fa6200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.067+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa62c0ccda0 con 0x7fa634102a20 2023-10-16T14:33:31.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.067+0000 7fa632ffd700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 0x7fa6200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.068+0000 7fa632ffd700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 0x7fa6200a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6341980f0 tx=0x7fa624009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.071+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa62c095f60 con 0x7fa634102a20 2023-10-16T14:33:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:30 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1039625194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:31.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:30 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3249601556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:30 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1039625194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:31.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:30 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3249601556' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:31.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.273+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa63410c740 con 0x7fa634102a20 2023-10-16T14:33:31.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.274+0000 7fa630ff9700 1 -- 172.21.15.178:0/3031086322 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa62c099580 con 0x7fa634102a20 2023-10-16T14:33:31.278 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:33:31.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 msgr2=0x7fa6200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 0x7fa6200a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa6341980f0 tx=0x7fa624009260 comp rx=0 tx=0).stop 2023-10-16T14:33:31.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 msgr2=0x7fa63419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00ef30 tx=0x7fa62c00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 shutdown_connections 2023-10-16T14:33:31.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6200a1510 0x7fa6200a39d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa634102a20 0x7fa63419e190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa634100e80 0x7fa634196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 --2- 172.21.15.178:0/3031086322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa634102060 0x7fa634197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 >> 172.21.15.178:0/3031086322 conn(0x7fa6340fc400 msgr2=0x7fa6340fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:31.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 shutdown_connections 2023-10-16T14:33:31.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.276+0000 7fa6395ff700 1 -- 172.21.15.178:0/3031086322 wait complete. 2023-10-16T14:33:31.299 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:33:31.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 -- 172.21.15.178:0/3115718251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102a20 msgr2=0x7f6e10109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3115718251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102a20 0x7f6e10109340 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800b0a0 tx=0x7f6e0800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 -- 172.21.15.178:0/3115718251 shutdown_connections 2023-10-16T14:33:31.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3115718251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102a20 0x7f6e10109340 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3115718251 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10100e80 0x7f6e101012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3115718251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10102060 0x7f6e101024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 -- 172.21.15.178:0/3115718251 >> 172.21.15.178:0/3115718251 conn(0x7f6e100fc400 msgr2=0x7f6e100fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:31.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 -- 172.21.15.178:0/3115718251 shutdown_connections 2023-10-16T14:33:31.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.496+0000 7f6e157ff700 1 -- 172.21.15.178:0/3115718251 wait complete. 2023-10-16T14:33:31.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.497+0000 7f6e157ff700 1 Processor -- start 2023-10-16T14:33:31.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.497+0000 7f6e157ff700 1 -- start start 2023-10-16T14:33:31.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.497+0000 7f6e157ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10100e80 0x7f6e10198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.497+0000 7f6e157ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.497+0000 7f6e157ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10102a20 0x7f6e101a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e157ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e101a0a80 con 0x7f6e10102060 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e157ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e101a0bc0 con 0x7f6e10102a20 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e157ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e101a0d50 con 0x7f6e10100e80 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59568/0 (socket says 172.21.15.178:59568) 2023-10-16T14:33:31.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 -- 172.21.15.178:0/3016333549 learned_addr learned my addr 172.21.15.178:0/3016333549 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:31.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0effd700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10100e80 0x7f6e10198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0f7fe700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10102a20 0x7f6e101a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 -- 172.21.15.178:0/3016333549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10100e80 msgr2=0x7f6e10198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10100e80 0x7f6e10198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 -- 172.21.15.178:0/3016333549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10102a20 msgr2=0x7f6e101a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10102a20 0x7f6e101a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 -- 172.21.15.178:0/3016333549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08009d00 con 0x7f6e10102060 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.498+0000 7f6e0e7fc700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e970 tx=0x7f6e0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.499+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04009790 con 0x7f6e10102060 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.499+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e0400c7e0 con 0x7f6e10102060 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.499+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e101a0fd0 con 0x7f6e10102060 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.499+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04003c70 con 0x7f6e10102060 2023-10-16T14:33:31.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.499+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e101a14b0 con 0x7f6e10102060 2023-10-16T14:33:31.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.500+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1010c740 con 0x7f6e10102060 2023-10-16T14:33:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.502+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f6e0400cc50 con 0x7f6e10102060 2023-10-16T14:33:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.503+0000 7f6df7fff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 0x7f6df80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.503+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6e04014070 con 0x7f6e10102060 2023-10-16T14:33:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.503+0000 7f6e0effd700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 0x7f6df80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.504+0000 7f6e0effd700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 0x7f6df80a39d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e10101bd0 tx=0x7f6e00007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.507+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6e040964c0 con 0x7f6e10102060 2023-10-16T14:33:31.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.711+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e101a1700 con 0x7f6e10102060 2023-10-16T14:33:31.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.711+0000 7f6df7fff700 1 -- 172.21.15.178:0/3016333549 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e04003dd0 con 0x7f6e10102060 2023-10-16T14:33:31.716 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:33:31.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 msgr2=0x7f6df80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 0x7f6df80a39d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e10101bd0 tx=0x7f6e00007400 comp rx=0 tx=0).stop 2023-10-16T14:33:31.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 msgr2=0x7f6e10199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e970 tx=0x7f6e0400ec80 comp rx=0 tx=0).stop 2023-10-16T14:33:31.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 shutdown_connections 2023-10-16T14:33:31.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6df80a1510 0x7f6df80a39d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6e10102a20 0x7f6e101a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e10100e80 0x7f6e10198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 --2- 172.21.15.178:0/3016333549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6e10102060 0x7f6e10199440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 >> 172.21.15.178:0/3016333549 conn(0x7f6e100fc400 msgr2=0x7f6e100fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:31.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 shutdown_connections 2023-10-16T14:33:31.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.714+0000 7f6e157ff700 1 -- 172.21.15.178:0/3016333549 wait complete. 2023-10-16T14:33:31.737 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:33:31.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 -- 172.21.15.178:0/1237143947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 msgr2=0x7fdbb410b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/1237143947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb410b240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00b0a0 tx=0x7fdbac00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 -- 172.21.15.178:0/1237143947 shutdown_connections 2023-10-16T14:33:31.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/1237143947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb410b240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/1237143947 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4074ef0 0x7fdbb4073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/1237143947 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4073910 0x7fdbb4073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.932+0000 7fdbbbdff700 1 -- 172.21.15.178:0/1237143947 >> 172.21.15.178:0/1237143947 conn(0x7fdbb40fc420 msgr2=0x7fdbb40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:31.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.933+0000 7fdbbbdff700 1 -- 172.21.15.178:0/1237143947 shutdown_connections 2023-10-16T14:33:31.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.933+0000 7fdbbbdff700 1 -- 172.21.15.178:0/1237143947 wait complete. 2023-10-16T14:33:31.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 Processor -- start 2023-10-16T14:33:31.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 -- start start 2023-10-16T14:33:31.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4074ef0 0x7fdbb4199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb41a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a0970 con 0x7fdbb4073910 2023-10-16T14:33:31.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a0ab0 con 0x7fdbb4074ef0 2023-10-16T14:33:31.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbbbdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41a0c40 con 0x7fdbb4108db0 2023-10-16T14:33:31.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbb9b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbb9b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59592/0 (socket says 172.21.15.178:59592) 2023-10-16T14:33:31.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbb9b9b700 1 -- 172.21.15.178:0/875645362 learned_addr learned my addr 172.21.15.178:0/875645362 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:31.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbb939a700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4074ef0 0x7fdbb4199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.934+0000 7fdbba39c700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb41a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 -- 172.21.15.178:0/875645362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 msgr2=0x7fdbb41a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb41a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 -- 172.21.15.178:0/875645362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4074ef0 msgr2=0x7fdbb4199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4074ef0 0x7fdbb4199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 -- 172.21.15.178:0/875645362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009520 con 0x7fdbb4073910 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbb9b9b700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000eb30 tx=0x7fdbb000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0009880 con 0x7fdbb4073910 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb00044e0 con 0x7fdbb4073910 2023-10-16T14:33:31.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0003d80 con 0x7fdbb4073910 2023-10-16T14:33:31.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb0009520 con 0x7fdbb4073910 2023-10-16T14:33:31.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.935+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac009d00 con 0x7fdbb4073910 2023-10-16T14:33:31.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.936+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb4107460 con 0x7fdbb4073910 2023-10-16T14:33:31.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.938+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fdbb0003f20 con 0x7fdbb4073910 2023-10-16T14:33:31.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.939+0000 7fdbaaffd700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 0x7fdba00a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:31.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.940+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdbb0014070 con 0x7fdbb4073910 2023-10-16T14:33:31.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.940+0000 7fdbb939a700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 0x7fdba00a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:31.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.940+0000 7fdbb939a700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 0x7fdba00a3b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4074c10 tx=0x7fdba4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:31.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:31.943+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fdbb0096870 con 0x7fdbb4073910 2023-10-16T14:33:32.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:31 smithi178 ceph-mon[98495]: pgmap v333: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:32.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:31 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2532948400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:31 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3031086322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:31 smithi186 ceph-mon[134892]: pgmap v333: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:31 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2532948400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:31 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3031086322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:31 smithi187 ceph-mon[135297]: pgmap v333: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:33:32.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:31 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2532948400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:31 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3031086322' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:32.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.191+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbb402d020 con 0x7fdbb4073910 2023-10-16T14:33:32.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.192+0000 7fdbaaffd700 1 -- 172.21.15.178:0/875645362 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7fdbb0099e90 con 0x7fdbb4073910 2023-10-16T14:33:32.195 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:32.196 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:33:32.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 msgr2=0x7fdba00a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 0x7fdba00a3b00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4074c10 tx=0x7fdba4009260 comp rx=0 tx=0).stop 2023-10-16T14:33:32.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 msgr2=0x7fdbb4198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000eb30 tx=0x7fdbb000ee40 comp rx=0 tx=0).stop 2023-10-16T14:33:32.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 shutdown_connections 2023-10-16T14:33:32.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdba00a1640 0x7fdba00a3b00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdbb4108db0 0x7fdbb41a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbb4073910 0x7fdbb4198df0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 --2- 172.21.15.178:0/875645362 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdbb4074ef0 0x7fdbb4199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 >> 172.21.15.178:0/875645362 conn(0x7fdbb40fc420 msgr2=0x7fdbb40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:32.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 shutdown_connections 2023-10-16T14:33:32.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.194+0000 7fdbbbdff700 1 -- 172.21.15.178:0/875645362 wait complete. 2023-10-16T14:33:32.200 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:33:32.217 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:33:32.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 -- 172.21.15.178:0/365130319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80102080 msgr2=0x7fae80102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 --2- 172.21.15.178:0/365130319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80102080 0x7fae80102500 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fae74009990 tx=0x7fae74009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 -- 172.21.15.178:0/365130319 shutdown_connections 2023-10-16T14:33:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 --2- 172.21.15.178:0/365130319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102a40 0x7fae80109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 --2- 172.21.15.178:0/365130319 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80100ea0 0x7fae801012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 --2- 172.21.15.178:0/365130319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80102080 0x7fae80102500 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 -- 172.21.15.178:0/365130319 >> 172.21.15.178:0/365130319 conn(0x7fae800fc420 msgr2=0x7fae800fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 -- 172.21.15.178:0/365130319 shutdown_connections 2023-10-16T14:33:32.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.410+0000 7fae859ff700 1 -- 172.21.15.178:0/365130319 wait complete. 2023-10-16T14:33:32.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.411+0000 7fae859ff700 1 Processor -- start 2023-10-16T14:33:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 -- start start 2023-10-16T14:33:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80102a40 0x7fae801a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a08e0 con 0x7fae80102080 2023-10-16T14:33:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a0a50 con 0x7fae80102a40 2023-10-16T14:33:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae859ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae801a0c10 con 0x7fae80100ea0 2023-10-16T14:33:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42230/0 (socket says 172.21.15.178:42230) 2023-10-16T14:33:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7effd700 1 -- 172.21.15.178:0/2474868956 learned_addr learned my addr 172.21.15.178:0/2474868956 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59622/0 (socket says 172.21.15.178:59622) 2023-10-16T14:33:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.412+0000 7fae7f7fe700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80102a40 0x7fae801a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 -- 172.21.15.178:0/2474868956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 msgr2=0x7fae80198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 -- 172.21.15.178:0/2474868956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80102a40 msgr2=0x7fae801a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80102a40 0x7fae801a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 -- 172.21.15.178:0/2474868956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae740095f0 con 0x7fae80102080 2023-10-16T14:33:32.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7effd700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae7e7fc700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b680 tx=0x7fae74009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7401d070 con 0x7fae80102080 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae74021410 con 0x7fae80102080 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae801a0e90 con 0x7fae80102080 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74017650 con 0x7fae80102080 2023-10-16T14:33:32.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.413+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae801a1200 con 0x7fae80102080 2023-10-16T14:33:32.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.414+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8010c760 con 0x7fae80102080 2023-10-16T14:33:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.416+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fae74021810 con 0x7fae80102080 2023-10-16T14:33:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.418+0000 7fae849fd700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 0x7fae680a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.418+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fae740ced80 con 0x7fae80102080 2023-10-16T14:33:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.418+0000 7fae7effd700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 0x7fae680a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.418+0000 7fae7effd700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 0x7fae680a3a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fae8019a270 tx=0x7fae70009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:32.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.420+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae74097f40 con 0x7fae80102080 2023-10-16T14:33:32.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.622+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae801a13f0 con 0x7fae80102080 2023-10-16T14:33:32.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.624+0000 7fae849fd700 1 -- 172.21.15.178:0/2474868956 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fae7409b560 con 0x7fae80102080 2023-10-16T14:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:32.628 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:33:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 msgr2=0x7fae680a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 0x7fae680a3a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fae8019a270 tx=0x7fae70009260 comp rx=0 tx=0).stop 2023-10-16T14:33:32.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 msgr2=0x7fae801992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b680 tx=0x7fae74009580 comp rx=0 tx=0).stop 2023-10-16T14:33:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 shutdown_connections 2023-10-16T14:33:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae680a15c0 0x7fae680a3a80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae80102a40 0x7fae801a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae80100ea0 0x7fae80198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 --2- 172.21.15.178:0/2474868956 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae80102080 0x7fae801992e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 >> 172.21.15.178:0/2474868956 conn(0x7fae800fc420 msgr2=0x7fae801079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 shutdown_connections 2023-10-16T14:33:32.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.626+0000 7fae859ff700 1 -- 172.21.15.178:0/2474868956 wait complete. 2023-10-16T14:33:32.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:33:32.650 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:33:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 -- 172.21.15.178:0/3911475595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102080 msgr2=0x7f70f0102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 --2- 172.21.15.178:0/3911475595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102080 0x7f70f0102500 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f70e0009990 tx=0x7f70e0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 -- 172.21.15.178:0/3911475595 shutdown_connections 2023-10-16T14:33:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 --2- 172.21.15.178:0/3911475595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102a40 0x7f70f0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 --2- 172.21.15.178:0/3911475595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 --2- 172.21.15.178:0/3911475595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102080 0x7f70f0102500 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 -- 172.21.15.178:0/3911475595 >> 172.21.15.178:0/3911475595 conn(0x7f70f00fc420 msgr2=0x7f70f00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 -- 172.21.15.178:0/3911475595 shutdown_connections 2023-10-16T14:33:32.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.850+0000 7f70f87ff700 1 -- 172.21.15.178:0/3911475595 wait complete. 2023-10-16T14:33:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.851+0000 7f70f87ff700 1 Processor -- start 2023-10-16T14:33:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.851+0000 7f70f87ff700 1 -- start start 2023-10-16T14:33:32.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.851+0000 7f70f87ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f87ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102080 0x7f70f00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f87ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f87ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f0073f80 con 0x7f70f0102a40 2023-10-16T14:33:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f87ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00740f0 con 0x7f70f0102080 2023-10-16T14:33:32.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f87ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f01a0cb0 con 0x7f70f0100ea0 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f659b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59654/0 (socket says 172.21.15.178:59654) 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 -- 172.21.15.178:0/4141952750 learned_addr learned my addr 172.21.15.178:0/4141952750 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f5d9a700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102080 0x7f70f00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 -- 172.21.15.178:0/4141952750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 msgr2=0x7f70f0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 -- 172.21.15.178:0/4141952750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102080 msgr2=0x7f70f00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102080 0x7f70f00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f6d9c700 1 -- 172.21.15.178:0/4141952750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00095f0 con 0x7f70f0102a40 2023-10-16T14:33:32.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.852+0000 7f70f659b700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70f6d9c700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f70e800ef30 tx=0x7f70e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8018070 con 0x7f70f0102a40 2023-10-16T14:33:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70e800f040 con 0x7f70f0102a40 2023-10-16T14:33:32.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f01a0f30 con 0x7f70f0102a40 2023-10-16T14:33:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e800cd00 con 0x7f70f0102a40 2023-10-16T14:33:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.853+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f01a13b0 con 0x7f70f0102a40 2023-10-16T14:33:32.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.854+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f010c760 con 0x7f70f0102a40 2023-10-16T14:33:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.856+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f70e8003a00 con 0x7f70f0102a40 2023-10-16T14:33:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.857+0000 7f70e77fe700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 0x7f70dc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:32.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.857+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f70e80ccd50 con 0x7f70f0102a40 2023-10-16T14:33:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.857+0000 7f70f659b700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 0x7f70dc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.858+0000 7f70f659b700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 0x7f70dc0a3a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70f0101bf0 tx=0x7f70ec009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:32.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:32.861+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f70e8095f10 con 0x7f70f0102a40 2023-10-16T14:33:33.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:32 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3016333549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:33.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:32 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/875645362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:32 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2474868956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.061+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70f0107e70 con 0x7f70f0102a40 2023-10-16T14:33:33.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.062+0000 7f70e77fe700 1 -- 172.21.15.178:0/4141952750 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f70e8099530 con 0x7f70f0102a40 2023-10-16T14:33:33.066 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:33:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 msgr2=0x7f70dc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 0x7f70dc0a3a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f70f0101bf0 tx=0x7f70ec009260 comp rx=0 tx=0).stop 2023-10-16T14:33:33.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 msgr2=0x7f70f00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f70e800ef30 tx=0x7f70e800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 shutdown_connections 2023-10-16T14:33:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70dc0a1590 0x7f70dc0a3a50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f0102a40 0x7f70f00737e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70f0100ea0 0x7f70f0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 --2- 172.21.15.178:0/4141952750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70f0102080 0x7f70f00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 >> 172.21.15.178:0/4141952750 conn(0x7f70f00fc420 msgr2=0x7f70f00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 shutdown_connections 2023-10-16T14:33:33.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.064+0000 7f70f87ff700 1 -- 172.21.15.178:0/4141952750 wait complete. 2023-10-16T14:33:33.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:32 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3016333549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:33.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:32 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/875645362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:32 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2474868956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.087 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:33:33.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:32 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3016333549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:33.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:32 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/875645362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:32 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2474868956' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:33.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 -- 172.21.15.178:0/219646733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8102080 msgr2=0x7fc8a8102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/219646733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8102080 0x7fc8a8102500 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc898009990 tx=0x7fc898009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 -- 172.21.15.178:0/219646733 shutdown_connections 2023-10-16T14:33:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/219646733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 0x7fc8a8109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/219646733 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8100ea0 0x7fc8a81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/219646733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8102080 0x7fc8a8102500 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 -- 172.21.15.178:0/219646733 >> 172.21.15.178:0/219646733 conn(0x7fc8a80fc420 msgr2=0x7fc8a80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 -- 172.21.15.178:0/219646733 shutdown_connections 2023-10-16T14:33:33.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.285+0000 7fc8b07ff700 1 -- 172.21.15.178:0/219646733 wait complete. 2023-10-16T14:33:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.286+0000 7fc8b07ff700 1 Processor -- start 2023-10-16T14:33:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.286+0000 7fc8b07ff700 1 -- start start 2023-10-16T14:33:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 0x7fc8a8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 0x7fc8a81a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81a09d0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81a0b10 con 0x7fc8a8102080 2023-10-16T14:33:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8b07ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81a0ca0 con 0x7fc8a8102a40 2023-10-16T14:33:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59672/0 (socket says 172.21.15.178:59672) 2023-10-16T14:33:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 -- 172.21.15.178:0/4179735035 learned_addr learned my addr 172.21.15.178:0/4179735035 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8aed9c700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 0x7fc8a81a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8add9a700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 0x7fc8a8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 -- 172.21.15.178:0/4179735035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 msgr2=0x7fc8a81a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 0x7fc8a81a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 -- 172.21.15.178:0/4179735035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 msgr2=0x7fc8a8199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 0x7fc8a8199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8ae59b700 1 -- 172.21.15.178:0/4179735035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8980095f0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.287+0000 7fc8add9a700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 0x7fc8a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc8ae59b700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e970 tx=0x7fc8a400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:33.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009790 con 0x7fc8a8100ea0 2023-10-16T14:33:33.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8a400eec0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a81a0f80 con 0x7fc8a8100ea0 2023-10-16T14:33:33.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4003cf0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.288+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81a13b0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.289+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a8193550 con 0x7fc8a8100ea0 2023-10-16T14:33:33.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.291+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fc8a400ccd0 con 0x7fc8a8100ea0 2023-10-16T14:33:33.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.292+0000 7fc89f7fe700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 0x7fc8940a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.292+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc8a4014070 con 0x7fc8a8100ea0 2023-10-16T14:33:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.293+0000 7fc8add9a700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 0x7fc8940a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.293+0000 7fc8add9a700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 0x7fc8940a3a50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc89800b2e0 tx=0x7fc898005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:33.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.296+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc8a4096590 con 0x7fc8a8100ea0 2023-10-16T14:33:33.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.495+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8a802d020 con 0x7fc8a8100ea0 2023-10-16T14:33:33.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.496+0000 7fc89f7fe700 1 -- 172.21.15.178:0/4179735035 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8a4094e60 con 0x7fc8a8100ea0 2023-10-16T14:33:33.500 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:33:33.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 msgr2=0x7fc8940a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 0x7fc8940a3a50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc89800b2e0 tx=0x7fc898005fd0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 msgr2=0x7fc8a8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e970 tx=0x7fc8a400ec80 comp rx=0 tx=0).stop 2023-10-16T14:33:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 shutdown_connections 2023-10-16T14:33:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc8940a1590 0x7fc8940a3a50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8a8102a40 0x7fc8a81a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc8a8100ea0 0x7fc8a8198e50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 --2- 172.21.15.178:0/4179735035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc8a8102080 0x7fc8a8199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.498+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 >> 172.21.15.178:0/4179735035 conn(0x7fc8a80fc420 msgr2=0x7fc8a8107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:33.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.499+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 shutdown_connections 2023-10-16T14:33:33.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.499+0000 7fc8b07ff700 1 -- 172.21.15.178:0/4179735035 wait complete. 2023-10-16T14:33:33.521 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:33:33.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 -- 172.21.15.178:0/1142143490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c074ef0 msgr2=0x7f3f7c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 --2- 172.21.15.178:0/1142143490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c073340 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009990 tx=0x7f3f6c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 -- 172.21.15.178:0/1142143490 shutdown_connections 2023-10-16T14:33:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 --2- 172.21.15.178:0/1142143490 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c108d90 0x7f3f7c10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 --2- 172.21.15.178:0/1142143490 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c073340 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 --2- 172.21.15.178:0/1142143490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c073910 0x7f3f7c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.720+0000 7f3f81bff700 1 -- 172.21.15.178:0/1142143490 >> 172.21.15.178:0/1142143490 conn(0x7f3f7c0fc400 msgr2=0x7f3f7c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.721+0000 7f3f81bff700 1 -- 172.21.15.178:0/1142143490 shutdown_connections 2023-10-16T14:33:33.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.721+0000 7f3f81bff700 1 -- 172.21.15.178:0/1142143490 wait complete. 2023-10-16T14:33:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.721+0000 7f3f81bff700 1 Processor -- start 2023-10-16T14:33:33.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 -- start start 2023-10-16T14:33:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c108d90 0x7f3f7c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1a09d0 con 0x7f3f7c073910 2023-10-16T14:33:33.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1a0b10 con 0x7f3f7c074ef0 2023-10-16T14:33:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f81bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c1a0ca0 con 0x7f3f7c108d90 2023-10-16T14:33:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59684/0 (socket says 172.21.15.178:59684) 2023-10-16T14:33:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f7b7fe700 1 -- 172.21.15.178:0/3930021880 learned_addr learned my addr 172.21.15.178:0/3930021880 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:33.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f7affd700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.722+0000 7f3f7bfff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c108d90 0x7f3f7c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 -- 172.21.15.178:0/3930021880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c108d90 msgr2=0x7f3f7c1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c108d90 0x7f3f7c1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 -- 172.21.15.178:0/3930021880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c074ef0 msgr2=0x7f3f7c199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 -- 172.21.15.178:0/3930021880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f70009520 con 0x7f3f7c073910 2023-10-16T14:33:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f7b7fe700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00b680 tx=0x7f3f6c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c01d070 con 0x7f3f7c073910 2023-10-16T14:33:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6c022470 con 0x7f3f7c073910 2023-10-16T14:33:33.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c0095f0 con 0x7f3f7c073910 2023-10-16T14:33:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c017650 con 0x7f3f7c073910 2023-10-16T14:33:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.723+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c1a11b0 con 0x7f3f7c073910 2023-10-16T14:33:33.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.724+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c107440 con 0x7f3f7c073910 2023-10-16T14:33:33.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.726+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f3f6c0225e0 con 0x7f3f7c073910 2023-10-16T14:33:33.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.727+0000 7f3f78ff9700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 0x7f3f640a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:33.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.728+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3f6c0ce7b0 con 0x7f3f7c073910 2023-10-16T14:33:33.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.728+0000 7f3f7affd700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 0x7f3f640a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.728+0000 7f3f7affd700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 0x7f3f640a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c074c10 tx=0x7f3f70009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:33.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.731+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3f6c097970 con 0x7f3f7c073910 2023-10-16T14:33:33.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:33 smithi178 ceph-mon[98495]: pgmap v334: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:33.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:33 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4141952750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:33.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:33 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4179735035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:33.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.932+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f7c02d020 con 0x7f3f7c073910 2023-10-16T14:33:33.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.932+0000 7f3f78ff9700 1 -- 172.21.15.178:0/3930021880 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f6c09af90 con 0x7f3f7c073910 2023-10-16T14:33:33.937 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:33:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 msgr2=0x7f3f640a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 0x7f3f640a39d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c074c10 tx=0x7f3f70009260 comp rx=0 tx=0).stop 2023-10-16T14:33:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 msgr2=0x7f3f7c198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:33.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00b680 tx=0x7f3f6c0057b0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 shutdown_connections 2023-10-16T14:33:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f640a1510 0x7f3f640a39d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f7c108d90 0x7f3f7c1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c073910 0x7f3f7c198e50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 --2- 172.21.15.178:0/3930021880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3f7c074ef0 0x7f3f7c199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 >> 172.21.15.178:0/3930021880 conn(0x7f3f7c0fc400 msgr2=0x7f3f7c0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 shutdown_connections 2023-10-16T14:33:33.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:33.935+0000 7f3f81bff700 1 -- 172.21.15.178:0/3930021880 wait complete. 2023-10-16T14:33:33.958 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:33:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:33 smithi186 ceph-mon[134892]: pgmap v334: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:33 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4141952750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:33 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4179735035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:34.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 -- 172.21.15.178:0/1706430634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102060 msgr2=0x7f24841024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 --2- 172.21.15.178:0/1706430634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102060 0x7f24841024e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f24740098f0 tx=0x7f2474009c00 comp rx=0 tx=0).stop 2023-10-16T14:33:34.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 -- 172.21.15.178:0/1706430634 shutdown_connections 2023-10-16T14:33:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 --2- 172.21.15.178:0/1706430634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484102a20 0x7f2484109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 --2- 172.21.15.178:0/1706430634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484100e80 0x7f24841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 --2- 172.21.15.178:0/1706430634 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102060 0x7f24841024e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 -- 172.21.15.178:0/1706430634 >> 172.21.15.178:0/1706430634 conn(0x7f24840fc400 msgr2=0x7f24840fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:34.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 -- 172.21.15.178:0/1706430634 shutdown_connections 2023-10-16T14:33:34.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.155+0000 7f248b3ff700 1 -- 172.21.15.178:0/1706430634 wait complete. 2023-10-16T14:33:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 Processor -- start 2023-10-16T14:33:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 -- start start 2023-10-16T14:33:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484100e80 0x7f2484196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102a20 0x7f248419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e800 con 0x7f2484102060 2023-10-16T14:33:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419e970 con 0x7f2484102a20 2023-10-16T14:33:34.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.156+0000 7f248b3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248419eb30 con 0x7f2484100e80 2023-10-16T14:33:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59702/0 (socket says 172.21.15.178:59702) 2023-10-16T14:33:34.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248919b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484100e80 0x7f2484196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 -- 172.21.15.178:0/543174045 learned_addr learned my addr 172.21.15.178:0/543174045 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248999c700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102a20 0x7f248419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 -- 172.21.15.178:0/543174045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484100e80 msgr2=0x7f2484196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484100e80 0x7f2484196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 -- 172.21.15.178:0/543174045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102a20 msgr2=0x7f248419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102a20 0x7f248419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 -- 172.21.15.178:0/543174045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800095f0 con 0x7f2484102060 2023-10-16T14:33:34.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.157+0000 7f248899a700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2474009fd0 tx=0x7f247400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:34.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.158+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247401c070 con 0x7f2484102060 2023-10-16T14:33:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.158+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2474009520 con 0x7f2484102060 2023-10-16T14:33:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.158+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247400f7d0 con 0x7f2484102060 2023-10-16T14:33:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.158+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474003b00 con 0x7f2484102060 2023-10-16T14:33:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.158+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248419f050 con 0x7f2484102060 2023-10-16T14:33:34.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.159+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248410c770 con 0x7f2484102060 2023-10-16T14:33:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.161+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f2474020030 con 0x7f2484102060 2023-10-16T14:33:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.162+0000 7f247a7fc700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 0x7f24700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.162+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f24740ce2f0 con 0x7f2484102060 2023-10-16T14:33:34.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.162+0000 7f248919b700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 0x7f24700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.163+0000 7f248919b700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 0x7f24700a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2484197b60 tx=0x7f2480009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:34.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.166+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f24740974b0 con 0x7f2484102060 2023-10-16T14:33:34.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:33 smithi187 ceph-mon[135297]: pgmap v334: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:33:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:33 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4141952750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:33:34.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:33 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4179735035' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:33:34.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.367+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2484107eb0 con 0x7f2484102060 2023-10-16T14:33:34.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.367+0000 7f247a7fc700 1 -- 172.21.15.178:0/543174045 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f247409aad0 con 0x7f2484102060 2023-10-16T14:33:34.371 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:33:34.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 msgr2=0x7f24700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 0x7f24700a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2484197b60 tx=0x7f2480009310 comp rx=0 tx=0).stop 2023-10-16T14:33:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 msgr2=0x7f24841971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2474009fd0 tx=0x7f247400b790 comp rx=0 tx=0).stop 2023-10-16T14:33:34.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 shutdown_connections 2023-10-16T14:33:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f24700a1510 0x7f24700a39d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2484102a20 0x7f248419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2484100e80 0x7f2484196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 --2- 172.21.15.178:0/543174045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2484102060 0x7f24841971d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 >> 172.21.15.178:0/543174045 conn(0x7f24840fc400 msgr2=0x7f24840fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:34.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 shutdown_connections 2023-10-16T14:33:34.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.370+0000 7f248b3ff700 1 -- 172.21.15.178:0/543174045 wait complete. 2023-10-16T14:33:34.393 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:33:34.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/834952355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38101a00 msgr2=0x7f9a38103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/834952355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38101a00 0x7f9a38103df0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009990 tx=0x7f9a28009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/834952355 shutdown_connections 2023-10-16T14:33:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/834952355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38106c60 0x7f9a38109070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/834952355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38101a00 0x7f9a38103df0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/834952355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38104330 0x7f9a38106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/834952355 >> 172.21.15.178:0/834952355 conn(0x7f9a380fb3b0 msgr2=0x7f9a380fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/834952355 shutdown_connections 2023-10-16T14:33:34.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.586+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/834952355 wait complete. 2023-10-16T14:33:34.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 Processor -- start 2023-10-16T14:33:34.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 -- start start 2023-10-16T14:33:34.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38101a00 0x7f9a38196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 0x7f9a3819e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3819e690 con 0x7f9a38104330 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3819e7d0 con 0x7f9a38101a00 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a3d5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3819e960 con 0x7f9a38106c60 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.588+0000 7f9a367fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38101a00 0x7f9a38196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59714/0 (socket says 172.21.15.178:59714) 2023-10-16T14:33:34.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 -- 172.21.15.178:0/1468116100 learned_addr learned my addr 172.21.15.178:0/1468116100 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:34.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 0x7f9a3819e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 -- 172.21.15.178:0/1468116100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 msgr2=0x7f9a3819e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 0x7f9a3819e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 -- 172.21.15.178:0/1468116100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38101a00 msgr2=0x7f9a38196b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38101a00 0x7f9a38196b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 -- 172.21.15.178:0/1468116100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280095f0 con 0x7f9a38104330 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a377fe700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 0x7f9a3819e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:34.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a367fc700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9a0 tx=0x7f9a2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c009790 con 0x7f9a38104330 2023-10-16T14:33:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2c010430 con 0x7f9a38104330 2023-10-16T14:33:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3819ec40 con 0x7f9a38104330 2023-10-16T14:33:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.589+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c018680 con 0x7f9a38104330 2023-10-16T14:33:34.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.590+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3819f1a0 con 0x7f9a38104330 2023-10-16T14:33:34.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.591+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a380fcf30 con 0x7f9a38104330 2023-10-16T14:33:34.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.592+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f9a2c0105a0 con 0x7f9a38104330 2023-10-16T14:33:34.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.594+0000 7f9a1ffff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 0x7f9a200a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:34.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.594+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9a2c014070 con 0x7f9a38104330 2023-10-16T14:33:34.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.594+0000 7f9a36ffd700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 0x7f9a200a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:34.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.594+0000 7f9a36ffd700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 0x7f9a200a3b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800b2e0 tx=0x7f9a28005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:34.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.597+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9a2c0964b0 con 0x7f9a38104330 2023-10-16T14:33:34.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.845+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a3810c900 con 0x7f9a38104330 2023-10-16T14:33:34.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.846+0000 7f9a1ffff700 1 -- 172.21.15.178:0/1468116100 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7f9a2c099ad0 con 0x7f9a38104330 2023-10-16T14:33:34.850 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:34.850 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:33:34.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 msgr2=0x7f9a200a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 0x7f9a200a3b00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9a2800b2e0 tx=0x7f9a28005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 msgr2=0x7f9a38197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:34.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00e9a0 tx=0x7f9a2c00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 shutdown_connections 2023-10-16T14:33:34.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.848+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a200a1640 0x7f9a200a3b00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a38106c60 0x7f9a3819e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a38101a00 0x7f9a38196b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 --2- 172.21.15.178:0/1468116100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a38104330 0x7f9a38197050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:34.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 >> 172.21.15.178:0/1468116100 conn(0x7f9a380fb3b0 msgr2=0x7f9a38102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:34.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 shutdown_connections 2023-10-16T14:33:34.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:34.849+0000 7f9a3d5ff700 1 -- 172.21.15.178:0/1468116100 wait complete. 2023-10-16T14:33:34.856 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:33:34.872 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:33:35.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:34 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3930021880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:34 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/543174045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 -- 172.21.15.178:0/893113899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 msgr2=0x7f339810f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 --2- 172.21.15.178:0/893113899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f339810f6a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f339000b0a0 tx=0x7f339000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 -- 172.21.15.178:0/893113899 shutdown_connections 2023-10-16T14:33:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 --2- 172.21.15.178:0/893113899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f339810f6a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 --2- 172.21.15.178:0/893113899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f3398073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 --2- 172.21.15.178:0/893113899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 0x7f3398073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 -- 172.21.15.178:0/893113899 >> 172.21.15.178:0/893113899 conn(0x7f33980fc210 msgr2=0x7f33980fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:35.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 -- 172.21.15.178:0/893113899 shutdown_connections 2023-10-16T14:33:35.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.066+0000 7f339d79d700 1 -- 172.21.15.178:0/893113899 wait complete. 2023-10-16T14:33:35.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.067+0000 7f339d79d700 1 Processor -- start 2023-10-16T14:33:35.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.067+0000 7f339d79d700 1 -- start start 2023-10-16T14:33:35.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 0x7f339819d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f33981a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981a4c30 con 0x7f339810d2b0 2023-10-16T14:33:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981a4d70 con 0x7f3398073910 2023-10-16T14:33:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339d79d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33981a4f00 con 0x7f3398074ef0 2023-10-16T14:33:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f33977fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f33981a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f3397fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 0x7f339819d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f33977fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42356/0 (socket says 172.21.15.178:42356) 2023-10-16T14:33:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f33977fe700 1 -- 172.21.15.178:0/3458113243 learned_addr learned my addr 172.21.15.178:0/3458113243 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 -- 172.21.15.178:0/3458113243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 msgr2=0x7f339819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 -- 172.21.15.178:0/3458113243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 msgr2=0x7f339819d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 0x7f339819d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f339cf9c700 1 -- 172.21.15.178:0/3458113243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009d00 con 0x7f339810d2b0 2023-10-16T14:33:35.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.068+0000 7f33977fe700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:35.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f339cf9c700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f33981a4660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3390009fd0 tx=0x7f3390007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:35.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339000e040 con 0x7f339810d2b0 2023-10-16T14:33:35.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3390004c50 con 0x7f339810d2b0 2023-10-16T14:33:35.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33981a5180 con 0x7f339810d2b0 2023-10-16T14:33:35.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339001b6c0 con 0x7f339810d2b0 2023-10-16T14:33:35.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.069+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33981a55a0 con 0x7f339810d2b0 2023-10-16T14:33:35.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.070+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3398102e00 con 0x7f339810d2b0 2023-10-16T14:33:35.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:34 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3930021880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:34 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/543174045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.072+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f339001d470 con 0x7f339810d2b0 2023-10-16T14:33:35.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.073+0000 7f33957fa700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 0x7f33800a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.073+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f33900ce870 con 0x7f339810d2b0 2023-10-16T14:33:35.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.074+0000 7f3397fff700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 0x7f33800a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.074+0000 7f3397fff700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 0x7f33800a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3388009c60 tx=0x7f3388009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:35.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.075+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3390097a30 con 0x7f339810d2b0 2023-10-16T14:33:35.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:34 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3930021880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:34 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/543174045' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:33:35.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.280+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3398112de0 con 0x7f339810d2b0 2023-10-16T14:33:35.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.281+0000 7f33957fa700 1 -- 172.21.15.178:0/3458113243 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7f339009b050 con 0x7f339810d2b0 2023-10-16T14:33:35.286 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:35.286 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:33:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 msgr2=0x7f33800a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 0x7f33800a39d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3388009c60 tx=0x7f3388009260 comp rx=0 tx=0).stop 2023-10-16T14:33:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 msgr2=0x7f33981a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f33981a4660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3390009fd0 tx=0x7f3390007930 comp rx=0 tx=0).stop 2023-10-16T14:33:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 shutdown_connections 2023-10-16T14:33:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f33800a1510 0x7f33800a39d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f339810d2b0 0x7f33981a4660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3398073910 0x7f339819d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 --2- 172.21.15.178:0/3458113243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3398074ef0 0x7f339819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 >> 172.21.15.178:0/3458113243 conn(0x7f33980fc210 msgr2=0x7f33980fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:35.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 shutdown_connections 2023-10-16T14:33:35.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.284+0000 7f339d79d700 1 -- 172.21.15.178:0/3458113243 wait complete. 2023-10-16T14:33:35.307 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:33:35.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 -- 172.21.15.178:0/4267558825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c102a50 msgr2=0x7fd88c109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 --2- 172.21.15.178:0/4267558825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c102a50 0x7fd88c109370 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd88400b0a0 tx=0x7fd88400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 -- 172.21.15.178:0/4267558825 shutdown_connections 2023-10-16T14:33:35.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 --2- 172.21.15.178:0/4267558825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c102a50 0x7fd88c109370 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 --2- 172.21.15.178:0/4267558825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c100eb0 0x7fd88c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 --2- 172.21.15.178:0/4267558825 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 0x7fd88c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 -- 172.21.15.178:0/4267558825 >> 172.21.15.178:0/4267558825 conn(0x7fd88c0fc430 msgr2=0x7fd88c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 -- 172.21.15.178:0/4267558825 shutdown_connections 2023-10-16T14:33:35.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.501+0000 7fd8917ff700 1 -- 172.21.15.178:0/4267558825 wait complete. 2023-10-16T14:33:35.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.502+0000 7fd8917ff700 1 Processor -- start 2023-10-16T14:33:35.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.502+0000 7fd8917ff700 1 -- start start 2023-10-16T14:33:35.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 0x7fd88c0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 0x7fd88c0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c073f80 con 0x7fd88c100eb0 2023-10-16T14:33:35.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0740f0 con 0x7fd88c102090 2023-10-16T14:33:35.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd8917ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1a0ca0 con 0x7fd88c102a50 2023-10-16T14:33:35.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59756/0 (socket says 172.21.15.178:59756) 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 -- 172.21.15.178:0/1896482654 learned_addr learned my addr 172.21.15.178:0/1896482654 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88a7fc700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 0x7fd88c0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88b7fe700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 0x7fd88c0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 -- 172.21.15.178:0/1896482654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 msgr2=0x7fd88c0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 0x7fd88c0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 -- 172.21.15.178:0/1896482654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 msgr2=0x7fd88c0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 0x7fd88c0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88affd700 1 -- 172.21.15.178:0/1896482654 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009d00 con 0x7fd88c100eb0 2023-10-16T14:33:35.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.503+0000 7fd88b7fe700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 0x7fd88c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:35.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd88affd700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e970 tx=0x7fd87c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:35.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c009790 con 0x7fd88c100eb0 2023-10-16T14:33:35.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd87c00eec0 con 0x7fd88c100eb0 2023-10-16T14:33:35.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c1a0f80 con 0x7fd88c100eb0 2023-10-16T14:33:35.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c003cf0 con 0x7fd88c100eb0 2023-10-16T14:33:35.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.504+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c1a13a0 con 0x7fd88c100eb0 2023-10-16T14:33:35.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.505+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c107dc0 con 0x7fd88c100eb0 2023-10-16T14:33:35.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.507+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fd87c00ccd0 con 0x7fd88c100eb0 2023-10-16T14:33:35.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.508+0000 7fd873fff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 0x7fd8740a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.509+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd87c014070 con 0x7fd88c100eb0 2023-10-16T14:33:35.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.509+0000 7fd88a7fc700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 0x7fd8740a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.509+0000 7fd88a7fc700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 0x7fd8740a3b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd88c100bd0 tx=0x7fd880009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:35.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.512+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd87c096600 con 0x7fd88c100eb0 2023-10-16T14:33:35.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.762+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd88c10ca20 con 0x7fd88c100eb0 2023-10-16T14:33:35.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.762+0000 7fd873fff700 1 -- 172.21.15.178:0/1896482654 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7fd87c099c20 con 0x7fd88c100eb0 2023-10-16T14:33:35.766 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:35.767 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:33:35.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 msgr2=0x7fd8740a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 0x7fd8740a3b00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd88c100bd0 tx=0x7fd880009260 comp rx=0 tx=0).stop 2023-10-16T14:33:35.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 msgr2=0x7fd88c074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e970 tx=0x7fd87c00ec80 comp rx=0 tx=0).stop 2023-10-16T14:33:35.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 shutdown_connections 2023-10-16T14:33:35.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd8740a1640 0x7fd8740a3b00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd88c102a50 0x7fd88c0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd88c100eb0 0x7fd88c074c60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 --2- 172.21.15.178:0/1896482654 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd88c102090 0x7fd88c0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 >> 172.21.15.178:0/1896482654 conn(0x7fd88c0fc430 msgr2=0x7fd88c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:35.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 shutdown_connections 2023-10-16T14:33:35.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.765+0000 7fd8917ff700 1 -- 172.21.15.178:0/1896482654 wait complete. 2023-10-16T14:33:35.772 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:33:35.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-16T14:31:36.554542+0000', 'modified': '2023-10-16T14:32:15.910283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24550, 'mds_2': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/1810043383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1810043383}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 1810043383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/2797759968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2797759968}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2797759968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/4191845467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 4191845467}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 4191845467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 2, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.186:6837/1524095342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1524095342}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1524095342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.187:6839/397706812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 397706812}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 397706812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6839/2475230127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2475230127}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2475230127}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T14:33:35.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-16T14:33:35.789 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-16T14:33:35.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 -- 172.21.15.178:0/3293637009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102080 msgr2=0x7fb3b4102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/3293637009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102080 0x7fb3b4102500 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009990 tx=0x7fb3a4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 -- 172.21.15.178:0/3293637009 shutdown_connections 2023-10-16T14:33:35.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/3293637009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102a40 0x7fb3b4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/3293637009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 0x7fb3b41012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/3293637009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102080 0x7fb3b4102500 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 -- 172.21.15.178:0/3293637009 >> 172.21.15.178:0/3293637009 conn(0x7fb3b40fc420 msgr2=0x7fb3b40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:35.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 -- 172.21.15.178:0/3293637009 shutdown_connections 2023-10-16T14:33:35.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.983+0000 7fb3bbfff700 1 -- 172.21.15.178:0/3293637009 wait complete. 2023-10-16T14:33:35.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 Processor -- start 2023-10-16T14:33:35.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 -- start start 2023-10-16T14:33:35.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 0x7fb3b4196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102080 0x7fb3b4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:35.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419e740 con 0x7fb3b4102a40 2023-10-16T14:33:35.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419e880 con 0x7fb3b4102080 2023-10-16T14:33:35.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.984+0000 7fb3bbfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b419ea10 con 0x7fb3b4100ea0 2023-10-16T14:33:35.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59782/0 (socket says 172.21.15.178:59782) 2023-10-16T14:33:35.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1863414323 learned_addr learned my addr 172.21.15.178:0/1863414323 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:35.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 0x7fb3b4196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3b959a700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102080 0x7fb3b4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:35.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1863414323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 msgr2=0x7fb3b4196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 0x7fb3b4196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1863414323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102080 msgr2=0x7fb3b4197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:35.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102080 0x7fb3b4197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:35.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1863414323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40095f0 con 0x7fb3b4102a40 2023-10-16T14:33:35.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.985+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00ef30 tx=0x7fb3ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:35.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.986+0000 7fb3aaffd700 1 -- 172.21.15.178:0/1863414323 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac018070 con 0x7fb3b4102a40 2023-10-16T14:33:35.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.986+0000 7fb3aaffd700 1 -- 172.21.15.178:0/1863414323 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ac00f040 con 0x7fb3b4102a40 2023-10-16T14:33:35.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.986+0000 7fb3aaffd700 1 -- 172.21.15.178:0/1863414323 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00cd00 con 0x7fb3b4102a40 2023-10-16T14:33:35.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.986+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b419ec90 con 0x7fb3b4102a40 2023-10-16T14:33:35.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.986+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b419f150 con 0x7fb3b4102a40 2023-10-16T14:33:36.005 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.989+0000 7fb3aaffd700 1 -- 172.21.15.178:0/1863414323 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7fb3ac003a00 con 0x7fb3b4102a40 2023-10-16T14:33:36.005 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 Processor -- start 2023-10-16T14:33:36.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 -- start start 2023-10-16T14:33:36.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3aaffd700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 0x7fb3a00a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3aaffd700 1 -- 172.21.15.178:0/1863414323 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb3ac0ccd00 con 0x7fb3b4102a40 2023-10-16T14:33:36.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a83d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a83de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a7acc20 con 0x56062a83d9e0 2023-10-16T14:33:36.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a791c10 con 0x56062a83c1e0 2023-10-16T14:33:36.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.990+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a78f530 con 0x56062a83cb40 2023-10-16T14:33:36.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b959a700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 0x7fb3a00a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48336/0 (socket says 172.21.15.178:48336) 2023-10-16T14:33:36.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/531594225 learned_addr learned my addr 172.21.15.178:0/531594225 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a83de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b959a700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a83d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/531594225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 msgr2=0x56062a83d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a83d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/531594225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 msgr2=0x56062a83de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a83de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/531594225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062a7483a0 con 0x56062a83c1e0 2023-10-16T14:33:36.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a83de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b959a700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a83d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.991+0000 7fb3b959a700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 0x7fb3a00a3a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400b2e0 tx=0x7fb3a400bc30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d30 tx=0x7fb3b0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fee6acc1b9190dae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb39ffff700 1 -- 172.21.15.178:0/531594225 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004220 con 0x56062a83c1e0 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb39ffff700 1 -- 172.21.15.178:0/531594225 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b0004380 con 0x56062a83c1e0 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb39ffff700 1 -- 172.21.15.178:0/531594225 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0019780 con 0x56062a83c1e0 2023-10-16T14:33:36.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb3cf921b80 1 -- 172.21.15.178:0/531594225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 msgr2=0x56062a83c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.992+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009d30 tx=0x7fb3b0011720 comp rx=0 tx=0).stop 2023-10-16T14:33:36.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 -- 172.21.15.178:0/531594225 shutdown_connections 2023-10-16T14:33:36.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a83de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a83c600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/531594225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a83d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 -- 172.21.15.178:0/531594225 >> 172.21.15.178:0/531594225 conn(0x56062a765d80 msgr2=0x56062a832f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 -- 172.21.15.178:0/531594225 shutdown_connections 2023-10-16T14:33:36.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 -- 172.21.15.178:0/531594225 wait complete. 2023-10-16T14:33:36.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.993+0000 7fb3cf921b80 1 -- start start 2023-10-16T14:33:36.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 Processor -- start 2023-10-16T14:33:36.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83cb40 0x56062a846510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 0x56062a84d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a78ae20 con 0x56062a83c1e0 2023-10-16T14:33:36.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a7276c0 con 0x56062a83cb40 2023-10-16T14:33:36.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a727360 con 0x56062a83d9e0 2023-10-16T14:33:36.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3cf921b80 10 client.0 _ll_register_callbacks cb 0x56062a5847b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.994+0000 7fb3b9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59794/0 (socket says 172.21.15.178:59794) 2023-10-16T14:33:36.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/219238255 learned_addr learned my addr 172.21.15.178:0/219238255 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b959a700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83cb40 0x56062a846510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 0x56062a84d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/219238255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 msgr2=0x56062a84d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 0x56062a84d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83cb40 msgr2=0x56062a846510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83cb40 0x56062a846510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 -- 172.21.15.178:0/219238255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b419f150 con 0x56062a83c1e0 2023-10-16T14:33:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 0x56062a84d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:36.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0011a70 tx=0x7fb3b0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d5fb20175fb93e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb39d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0005040 con 0x56062a83c1e0 2023-10-16T14:33:36.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb3b00051a0 con 0x56062a83c1e0 2023-10-16T14:33:36.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.995+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56062a78fea0 con 0x56062a83c1e0 2023-10-16T14:33:36.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3b0005360 con 0x56062a83c1e0 2023-10-16T14:33:36.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 10 client.15156 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fb3b0009960 con 0x56062a83c1e0 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56062a78fea0 con 0x56062a83c1e0 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb3b0021040 con 0x56062a83c1e0 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 10 client.15156 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 10 client.15156 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 20 client.15156 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a851b30 0x56062a850600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 4 client.15156 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:33:36.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.996+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56062a807bc0 con 0x56062a851b30 2023-10-16T14:33:36.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.997+0000 7fb3b959a700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a851b30 0x56062a850600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.997+0000 7fb3b959a700 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a851b30 0x56062a850600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49f2b4be92a70d04 server_cookie=da7c49b7c1a9e7ca in_seq=0 out_seq=0 2023-10-16T14:33:36.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.997+0000 7fb39d7fa700 10 client.15156 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:33:36.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb39d7fa700 1 -- 172.21.15.178:0/219238255 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56062a807bc0 con 0x56062a851b30 2023-10-16T14:33:36.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb39d7fa700 10 client.15156 handle_command_reply: tid=0 2023-10-16T14:33:36.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 client.15156 shutdown 2023-10-16T14:33:36.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 msgr2=0x56062a845fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0011a70 tx=0x7fb3b0005d50 comp rx=0 tx=0).stop 2023-10-16T14:33:36.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 shutdown_connections 2023-10-16T14:33:36.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a851b30 0x56062a850600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83d9e0 0x56062a84d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83c1e0 0x56062a845fd0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/219238255 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83cb40 0x56062a846510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb39d7fa700 0 client.15156 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:33:36.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.998+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 >> 172.21.15.178:0/219238255 conn(0x56062a765d80 msgr2=0x56062a832ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.999+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 shutdown_connections 2023-10-16T14:33:36.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.999+0000 7fb3cf921b80 1 -- 172.21.15.178:0/219238255 wait complete. 2023-10-16T14:33:36.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:35.999+0000 7fb3cf921b80 20 client.15156 trim_cache size 0 max 16384 2023-10-16T14:33:36.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 Processor -- start 2023-10-16T14:33:36.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 -- start start 2023-10-16T14:33:36.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a840950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a840e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a60b8d0 con 0x56062a83d9e0 2023-10-16T14:33:36.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a807bc0 con 0x56062a83c1e0 2023-10-16T14:33:36.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a846d60 con 0x56062a83cb40 2023-10-16T14:33:36.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3ba59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3ba59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59806/0 (socket says 172.21.15.178:59806) 2023-10-16T14:33:36.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1660262398 learned_addr learned my addr 172.21.15.178:0/1660262398 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.017+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a840950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3b959a700 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a840e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1660262398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 msgr2=0x56062a840e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a840e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1660262398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 msgr2=0x56062a840950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a840950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 -- 172.21.15.178:0/1660262398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062a5b5d90 con 0x56062a83d9e0 2023-10-16T14:33:36.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ca310 tx=0x7fb3ac095850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d421bebc165ec06d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3a8ff9700 1 -- 172.21.15.178:0/1660262398 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac01b320 con 0x56062a83d9e0 2023-10-16T14:33:36.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3a8ff9700 1 -- 172.21.15.178:0/1660262398 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3ac09a280 con 0x56062a83d9e0 2023-10-16T14:33:36.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.018+0000 7fb3a8ff9700 1 -- 172.21.15.178:0/1660262398 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac0a1f10 con 0x56062a83d9e0 2023-10-16T14:33:36.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- 172.21.15.178:0/1660262398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 msgr2=0x56062a84d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0ca310 tx=0x7fb3ac095850 comp rx=0 tx=0).stop 2023-10-16T14:33:36.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- 172.21.15.178:0/1660262398 shutdown_connections 2023-10-16T14:33:36.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83d9e0 0x56062a84d510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83c1e0 0x56062a840950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/1660262398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83cb40 0x56062a840e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- 172.21.15.178:0/1660262398 >> 172.21.15.178:0/1660262398 conn(0x56062a765d80 msgr2=0x56062a843dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- 172.21.15.178:0/1660262398 shutdown_connections 2023-10-16T14:33:36.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- 172.21.15.178:0/1660262398 wait complete. 2023-10-16T14:33:36.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 -- start start 2023-10-16T14:33:36.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.019+0000 7fb3cf921b80 1 Processor -- start 2023-10-16T14:33:36.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83c1e0 0x56062a84d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83d9e0 0x56062a84a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a8a5550 con 0x56062a83cb40 2023-10-16T14:33:36.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a8a5690 con 0x56062a83d9e0 2023-10-16T14:33:36.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062a8a57d0 con 0x56062a83c1e0 2023-10-16T14:33:36.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3cf921b80 10 client.0 _ll_register_callbacks cb 0x7fb3ac0957a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59816/0 (socket says 172.21.15.178:59816) 2023-10-16T14:33:36.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b9d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83c1e0 0x56062a84d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 -- 172.21.15.178:0/3463966867 learned_addr learned my addr 172.21.15.178:0/3463966867 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3ba59c700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83d9e0 0x56062a84a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 -- 172.21.15.178:0/3463966867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83c1e0 msgr2=0x56062a84d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83c1e0 0x56062a84d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 -- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83d9e0 msgr2=0x56062a84a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83d9e0 0x56062a84a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.020+0000 7fb3b959a700 1 -- 172.21.15.178:0/3463966867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b419ec90 con 0x56062a83cb40 2023-10-16T14:33:36.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb3b959a700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4100bc0 tx=0x7fb3a401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d4955a85e1c4c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb39d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a401d070 con 0x56062a83cb40 2023-10-16T14:33:36.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3b0004380 con 0x56062a83cb40 2023-10-16T14:33:36.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb3a4037df0 con 0x56062a83cb40 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3a4037050 con 0x56062a83cb40 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb3cf921b80 10 client.15162 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56062a59ec10 con 0x56062a83cb40 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.021+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b0004380 con 0x56062a83cb40 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb3a4004cd0 con 0x56062a83cb40 2023-10-16T14:33:36.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 10 client.15162 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-16T14:33:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 20 client.15162 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a8aab40 0x56062a8a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 4 client.15162 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:33:36.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56062a783e00 con 0x56062a8aab40 2023-10-16T14:33:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a8aab40 0x56062a8a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb3b9d9b700 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a8aab40 0x56062a8a6f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=809193efa8caa127 server_cookie=f01739374901b8f4 in_seq=0 out_seq=0 2023-10-16T14:33:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.022+0000 7fb39d7fa700 10 client.15162 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:33:36.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb39d7fa700 1 -- 172.21.15.178:0/3463966867 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56062a783e00 con 0x56062a8aab40 2023-10-16T14:33:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb39d7fa700 10 client.15162 handle_command_reply: tid=0 2023-10-16T14:33:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 client.15162 shutdown 2023-10-16T14:33:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 msgr2=0x56062a849f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4100bc0 tx=0x7fb3a401a050 comp rx=0 tx=0).stop 2023-10-16T14:33:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 shutdown_connections 2023-10-16T14:33:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56062a8aab40 0x56062a8a6f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56062a83d9e0 0x56062a84a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56062a83c1e0 0x56062a84d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 --2- 172.21.15.178:0/3463966867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56062a83cb40 0x56062a849f50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb39d7fa700 0 client.15162 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:33:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 >> 172.21.15.178:0/3463966867 conn(0x56062a765d80 msgr2=0x56062a840950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 shutdown_connections 2023-10-16T14:33:36.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 1 -- 172.21.15.178:0/3463966867 wait complete. 2023-10-16T14:33:36.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.023+0000 7fb3cf921b80 20 client.15162 trim_cache size 0 max 16384 2023-10-16T14:33:36.050 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:33:36.050 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:33:36.051 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 2, 2023-10-16T14:33:36.051 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24548, 2023-10-16T14:33:36.052 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:33:36.052 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:33:36.053 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:33:36.053 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 87.331630133000004, 2023-10-16T14:33:36.053 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:33:36.053 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:33:36.053 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:33:36.054 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 135.76953238499999 2023-10-16T14:33:36.054 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:33:36.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 msgr2=0x7fb3a00a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 0x7fb3a00a3a90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb3a400b2e0 tx=0x7fb3a400bc30 comp rx=0 tx=0).stop 2023-10-16T14:33:36.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 msgr2=0x7fb3b419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00ef30 tx=0x7fb3ac00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 shutdown_connections 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb3a00a15d0 0x7fb3a00a3a90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb3b4102a40 0x7fb3b419e170 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb3b4100ea0 0x7fb3b4196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 --2- 172.21.15.178:0/1863414323 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb3b4102080 0x7fb3b4197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 >> 172.21.15.178:0/1863414323 conn(0x7fb3b40fc420 msgr2=0x7fb3b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 shutdown_connections 2023-10-16T14:33:36.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.024+0000 7fb3bbfff700 1 -- 172.21.15.178:0/1863414323 wait complete. 2023-10-16T14:33:36.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:35 smithi178 ceph-mon[98495]: pgmap v335: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:36.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:35 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1468116100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:35 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3458113243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.059 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-16T14:33:36.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:35 smithi186 ceph-mon[134892]: pgmap v335: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:35 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1468116100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:35 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3458113243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:35 smithi187 ceph-mon[135297]: pgmap v335: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:35 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1468116100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:35 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3458113243' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:36.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 -- 172.21.15.178:0/257055131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4104370 msgr2=0x7f72a4106760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 --2- 172.21.15.178:0/257055131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4104370 0x7f72a4106760 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f72980098f0 tx=0x7f7298009c00 comp rx=0 tx=0).stop 2023-10-16T14:33:36.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 -- 172.21.15.178:0/257055131 shutdown_connections 2023-10-16T14:33:36.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 --2- 172.21.15.178:0/257055131 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4106ca0 0x7f72a4109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 --2- 172.21.15.178:0/257055131 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4101a40 0x7f72a4103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 --2- 172.21.15.178:0/257055131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4104370 0x7f72a4106760 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 -- 172.21.15.178:0/257055131 >> 172.21.15.178:0/257055131 conn(0x7f72a40fb3b0 msgr2=0x7f72a40fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 -- 172.21.15.178:0/257055131 shutdown_connections 2023-10-16T14:33:36.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.254+0000 7f72a8bff700 1 -- 172.21.15.178:0/257055131 wait complete. 2023-10-16T14:33:36.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.255+0000 7f72a8bff700 1 Processor -- start 2023-10-16T14:33:36.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.255+0000 7f72a8bff700 1 -- start start 2023-10-16T14:33:36.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4106ca0 0x7f72a40fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41006d0 con 0x7f72a4104370 2023-10-16T14:33:36.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a4100840 con 0x7f72a4106ca0 2023-10-16T14:33:36.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a8bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41a0bf0 con 0x7f72a4101a40 2023-10-16T14:33:36.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a2d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4106ca0 0x7f72a40fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59834/0 (socket says 172.21.15.178:59834) 2023-10-16T14:33:36.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42438/0 (socket says 172.21.15.178:42438) 2023-10-16T14:33:36.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a1d9b700 1 -- 172.21.15.178:0/650934226 learned_addr learned my addr 172.21.15.178:0/650934226 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 -- 172.21.15.178:0/650934226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4106ca0 msgr2=0x7f72a40fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4106ca0 0x7f72a40fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 -- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 msgr2=0x7f72a40ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a259c700 1 -- 172.21.15.178:0/650934226 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940095f0 con 0x7f72a4101a40 2023-10-16T14:33:36.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.256+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72a259c700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f729400eb90 tx=0x7f729400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72937fe700 1 -- 172.21.15.178:0/650934226 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294009970 con 0x7f72a4101a40 2023-10-16T14:33:36.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298009520 con 0x7f72a4101a40 2023-10-16T14:33:36.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72937fe700 1 -- 172.21.15.178:0/650934226 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72940044e0 con 0x7f72a4101a40 2023-10-16T14:33:36.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72937fe700 1 -- 172.21.15.178:0/650934226 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294003e50 con 0x7f72a4101a40 2023-10-16T14:33:36.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.257+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a1190 con 0x7f72a4101a40 2023-10-16T14:33:36.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.260+0000 7f72937fe700 1 -- 172.21.15.178:0/650934226 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f7294004000 con 0x7f72a4101a40 2023-10-16T14:33:36.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 Processor -- start 2023-10-16T14:33:36.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 -- start start 2023-10-16T14:33:36.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2ddd0 0x55d5fef2e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef2eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef2fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5feea2c20 con 0x55d5fef2ddd0 2023-10-16T14:33:36.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee87c10 con 0x55d5fef2e730 2023-10-16T14:33:36.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72bc509b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee85530 con 0x55d5fef2f5d0 2023-10-16T14:33:36.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.261+0000 7f72937fe700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 0x7f728c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72937fe700 1 -- 172.21.15.178:0/650934226 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7294014070 con 0x7f72a4101a40 2023-10-16T14:33:36.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef2fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef2fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:42440/0 (socket says 172.21.15.178:42440) 2023-10-16T14:33:36.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef2eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a1d9b700 1 -- 172.21.15.178:0/1404536782 learned_addr learned my addr 172.21.15.178:0/1404536782 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2ddd0 0x55d5fef2e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 0x7f728c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 -- 172.21.15.178:0/1404536782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 msgr2=0x55d5fef2fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef2fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 -- 172.21.15.178:0/1404536782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2ddd0 msgr2=0x55d5fef2e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2ddd0 0x55d5fef2e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 -- 172.21.15.178:0/1404536782 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fee3e3a0 con 0x55d5fef2e730 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.262+0000 7f72a259c700 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef2eff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72940cacc0 tx=0x7f7294096890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=938b5a91f5a34717 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 0x7f728c0a3a10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f729c009ec0 tx=0x7f729c00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f7290ff9700 1 -- 172.21.15.178:0/1404536782 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729400ce20 con 0x55d5fef2e730 2023-10-16T14:33:36.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f7290ff9700 1 -- 172.21.15.178:0/1404536782 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729409abd0 con 0x55d5fef2e730 2023-10-16T14:33:36.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f7290ff9700 1 -- 172.21.15.178:0/1404536782 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7294016ea0 con 0x55d5fef2e730 2023-10-16T14:33:36.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 -- 172.21.15.178:0/1404536782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 msgr2=0x55d5fef2eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef2eff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72940cacc0 tx=0x7f7294096890 comp rx=0 tx=0).stop 2023-10-16T14:33:36.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 -- 172.21.15.178:0/1404536782 shutdown_connections 2023-10-16T14:33:36.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef2fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2ddd0 0x55d5fef2e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 --2- 172.21.15.178:0/1404536782 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef2eff0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 -- 172.21.15.178:0/1404536782 >> 172.21.15.178:0/1404536782 conn(0x55d5fee5bd80 msgr2=0x55d5fef28f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 -- 172.21.15.178:0/1404536782 shutdown_connections 2023-10-16T14:33:36.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.263+0000 7f72bc509b80 1 -- 172.21.15.178:0/1404536782 wait complete. 2023-10-16T14:33:36.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.264+0000 7f72bc509b80 1 -- start start 2023-10-16T14:33:36.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.264+0000 7f72bc509b80 1 Processor -- start 2023-10-16T14:33:36.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.264+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.264+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72bc509b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee80e20 con 0x55d5fef2f5d0 2023-10-16T14:33:36.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72bc509b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee1d6c0 con 0x55d5fef2e730 2023-10-16T14:33:36.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72bc509b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee1d360 con 0x55d5fef2ddd0 2023-10-16T14:33:36.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72bc509b80 10 client.0 _ll_register_callbacks cb 0x55d5fec7a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59852/0 (socket says 172.21.15.178:59852) 2023-10-16T14:33:36.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 -- 172.21.15.178:0/3136501885 learned_addr learned my addr 172.21.15.178:0/3136501885 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 -- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 msgr2=0x55d5fef37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 -- 172.21.15.178:0/3136501885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 msgr2=0x55d5fef381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a2d9d700 1 -- 172.21.15.178:0/3136501885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a41a1190 con 0x55d5fef2f5d0 2023-10-16T14:33:36.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.265+0000 7f72a259c700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f729c019b00 tx=0x7f729c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be306295bc8bb68 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00e070 con 0x55d5fef2f5d0 2023-10-16T14:33:36.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5fee85ea0 con 0x55d5fef2f5d0 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f729c007e40 con 0x55d5fef2f5d0 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f729c019e10 con 0x55d5fef2f5d0 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f72bc509b80 10 client.15174 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d5fecabd90 con 0x55d5fef2f5d0 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5fee85ea0 con 0x55d5fef2f5d0 2023-10-16T14:33:36.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.266+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f729c024740 con 0x55d5fef2f5d0 2023-10-16T14:33:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 10 client.15174 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-16T14:33:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 20 client.15174 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef43880 0x55d5fef42360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 4 client.15174 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:33:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5feefdbc0 con 0x55d5fef43880 2023-10-16T14:33:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.267+0000 7f72a259c700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef43880 0x55d5fef42360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.268+0000 7f7289ffb700 10 client.15174 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:33:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.268+0000 7f72a259c700 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef43880 0x55d5fef42360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89d176508619bdae server_cookie=9d1b7e847bb2cc7f in_seq=0 out_seq=0 2023-10-16T14:33:36.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.268+0000 7f7289ffb700 1 -- 172.21.15.178:0/3136501885 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d5feefdbc0 con 0x55d5fef43880 2023-10-16T14:33:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.268+0000 7f7289ffb700 10 client.15174 handle_command_reply: tid=0 2023-10-16T14:33:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.268+0000 7f72bc509b80 1 client.15174 shutdown 2023-10-16T14:33:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 msgr2=0x55d5fef3f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f729c019b00 tx=0x7f729c004ad0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 shutdown_connections 2023-10-16T14:33:36.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef43880 0x55d5fef42360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3f200 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 --2- 172.21.15.178:0/3136501885 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2e730 0x55d5fef381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f7289ffb700 0 client.15174 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:33:36.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 >> 172.21.15.178:0/3136501885 conn(0x55d5fee5bd80 msgr2=0x55d5fef28ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 shutdown_connections 2023-10-16T14:33:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 1 -- 172.21.15.178:0/3136501885 wait complete. 2023-10-16T14:33:36.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.269+0000 7f72bc509b80 20 client.15174 trim_cache size 0 max 16384 2023-10-16T14:33:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 Processor -- start 2023-10-16T14:33:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 -- start start 2023-10-16T14:33:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2f5d0 0x55d5fef35a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fed018d0 con 0x55d5fef2e730 2023-10-16T14:33:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee7cf10 con 0x55d5fef2f5d0 2023-10-16T14:33:36.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.287+0000 7f72bc509b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fee3f590 con 0x55d5fef2ddd0 2023-10-16T14:33:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59866/0 (socket says 172.21.15.178:59866) 2023-10-16T14:33:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2344014483 learned_addr learned my addr 172.21.15.178:0/2344014483 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a259c700 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2f5d0 0x55d5fef35a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2344014483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 msgr2=0x55d5fef3fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2344014483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2f5d0 msgr2=0x55d5fef35a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2f5d0 0x55d5fef35a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2344014483 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5fec94c10 con 0x55d5fef2e730 2023-10-16T14:33:36.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.288+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d5fef2edb0 tx=0x7f7298005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb7f350a2b04095 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72917fa700 1 -- 172.21.15.178:0/2344014483 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729801c040 con 0x55d5fef2e730 2023-10-16T14:33:36.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72917fa700 1 -- 172.21.15.178:0/2344014483 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729801d040 con 0x55d5fef2e730 2023-10-16T14:33:36.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72917fa700 1 -- 172.21.15.178:0/2344014483 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980056a0 con 0x55d5fef2e730 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 -- 172.21.15.178:0/2344014483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 msgr2=0x55d5fef3f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55d5fef2edb0 tx=0x7f7298005dc0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 -- 172.21.15.178:0/2344014483 shutdown_connections 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2f5d0 0x55d5fef35a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2344014483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3f120 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 -- 172.21.15.178:0/2344014483 >> 172.21.15.178:0/2344014483 conn(0x55d5fee5bd80 msgr2=0x55d5fef368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 -- 172.21.15.178:0/2344014483 shutdown_connections 2023-10-16T14:33:36.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.289+0000 7f72bc509b80 1 -- 172.21.15.178:0/2344014483 wait complete. 2023-10-16T14:33:36.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.290+0000 7f72bc509b80 1 -- start start 2023-10-16T14:33:36.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.290+0000 7f72bc509b80 1 Processor -- start 2023-10-16T14:33:36.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fef3c480 con 0x55d5fef2e730 2023-10-16T14:33:36.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fef3f440 con 0x55d5fef2ddd0 2023-10-16T14:33:36.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5fef3f5d0 con 0x55d5fef2f5d0 2023-10-16T14:33:36.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72bc509b80 10 client.0 _ll_register_callbacks cb 0x7f729c0194a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59876/0 (socket says 172.21.15.178:59876) 2023-10-16T14:33:36.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2412498821 learned_addr learned my addr 172.21.15.178:0/2412498821 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a259c700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a2d9d700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 msgr2=0x55d5fef3bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2412498821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2ddd0 msgr2=0x55d5fef3edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.291+0000 7f72a1d9b700 1 -- 172.21.15.178:0/2412498821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a4109c60 con 0x55d5fef2e730 2023-10-16T14:33:36.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f72a1d9b700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f729800fa40 tx=0x7f7298005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451af1edc397c4a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f7289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729801c070 con 0x55d5fef2e730 2023-10-16T14:33:36.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f729409abd0 con 0x55d5fef2e730 2023-10-16T14:33:36.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f729800b9f0 con 0x55d5fef2e730 2023-10-16T14:33:36.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f729800bbb0 con 0x55d5fef2e730 2023-10-16T14:33:36.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f72bc509b80 10 client.15180 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.292+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d5fecd2200 con 0x55d5fef2e730 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f729409abd0 con 0x55d5fef2e730 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f7298009920 con 0x55d5fef2e730 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 20 client.15180 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef9bf00 0x55d5fef98330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 4 client.15180 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:33:36.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5fee79e00 con 0x55d5fef9bf00 2023-10-16T14:33:36.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.293+0000 7f72a259c700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef9bf00 0x55d5fef98330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72a259c700 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef9bf00 0x55d5fef98330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66b2a5761a50e7b2 server_cookie=9b251a07550dc75a in_seq=0 out_seq=0 2023-10-16T14:33:36.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f7289ffb700 10 client.15180 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:33:36.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f7289ffb700 1 -- 172.21.15.178:0/2412498821 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d5fee79e00 con 0x55d5fef9bf00 2023-10-16T14:33:36.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f7289ffb700 10 client.15180 handle_command_reply: tid=0 2023-10-16T14:33:36.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 client.15180 shutdown 2023-10-16T14:33:36.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 msgr2=0x55d5fef3b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f729800fa40 tx=0x7f7298005a00 comp rx=0 tx=0).stop 2023-10-16T14:33:36.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 shutdown_connections 2023-10-16T14:33:36.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d5fef9bf00 0x55d5fef98330 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5fef2f5d0 0x55d5fef3bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d5fef2ddd0 0x55d5fef3edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f7289ffb700 0 client.15180 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:33:36.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 --2- 172.21.15.178:0/2412498821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d5fef2e730 0x55d5fef3b860 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 >> 172.21.15.178:0/2412498821 conn(0x55d5fee5bd80 msgr2=0x55d5fef35a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 shutdown_connections 2023-10-16T14:33:36.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 1 -- 172.21.15.178:0/2412498821 wait complete. 2023-10-16T14:33:36.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.294+0000 7f72bc509b80 20 client.15180 trim_cache size 0 max 16384 2023-10-16T14:33:36.312 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:33:36.312 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:33:36.312 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 1, 2023-10-16T14:33:36.313 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24550, 2023-10-16T14:33:36.313 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:33:36.313 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:33:36.313 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:33:36.313 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 89.615662998000005, 2023-10-16T14:33:36.314 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:33:36.314 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:33:36.314 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:33:36.314 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 133.86059666599999 2023-10-16T14:33:36.314 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:33:36.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 msgr2=0x7f728c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 0x7f728c0a3a10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f729c009ec0 tx=0x7f729c00b040 comp rx=0 tx=0).stop 2023-10-16T14:33:36.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 msgr2=0x7f72a41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f729400eb90 tx=0x7f729400eea0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 shutdown_connections 2023-10-16T14:33:36.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f728c0a1550 0x7f728c0a3a10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f72a4106ca0 0x7f72a40fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72a4101a40 0x7f72a41013b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 --2- 172.21.15.178:0/650934226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72a4104370 0x7f72a40ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 >> 172.21.15.178:0/650934226 conn(0x7f72a40fb3b0 msgr2=0x7f72a41026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 shutdown_connections 2023-10-16T14:33:36.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.295+0000 7f72a8bff700 1 -- 172.21.15.178:0/650934226 wait complete. 2023-10-16T14:33:36.318 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T14:33:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 -- 172.21.15.178:0/637455752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0280073910 msgr2=0x7f0280073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 --2- 172.21.15.178:0/637455752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0280073910 0x7f0280073d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02740098f0 tx=0x7f0274009c00 comp rx=0 tx=0).stop 2023-10-16T14:33:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 -- 172.21.15.178:0/637455752 shutdown_connections 2023-10-16T14:33:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 --2- 172.21.15.178:0/637455752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f028010d4c0 0x7f028010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 --2- 172.21.15.178:0/637455752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280074ef0 0x7f0280073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 --2- 172.21.15.178:0/637455752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0280073910 0x7f0280073d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 -- 172.21.15.178:0/637455752 >> 172.21.15.178:0/637455752 conn(0x7f02800fc420 msgr2=0x7f02800fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 -- 172.21.15.178:0/637455752 shutdown_connections 2023-10-16T14:33:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.513+0000 7f02855ff700 1 -- 172.21.15.178:0/637455752 wait complete. 2023-10-16T14:33:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.514+0000 7f02855ff700 1 Processor -- start 2023-10-16T14:33:36.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.514+0000 7f02855ff700 1 -- start start 2023-10-16T14:33:36.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.514+0000 7f02855ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.514+0000 7f02855ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0280074ef0 0x7f028019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f02855ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 0x7f02801a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f02855ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a4db0 con 0x7f0280073910 2023-10-16T14:33:36.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f02855ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a4ef0 con 0x7f028010d4c0 2023-10-16T14:33:36.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f02855ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02801a5080 con 0x7f0280074ef0 2023-10-16T14:33:36.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59892/0 (socket says 172.21.15.178:59892) 2023-10-16T14:33:36.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 -- 172.21.15.178:0/256477999 learned_addr learned my addr 172.21.15.178:0/256477999 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027f7fe700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 0x7f02801a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027e7fc700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0280074ef0 0x7f028019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 -- 172.21.15.178:0/256477999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0280074ef0 msgr2=0x7f028019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0280074ef0 0x7f028019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 -- 172.21.15.178:0/256477999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 msgr2=0x7f02801a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 0x7f02801a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 -- 172.21.15.178:0/256477999 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02700095f0 con 0x7f0280073910 2023-10-16T14:33:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027f7fe700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 0x7f02801a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.515+0000 7f027effd700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f027000ec10 tx=0x7f027000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.516+0000 7f0267fff700 1 -- 172.21.15.178:0/256477999 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000cc90 con 0x7f0280073910 2023-10-16T14:33:36.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.516+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0274009520 con 0x7f0280073910 2023-10-16T14:33:36.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.516+0000 7f0267fff700 1 -- 172.21.15.178:0/256477999 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02700044e0 con 0x7f0280073910 2023-10-16T14:33:36.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.516+0000 7f0267fff700 1 -- 172.21.15.178:0/256477999 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270018430 con 0x7f0280073910 2023-10-16T14:33:36.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.516+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02801a5730 con 0x7f0280073910 2023-10-16T14:33:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.519+0000 7f0267fff700 1 -- 172.21.15.178:0/256477999 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 17) v1 ==== 136055+0+0 (secure 0 0 0) 0x7f0270003f90 con 0x7f0280073910 2023-10-16T14:33:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0298fd3b80 1 Processor -- start 2023-10-16T14:33:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0267fff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 0x7f02680a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0267fff700 1 -- 172.21.15.178:0/256477999 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0270014070 con 0x7f0280073910 2023-10-16T14:33:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0298fd3b80 1 -- start start 2023-10-16T14:33:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f027f7fe700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 0x7f02680a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b8013851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.520+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8013865d0 0x55b801386a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012f9c20 con 0x55b801385730 2023-10-16T14:33:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012dec10 con 0x55b8013865d0 2023-10-16T14:33:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012dc530 con 0x55b801384dd0 2023-10-16T14:33:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59906/0 (socket says 172.21.15.178:59906) 2023-10-16T14:33:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 -- 172.21.15.178:0/2802768629 learned_addr learned my addr 172.21.15.178:0/2802768629 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027effd700 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b8013851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027f7fe700 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8013865d0 0x55b801386a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 -- 172.21.15.178:0/2802768629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 msgr2=0x55b8013851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b8013851f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 -- 172.21.15.178:0/2802768629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8013865d0 msgr2=0x55b801386a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8013865d0 0x55b801386a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 -- 172.21.15.178:0/2802768629 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8012953a0 con 0x55b801385730 2023-10-16T14:33:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027f7fe700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 0x7f02680a3a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0278003e80 tx=0x7f027800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.521+0000 7f027e7fc700 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02740098c0 tx=0x7f0274005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bf9753c7d3cf6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.522+0000 7f02657fa700 1 -- 172.21.15.178:0/2802768629 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400f720 con 0x55b801385730 2023-10-16T14:33:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.522+0000 7f02657fa700 1 -- 172.21.15.178:0/2802768629 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027401d040 con 0x55b801385730 2023-10-16T14:33:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.522+0000 7f02657fa700 1 -- 172.21.15.178:0/2802768629 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274005680 con 0x55b801385730 2023-10-16T14:33:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.522+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2802768629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 msgr2=0x55b801385ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.522+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02740098c0 tx=0x7f0274005dc0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2802768629 shutdown_connections 2023-10-16T14:33:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8013865d0 0x55b801386a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b8013851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2802768629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b801385730 0x55b801385ff0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2802768629 >> 172.21.15.178:0/2802768629 conn(0x55b8012b2d80 msgr2=0x55b80137ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2802768629 shutdown_connections 2023-10-16T14:33:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2802768629 wait complete. 2023-10-16T14:33:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 -- start start 2023-10-16T14:33:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.523+0000 7f0298fd3b80 1 Processor -- start 2023-10-16T14:33:36.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b80138ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b80138f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012d7e20 con 0x55b8013865d0 2023-10-16T14:33:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012746c0 con 0x55b801384dd0 2023-10-16T14:33:36.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b801274360 con 0x55b801385730 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f0298fd3b80 10 client.0 _ll_register_callbacks cb 0x55b8010d17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b80138ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59918/0 (socket says 172.21.15.178:59918) 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 -- 172.21.15.178:0/3688052827 learned_addr learned my addr 172.21.15.178:0/3688052827 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027e7fc700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b80138f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 -- 172.21.15.178:0/3688052827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 msgr2=0x55b80138f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b80138f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 -- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 msgr2=0x55b80138ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.524+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b80138ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f027f7fe700 1 -- 172.21.15.178:0/3688052827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0280110260 con 0x55b8013865d0 2023-10-16T14:33:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f027effd700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b80138ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f027800eec0 tx=0x7f0278004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af85566577c3ab12 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0262ff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278017e60 con 0x55b8013865d0 2023-10-16T14:33:36.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f027802a460 con 0x55b8013865d0 2023-10-16T14:33:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8012dcea0 con 0x55b8013865d0 2023-10-16T14:33:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027802a620 con 0x55b8013865d0 2023-10-16T14:33:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0298fd3b80 10 client.15192 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.525+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b801102d90 con 0x55b8013865d0 2023-10-16T14:33:36.544 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8012dcea0 con 0x55b8013865d0 2023-10-16T14:33:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0278021510 con 0x55b8013865d0 2023-10-16T14:33:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 10 client.15192 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:33:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 20 client.15192 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b80139a800 0x55b8013992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 4 client.15192 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:33:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b801354bc0 con 0x55b80139a800 2023-10-16T14:33:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.526+0000 7f027effd700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b80139a800 0x55b8013992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.527+0000 7f027effd700 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b80139a800 0x55b8013992a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93a1599a3dbcdf server_cookie=a610bb9a10d8c00c in_seq=0 out_seq=0 2023-10-16T14:33:36.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.527+0000 7f0262ff5700 10 client.15192 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:33:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0262ff5700 1 -- 172.21.15.178:0/3688052827 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b801354bc0 con 0x55b80139a800 2023-10-16T14:33:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0262ff5700 10 client.15192 handle_command_reply: tid=0 2023-10-16T14:33:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 client.15192 shutdown 2023-10-16T14:33:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 msgr2=0x55b801396140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f027800eec0 tx=0x7f0278004ad0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 shutdown_connections 2023-10-16T14:33:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b80139a800 0x55b8013992a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801396140 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0262ff5700 0 client.15192 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b80138ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3688052827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b80138f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 >> 172.21.15.178:0/3688052827 conn(0x55b8012b2d80 msgr2=0x55b80137fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 shutdown_connections 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3688052827 wait complete. 2023-10-16T14:33:36.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.528+0000 7f0298fd3b80 20 client.15192 trim_cache size 0 max 16384 2023-10-16T14:33:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.546+0000 7f0298fd3b80 1 Processor -- start 2023-10-16T14:33:36.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.546+0000 7f0298fd3b80 1 -- start start 2023-10-16T14:33:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b80138ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801385730 0x55b801396100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b80138e790 con 0x55b8013865d0 2023-10-16T14:33:36.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8011588d0 con 0x55b801385730 2023-10-16T14:33:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8012dcea0 con 0x55b801384dd0 2023-10-16T14:33:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027e7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801385730 0x55b801396100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59920/0 (socket says 172.21.15.178:59920) 2023-10-16T14:33:36.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b80138ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027f7fe700 1 -- 172.21.15.178:0/2804569835 learned_addr learned my addr 172.21.15.178:0/2804569835 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 -- 172.21.15.178:0/2804569835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801385730 msgr2=0x55b801396100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801385730 0x55b801396100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 -- 172.21.15.178:0/2804569835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 msgr2=0x55b80137fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027effd700 1 -- 172.21.15.178:0/2804569835 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8010ebc10 con 0x55b801384dd0 2023-10-16T14:33:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.547+0000 7f027f7fe700 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f027effd700 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b80138ca20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02700cad30 tx=0x7f027000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d65ea3f66697b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f0265ffb700 1 -- 172.21.15.178:0/2804569835 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027009ac40 con 0x55b801384dd0 2023-10-16T14:33:36.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f0265ffb700 1 -- 172.21.15.178:0/2804569835 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02700b6070 con 0x55b801384dd0 2023-10-16T14:33:36.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f0265ffb700 1 -- 172.21.15.178:0/2804569835 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270004240 con 0x55b801384dd0 2023-10-16T14:33:36.559 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2804569835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 msgr2=0x55b80138ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.548+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b80138ca20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02700cad30 tx=0x7f027000ef90 comp rx=0 tx=0).stop 2023-10-16T14:33:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2804569835 shutdown_connections 2023-10-16T14:33:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b80137fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801384dd0 0x55b80138ca20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/2804569835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801385730 0x55b801396100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2804569835 >> 172.21.15.178:0/2804569835 conn(0x55b8012b2d80 msgr2=0x55b80137e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2804569835 shutdown_connections 2023-10-16T14:33:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 -- 172.21.15.178:0/2804569835 wait complete. 2023-10-16T14:33:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 -- start start 2023-10-16T14:33:36.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.549+0000 7f0298fd3b80 1 Processor -- start 2023-10-16T14:33:36.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b801396140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b801392be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8013ee220 con 0x55b8013865d0 2023-10-16T14:33:36.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8013ee360 con 0x55b801384dd0 2023-10-16T14:33:36.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8013ee4a0 con 0x55b801385730 2023-10-16T14:33:36.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f0298fd3b80 10 client.0 _ll_register_callbacks cb 0x7f027800e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:33:36.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:59922/0 (socket says 172.21.15.178:59922) 2023-10-16T14:33:36.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 -- 172.21.15.178:0/3707377793 learned_addr learned my addr 172.21.15.178:0/3707377793 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:36.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027effd700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b801396140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027e7fc700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b801392be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 -- 172.21.15.178:0/3707377793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 msgr2=0x55b801392be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b801392be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 -- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 msgr2=0x55b801396140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:33:36.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b801396140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 -- 172.21.15.178:0/3707377793 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b801102d90 con 0x55b8013865d0 2023-10-16T14:33:36.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.550+0000 7f027f7fe700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0278009fd0 tx=0x7f02780213f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19228a1c70a8dd2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:36.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:33:36.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278004d40 con 0x55b8013865d0 2023-10-16T14:33:36.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027401d040 con 0x55b8013865d0 2023-10-16T14:33:36.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0278021e40 con 0x55b8013865d0 2023-10-16T14:33:36.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0278004ea0 con 0x55b8013865d0 2023-10-16T14:33:36.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0298fd3b80 10 client.15198 fetch_fsmap learned FSMap version 26 2023-10-16T14:33:36.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f02801a5730 con 0x55b8013865d0 2023-10-16T14:33:36.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f027401d040 con 0x55b8013865d0 2023-10-16T14:33:36.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0278008810 con 0x55b8013865d0 2023-10-16T14:33:36.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.551+0000 7f0298fd3b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:33:36.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0298fd3b80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:33:36.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0298fd3b80 20 client.15198 populate_metadata read hostname 'smithi178' 2023-10-16T14:33:36.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b8013f3730 0x55b8013efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:36.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0298fd3b80 4 client.15198 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:33:36.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b8012d0e00 con 0x55b8013f3730 2023-10-16T14:33:36.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f027effd700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b8013f3730 0x55b8013efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:36.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f027effd700 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b8013f3730 0x55b8013efb80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc54be790e31d09 server_cookie=ddaa09ca962d298b in_seq=0 out_seq=0 2023-10-16T14:33:36.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.552+0000 7f0262ff5700 10 client.15198 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:33:36.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0262ff5700 1 -- 172.21.15.178:0/3707377793 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x55b8012d0e00 con 0x55b8013f3730 2023-10-16T14:33:36.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0262ff5700 10 client.15198 handle_command_reply: tid=0 2023-10-16T14:33:36.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 client.15198 shutdown 2023-10-16T14:33:36.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 msgr2=0x55b801393180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0278009fd0 tx=0x7f02780213f0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 shutdown_connections 2023-10-16T14:33:36.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b8013f3730 0x55b8013efb80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8013865d0 0x55b801393180 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0262ff5700 0 client.15198 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:33:36.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b801384dd0 0x55b801396140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 --2- 172.21.15.178:0/3707377793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b801385730 0x55b801392be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 >> 172.21.15.178:0/3707377793 conn(0x55b8012b2d80 msgr2=0x55b801389530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 shutdown_connections 2023-10-16T14:33:36.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 1 -- 172.21.15.178:0/3707377793 wait complete. 2023-10-16T14:33:36.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.553+0000 7f0298fd3b80 20 client.15198 trim_cache size 0 max 16384 2023-10-16T14:33:36.572 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:33:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:33:36.572 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 0, 2023-10-16T14:33:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24593, 2023-10-16T14:33:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:33:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:33:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:33:36.573 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 119.50217508, 2023-10-16T14:33:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:33:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:33:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:33:36.574 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 125.648012204 2023-10-16T14:33:36.574 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:33:36.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 msgr2=0x7f02680a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 0x7f02680a3a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0278003e80 tx=0x7f027800b040 comp rx=0 tx=0).stop 2023-10-16T14:33:36.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 msgr2=0x7f028019d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:36.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f027000ec10 tx=0x7f027000ef20 comp rx=0 tx=0).stop 2023-10-16T14:33:36.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 shutdown_connections 2023-10-16T14:33:36.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f02680a1550 0x7f02680a3a10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f028010d4c0 0x7f02801a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0280073910 0x7f028019d230 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 --2- 172.21.15.178:0/256477999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0280074ef0 0x7f028019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:36.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 >> 172.21.15.178:0/256477999 conn(0x7f02800fc420 msgr2=0x7f02800fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:36.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 shutdown_connections 2023-10-16T14:33:36.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:36.554+0000 7f02855ff700 1 -- 172.21.15.178:0/256477999 wait complete. 2023-10-16T14:33:36.579 INFO:tasks.cephadm:Setup complete, yielding 2023-10-16T14:33:36.579 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-16T14:33:36.588 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:33:36.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch status' 2023-10-16T14:33:37.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:36 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1896482654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:37.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:36 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1896482654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:36 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1896482654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:33:37.353 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:38.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:37 smithi178 ceph-mon[98495]: pgmap v336: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:38.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:37 smithi178 ceph-mon[98495]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-16T14:33:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:37 smithi186 ceph-mon[134892]: pgmap v336: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:37 smithi186 ceph-mon[134892]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-16T14:33:38.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:37 smithi187 ceph-mon[135297]: pgmap v336: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:38.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:37 smithi187 ceph-mon[135297]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-10-16T14:33:39.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 -- 172.21.15.178:0/1130111675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 msgr2=0x7fef181093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:39.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 --2- 172.21.15.178:0/1130111675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181093c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef10009fc0 tx=0x7fef1000b330 comp rx=0 tx=0).stop 2023-10-16T14:33:39.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 -- 172.21.15.178:0/1130111675 shutdown_connections 2023-10-16T14:33:39.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 --2- 172.21.15.178:0/1130111675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181093c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 --2- 172.21.15.178:0/1130111675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 0x7fef18101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 --2- 172.21.15.178:0/1130111675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 0x7fef181025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 -- 172.21.15.178:0/1130111675 >> 172.21.15.178:0/1130111675 conn(0x7fef180fc4d0 msgr2=0x7fef180fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:39.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 -- 172.21.15.178:0/1130111675 shutdown_connections 2023-10-16T14:33:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.121+0000 7fef1c952700 1 -- 172.21.15.178:0/1130111675 wait complete. 2023-10-16T14:33:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.122+0000 7fef1c952700 1 Processor -- start 2023-10-16T14:33:39.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.122+0000 7fef1c952700 1 -- start start 2023-10-16T14:33:39.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 0x7fef18198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:39.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 0x7fef18199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:39.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:39.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181a0ae0 con 0x7fef18102b10 2023-10-16T14:33:39.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181a0c20 con 0x7fef18102170 2023-10-16T14:33:39.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1c952700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef181a0db0 con 0x7fef18100f70 2023-10-16T14:33:39.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:39.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60616/0 (socket says 172.21.15.178:60616) 2023-10-16T14:33:39.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 -- 172.21.15.178:0/3755284168 learned_addr learned my addr 172.21.15.178:0/3755284168 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef1659c700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 0x7fef18198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef15d9b700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 0x7fef18199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 -- 172.21.15.178:0/3755284168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 msgr2=0x7fef18198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 0x7fef18198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.123+0000 7fef16d9d700 1 -- 172.21.15.178:0/3755284168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 msgr2=0x7fef18199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:39.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef16d9d700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 0x7fef18199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef16d9d700 1 -- 172.21.15.178:0/3755284168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009c40 con 0x7fef18102b10 2023-10-16T14:33:39.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef16d9d700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fef10004c70 tx=0x7fef10004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:39.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1000e040 con 0x7fef18102b10 2023-10-16T14:33:39.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef1000bbf0 con 0x7fef18102b10 2023-10-16T14:33:39.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef181a1030 con 0x7fef18102b10 2023-10-16T14:33:39.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef100052a0 con 0x7fef18102b10 2023-10-16T14:33:39.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.124+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef181a14a0 con 0x7fef18102b10 2023-10-16T14:33:39.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.125+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef181935c0 con 0x7fef18102b10 2023-10-16T14:33:39.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.127+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fef1001d470 con 0x7fef18102b10 2023-10-16T14:33:39.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.128+0000 7fef077fe700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 0x7fef000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:39.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.129+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fef100cee30 con 0x7fef18102b10 2023-10-16T14:33:39.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.129+0000 7fef1659c700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 0x7fef000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:39.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.129+0000 7fef1659c700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 0x7fef000a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef18101fd0 tx=0x7fef08009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:39.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.131+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fef10098150 con 0x7fef18102b10 2023-10-16T14:33:39.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.344+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef1802d1a0 con 0x7fef000a1480 2023-10-16T14:33:39.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.345+0000 7fef077fe700 1 -- 172.21.15.178:0/3755284168 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fef1802d1a0 con 0x7fef000a1480 2023-10-16T14:33:39.349 INFO:teuthology.orchestra.run.smithi178.stdout:Backend: cephadm 2023-10-16T14:33:39.349 INFO:teuthology.orchestra.run.smithi178.stdout:Available: Yes 2023-10-16T14:33:39.349 INFO:teuthology.orchestra.run.smithi178.stdout:Paused: No 2023-10-16T14:33:39.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 msgr2=0x7fef000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:39.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 0x7fef000a3940 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef18101fd0 tx=0x7fef08009310 comp rx=0 tx=0).stop 2023-10-16T14:33:39.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 msgr2=0x7fef181a0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:39.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fef10004c70 tx=0x7fef10004ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 shutdown_connections 2023-10-16T14:33:39.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fef000a1480 0x7fef000a3940 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef18102b10 0x7fef181a0510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fef18100f70 0x7fef18198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 --2- 172.21.15.178:0/3755284168 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fef18102170 0x7fef18199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:39.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.347+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 >> 172.21.15.178:0/3755284168 conn(0x7fef180fc4d0 msgr2=0x7fef18107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:39.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.348+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 shutdown_connections 2023-10-16T14:33:39.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:39.348+0000 7fef1c952700 1 -- 172.21.15.178:0/3755284168 wait complete. 2023-10-16T14:33:39.951 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ps' 2023-10-16T14:33:40.016 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:39 smithi178 ceph-mon[98495]: pgmap v337: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:33:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:39 smithi186 ceph-mon[134892]: pgmap v337: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:33:40.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:39 smithi187 ceph-mon[135297]: pgmap v337: 129 pgs: 129 active+clean; 654 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:33:40.687 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:40.985 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:40 smithi178 ceph-mon[98495]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:41.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:40 smithi186 ceph-mon[134892]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:41.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:40 smithi187 ceph-mon[135297]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:42.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:41 smithi178 ceph-mon[98495]: pgmap v338: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:42.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:41 smithi186 ceph-mon[134892]: pgmap v338: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:42.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:41 smithi187 ceph-mon[135297]: pgmap v338: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:42.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 -- 172.21.15.178:0/1173677194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0100f70 msgr2=0x7fc0e0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:42.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/1173677194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0100f70 0x7fc0e0101390 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc0d00099c0 tx=0x7fc0d0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 -- 172.21.15.178:0/1173677194 shutdown_connections 2023-10-16T14:33:42.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/1173677194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0102b10 0x7fc0e01093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/1173677194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0100f70 0x7fc0e0101390 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/1173677194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102170 0x7fc0e01025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 -- 172.21.15.178:0/1173677194 >> 172.21.15.178:0/1173677194 conn(0x7fc0e00fc4d0 msgr2=0x7fc0e00fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:42.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 -- 172.21.15.178:0/1173677194 shutdown_connections 2023-10-16T14:33:42.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.320+0000 7fc0e4b40700 1 -- 172.21.15.178:0/1173677194 wait complete. 2023-10-16T14:33:42.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.321+0000 7fc0e4b40700 1 Processor -- start 2023-10-16T14:33:42.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.321+0000 7fc0e4b40700 1 -- start start 2023-10-16T14:33:42.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0100f70 0x7fc0e0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:42.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:42.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102b10 0x7fc0e01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:42.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01a09f0 con 0x7fc0e0102170 2023-10-16T14:33:42.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01a0b30 con 0x7fc0e0102b10 2023-10-16T14:33:42.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0e4b40700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e01a0cc0 con 0x7fc0e0100f70 2023-10-16T14:33:42.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:42.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60642/0 (socket says 172.21.15.178:60642) 2023-10-16T14:33:42.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 -- 172.21.15.178:0/3274664798 learned_addr learned my addr 172.21.15.178:0/3274664798 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:42.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0de59c700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0100f70 0x7fc0e0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:42.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ded9d700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102b10 0x7fc0e01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:42.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 -- 172.21.15.178:0/3274664798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0100f70 msgr2=0x7fc0e0198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:42.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0100f70 0x7fc0e0198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 -- 172.21.15.178:0/3274664798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102b10 msgr2=0x7fc0e01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:42.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.322+0000 7fc0ddd9b700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102b10 0x7fc0e01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0ddd9b700 1 -- 172.21.15.178:0/3274664798 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d00095f0 con 0x7fc0e0102170 2023-10-16T14:33:42.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0ddd9b700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400d980 tx=0x7fc0d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:42.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40098d0 con 0x7fc0e0102170 2023-10-16T14:33:42.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0d4010460 con 0x7fc0e0102170 2023-10-16T14:33:42.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e01a0fa0 con 0x7fc0e0102170 2023-10-16T14:33:42.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400f620 con 0x7fc0e0102170 2023-10-16T14:33:42.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.323+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e01a1460 con 0x7fc0e0102170 2023-10-16T14:33:42.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.324+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e01935b0 con 0x7fc0e0102170 2023-10-16T14:33:42.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.326+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc0d4010860 con 0x7fc0e0102170 2023-10-16T14:33:42.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.328+0000 7fc0cf7fe700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 0x7fc0c80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:42.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.328+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0d40ccdb0 con 0x7fc0e0102170 2023-10-16T14:33:42.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.329+0000 7fc0de59c700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 0x7fc0c80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:42.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.329+0000 7fc0de59c700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 0x7fc0c80a39c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009990 tx=0x7fc0d0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:42.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.333+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc0d40d1070 con 0x7fc0e0102170 2023-10-16T14:33:42.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.540+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e002d2b0 con 0x7fc0c80a1500 2023-10-16T14:33:42.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.554+0000 7fc0cf7fe700 1 -- 172.21.15.178:0/3274664798 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4092 (secure 0 0 0) 0x7fc0e002d2b0 con 0x7fc0c80a1500 2023-10-16T14:33:42.557 INFO:teuthology.orchestra.run.smithi178.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-10-16T14:33:42.558 INFO:teuthology.orchestra.run.smithi178.stdout:mds.a smithi178 running (2m) 42s ago 2m 13.7M - 18.0.0-6730-g4ba8a190 6c414e27a22d b8fc1d471b3a 2023-10-16T14:33:42.558 INFO:teuthology.orchestra.run.smithi178.stdout:mds.b smithi186 running (2m) 60s ago 2m 33.3M - 18.0.0-6730-g4ba8a190 6c414e27a22d a340a1d15ad3 2023-10-16T14:33:42.558 INFO:teuthology.orchestra.run.smithi178.stdout:mds.c smithi187 running (2m) 60s ago 2m 18.7M - 18.0.0-6730-g4ba8a190 6c414e27a22d 90fd5d93f022 2023-10-16T14:33:42.558 INFO:teuthology.orchestra.run.smithi178.stdout:mds.d smithi178 running (2m) 42s ago 2m 13.8M - 18.0.0-6730-g4ba8a190 6c414e27a22d 9f05cdfcc262 2023-10-16T14:33:42.559 INFO:teuthology.orchestra.run.smithi178.stdout:mds.e smithi186 running (2m) 60s ago 2m 19.6M - 18.0.0-6730-g4ba8a190 6c414e27a22d 989d7470f469 2023-10-16T14:33:42.559 INFO:teuthology.orchestra.run.smithi178.stdout:mds.f smithi187 running (2m) 60s ago 2m 17.5M - 18.0.0-6730-g4ba8a190 6c414e27a22d 3cc719f8eae2 2023-10-16T14:33:42.559 INFO:teuthology.orchestra.run.smithi178.stdout:mds.g smithi178 running (2m) 42s ago 2m 13.8M - 18.0.0-6730-g4ba8a190 6c414e27a22d 9ea88cecb601 2023-10-16T14:33:42.559 INFO:teuthology.orchestra.run.smithi178.stdout:mds.h smithi186 running (2m) 60s ago 2m 20.7M - 18.0.0-6730-g4ba8a190 6c414e27a22d c1a4aa4afb05 2023-10-16T14:33:42.560 INFO:teuthology.orchestra.run.smithi178.stdout:mds.i smithi187 running (2m) 60s ago 2m 19.7M - 18.0.0-6730-g4ba8a190 6c414e27a22d b9fad82e4ea7 2023-10-16T14:33:42.560 INFO:teuthology.orchestra.run.smithi178.stdout:mds.j smithi178 running (2m) 42s ago 2m 13.9M - 18.0.0-6730-g4ba8a190 6c414e27a22d 562f27405a8a 2023-10-16T14:33:42.560 INFO:teuthology.orchestra.run.smithi178.stdout:mds.k smithi186 running (2m) 60s ago 2m 15.1M - 18.0.0-6730-g4ba8a190 6c414e27a22d 3f0981aa9daa 2023-10-16T14:33:42.560 INFO:teuthology.orchestra.run.smithi178.stdout:mds.l smithi187 running (2m) 60s ago 2m 17.3M - 18.0.0-6730-g4ba8a190 6c414e27a22d 8e8bdfcd5269 2023-10-16T14:33:42.560 INFO:teuthology.orchestra.run.smithi178.stdout:mgr.x smithi178 *:9283,8765 running (9m) 42s ago 9m 527M - 18.0.0-6730-g4ba8a190 6c414e27a22d 8ff480ef2ed5 2023-10-16T14:33:42.561 INFO:teuthology.orchestra.run.smithi178.stdout:mgr.y smithi186 *:8443,8765 running (7m) 60s ago 7m 450M - 18.0.0-6730-g4ba8a190 6c414e27a22d 4be6c61c8e58 2023-10-16T14:33:42.561 INFO:teuthology.orchestra.run.smithi178.stdout:mgr.z smithi187 *:8443,8765 running (7m) 60s ago 7m 452M - 18.0.0-6730-g4ba8a190 6c414e27a22d fe8e286c133b 2023-10-16T14:33:42.561 INFO:teuthology.orchestra.run.smithi178.stdout:mon.a smithi178 running (9m) 42s ago 9m 97.6M 2048M 18.0.0-6730-g4ba8a190 6c414e27a22d b22ad7fb7bba 2023-10-16T14:33:42.561 INFO:teuthology.orchestra.run.smithi178.stdout:mon.b smithi186 running (7m) 60s ago 7m 66.9M 2048M 18.0.0-6730-g4ba8a190 6c414e27a22d c242daa6e1cd 2023-10-16T14:33:42.562 INFO:teuthology.orchestra.run.smithi178.stdout:mon.c smithi187 running (7m) 60s ago 7m 64.8M 2048M 18.0.0-6730-g4ba8a190 6c414e27a22d 6289e3aa074a 2023-10-16T14:33:42.562 INFO:teuthology.orchestra.run.smithi178.stdout:osd.0 smithi178 running (7m) 42s ago 7m 71.1M 4317M 18.0.0-6730-g4ba8a190 6c414e27a22d 27e8c304ecab 2023-10-16T14:33:42.562 INFO:teuthology.orchestra.run.smithi178.stdout:osd.1 smithi186 running (6m) 60s ago 6m 77.9M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d e98d2c8fd557 2023-10-16T14:33:42.562 INFO:teuthology.orchestra.run.smithi178.stdout:osd.2 smithi187 running (6m) 60s ago 6m 76.6M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d e0808b4e78cd 2023-10-16T14:33:42.562 INFO:teuthology.orchestra.run.smithi178.stdout:osd.3 smithi178 running (6m) 42s ago 6m 66.6M 4317M 18.0.0-6730-g4ba8a190 6c414e27a22d c5bafe19ba55 2023-10-16T14:33:42.563 INFO:teuthology.orchestra.run.smithi178.stdout:osd.4 smithi186 running (5m) 60s ago 5m 63.7M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d d2493dd472a0 2023-10-16T14:33:42.563 INFO:teuthology.orchestra.run.smithi178.stdout:osd.5 smithi187 running (5m) 60s ago 5m 69.0M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d 4350d7c93191 2023-10-16T14:33:42.563 INFO:teuthology.orchestra.run.smithi178.stdout:osd.6 smithi178 running (4m) 42s ago 4m 62.0M 4317M 18.0.0-6730-g4ba8a190 6c414e27a22d 24bc20db4e4d 2023-10-16T14:33:42.564 INFO:teuthology.orchestra.run.smithi178.stdout:osd.7 smithi186 running (4m) 60s ago 4m 68.9M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d a3b2d949676b 2023-10-16T14:33:42.564 INFO:teuthology.orchestra.run.smithi178.stdout:osd.8 smithi187 running (4m) 60s ago 4m 62.2M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d 3215be50fbf7 2023-10-16T14:33:42.564 INFO:teuthology.orchestra.run.smithi178.stdout:osd.9 smithi178 running (3m) 42s ago 3m 60.7M 4317M 18.0.0-6730-g4ba8a190 6c414e27a22d 9dbcbeaaa376 2023-10-16T14:33:42.564 INFO:teuthology.orchestra.run.smithi178.stdout:osd.10 smithi186 running (3m) 60s ago 3m 61.7M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d 588ad4ef6671 2023-10-16T14:33:42.564 INFO:teuthology.orchestra.run.smithi178.stdout:osd.11 smithi187 running (3m) 60s ago 3m 54.8M 4292M 18.0.0-6730-g4ba8a190 6c414e27a22d fe89feac79eb 2023-10-16T14:33:42.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 msgr2=0x7fc0c80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:42.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 0x7fc0c80a39c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009990 tx=0x7fc0d0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 msgr2=0x7fc0e01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:42.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400d980 tx=0x7fc0d400dc90 comp rx=0 tx=0).stop 2023-10-16T14:33:42.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 shutdown_connections 2023-10-16T14:33:42.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0c80a1500 0x7fc0c80a39c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0e0102b10 0x7fc0e01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0e0100f70 0x7fc0e0198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 --2- 172.21.15.178:0/3274664798 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0e0102170 0x7fc0e01993a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:42.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 >> 172.21.15.178:0/3274664798 conn(0x7fc0e00fc4d0 msgr2=0x7fc0e0107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:42.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 shutdown_connections 2023-10-16T14:33:42.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:42.557+0000 7fc0e4b40700 1 -- 172.21.15.178:0/3274664798 wait complete. 2023-10-16T14:33:43.083 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch ls' 2023-10-16T14:33:43.817 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:44.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:43 smithi178 ceph-mon[98495]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:44.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:43 smithi178 ceph-mon[98495]: pgmap v339: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:44.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:43 smithi186 ceph-mon[134892]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:44.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:43 smithi186 ceph-mon[134892]: pgmap v339: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:44.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:43 smithi187 ceph-mon[135297]: from='client.15210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:43 smithi187 ceph-mon[135297]: pgmap v339: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-10-16T14:33:45.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/1479941119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 msgr2=0x7f9d841025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:45.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/1479941119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d841025d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9d740099c0 tx=0x7f9d74009cd0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/1479941119 shutdown_connections 2023-10-16T14:33:45.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/1479941119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 0x7f9d841093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/1479941119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/1479941119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d841025d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/1479941119 >> 172.21.15.178:0/1479941119 conn(0x7f9d840fc4d0 msgr2=0x7f9d840fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/1479941119 shutdown_connections 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.476+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/1479941119 wait complete. 2023-10-16T14:33:45.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.477+0000 7f9d8c2ae700 1 Processor -- start 2023-10-16T14:33:45.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.477+0000 7f9d8c2ae700 1 -- start start 2023-10-16T14:33:45.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.477+0000 7f9d8c2ae700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:45.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.477+0000 7f9d8c2ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:45.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8c2ae700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 0x7f9d840737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:45.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8c2ae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84073f60 con 0x7f9d84102170 2023-10-16T14:33:45.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8c2ae700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840740d0 con 0x7f9d84102b10 2023-10-16T14:33:45.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8c2ae700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d841a0cc0 con 0x7f9d84100f70 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d89849700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d89849700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60662/0 (socket says 172.21.15.178:60662) 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d89849700 1 -- 172.21.15.178:0/539501823 learned_addr learned my addr 172.21.15.178:0/539501823 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a84b700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 0x7f9d840737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:45.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 -- 172.21.15.178:0/539501823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 msgr2=0x7f9d840737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:45.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 0x7f9d840737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 -- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 msgr2=0x7f9d840732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:45.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d8a04a700 1 -- 172.21.15.178:0/539501823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740095f0 con 0x7f9d84100f70 2023-10-16T14:33:45.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.478+0000 7f9d89849700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:45.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d8a04a700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84074c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000e9c0 tx=0x7f9d8000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:45.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80004220 con 0x7f9d84100f70 2023-10-16T14:33:45.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d841a0fa0 con 0x7f9d84100f70 2023-10-16T14:33:45.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d80004380 con 0x7f9d84100f70 2023-10-16T14:33:45.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80003be0 con 0x7f9d84100f70 2023-10-16T14:33:45.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.479+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d841a13b0 con 0x7f9d84100f70 2023-10-16T14:33:45.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.480+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d84107eb0 con 0x7f9d84100f70 2023-10-16T14:33:45.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.482+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9d80003d40 con 0x7f9d84100f70 2023-10-16T14:33:45.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.483+0000 7f9d7b7fe700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 0x7f9d700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:45.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.483+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9d80014070 con 0x7f9d84100f70 2023-10-16T14:33:45.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.483+0000 7f9d89849700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 0x7f9d700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:45.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.484+0000 7f9d89849700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 0x7f9d700a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d74004800 tx=0x7f9d7401a040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:45.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.485+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9d800969e0 con 0x7f9d84100f70 2023-10-16T14:33:45.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.693+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d8410c7f0 con 0x7f9d700a1480 2023-10-16T14:33:45.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.699+0000 7f9d7b7fe700 1 -- 172.21.15.178:0/539501823 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f9d8410c7f0 con 0x7f9d700a1480 2023-10-16T14:33:45.702 INFO:teuthology.orchestra.run.smithi178.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-10-16T14:33:45.702 INFO:teuthology.orchestra.run.smithi178.stdout:mds.all 12/12 63s ago 2m smithi178=a;smithi178=d;smithi178=g;smithi178=j;smithi186=b;smithi186=e;smithi186=h;smithi186=k;smithi187=c;smithi187=f;smithi187=i;smithi187=l;count:12 2023-10-16T14:33:45.702 INFO:teuthology.orchestra.run.smithi178.stdout:mgr 3/3 63s ago 7m smithi178=x;smithi186=y;smithi187=z;count:3 2023-10-16T14:33:45.702 INFO:teuthology.orchestra.run.smithi178.stdout:mon 3/3 63s ago 8m smithi178:172.21.15.178=a;smithi186:172.21.15.186=b;smithi187:172.21.15.187=c;count:3 2023-10-16T14:33:45.703 INFO:teuthology.orchestra.run.smithi178.stdout:osd 12 63s ago - 2023-10-16T14:33:45.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 msgr2=0x7f9d700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:45.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 0x7f9d700a3940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9d74004800 tx=0x7f9d7401a040 comp rx=0 tx=0).stop 2023-10-16T14:33:45.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 msgr2=0x7f9d84074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:45.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84074c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000e9c0 tx=0x7f9d8000ecd0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 shutdown_connections 2023-10-16T14:33:45.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9d700a1480 0x7f9d700a3940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9d84102b10 0x7f9d840737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9d84100f70 0x7f9d84074c50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 --2- 172.21.15.178:0/539501823 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84102170 0x7f9d840732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:45.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 >> 172.21.15.178:0/539501823 conn(0x7f9d840fc4d0 msgr2=0x7f9d840fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:45.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 shutdown_connections 2023-10-16T14:33:45.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:45.701+0000 7f9d8c2ae700 1 -- 172.21.15.178:0/539501823 wait complete. 2023-10-16T14:33:46.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:45 smithi178 ceph-mon[98495]: pgmap v340: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:45 smithi186 ceph-mon[134892]: pgmap v340: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:45 smithi187 ceph-mon[135297]: pgmap v340: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:46.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch host ls' 2023-10-16T14:33:46.950 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:47.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:46 smithi178 ceph-mon[98495]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:47.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:46 smithi186 ceph-mon[134892]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:47.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:46 smithi187 ceph-mon[135297]: from='client.25246 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:48.005 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:47 smithi178 ceph-mon[98495]: pgmap v341: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:47 smithi186 ceph-mon[134892]: pgmap v341: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:47 smithi187 ceph-mon[135297]: pgmap v341: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:48.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.711+0000 7f157d1c6700 1 -- 172.21.15.178:0/282083446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578102b10 msgr2=0x7f15781093c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:48.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.711+0000 7f157d1c6700 1 --2- 172.21.15.178:0/282083446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578102b10 0x7f15781093c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f157000b0d0 tx=0x7f157000b3e0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 -- 172.21.15.178:0/282083446 shutdown_connections 2023-10-16T14:33:48.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 --2- 172.21.15.178:0/282083446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578102b10 0x7f15781093c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 --2- 172.21.15.178:0/282083446 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578100f70 0x7f1578101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 --2- 172.21.15.178:0/282083446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15781025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 -- 172.21.15.178:0/282083446 >> 172.21.15.178:0/282083446 conn(0x7f15780fc4d0 msgr2=0x7f15780fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:48.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 -- 172.21.15.178:0/282083446 shutdown_connections 2023-10-16T14:33:48.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.712+0000 7f157d1c6700 1 -- 172.21.15.178:0/282083446 wait complete. 2023-10-16T14:33:48.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 Processor -- start 2023-10-16T14:33:48.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 -- start start 2023-10-16T14:33:48.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:48.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:48.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 0x7f15780737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:48.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1578073f60 con 0x7f1578100f70 2023-10-16T14:33:48.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780740d0 con 0x7f1578102b10 2023-10-16T14:33:48.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157d1c6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15781a0dc0 con 0x7f1578102170 2023-10-16T14:33:48.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:48.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:48.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f1576d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60680/0 (socket says 172.21.15.178:60680) 2023-10-16T14:33:48.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f1576d9d700 1 -- 172.21.15.178:0/2888601591 learned_addr learned my addr 172.21.15.178:0/2888601591 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:48.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.713+0000 7f157659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15780732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:41846/0 (socket says 172.21.15.178:41846) 2023-10-16T14:33:48.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f157759e700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 0x7f15780737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:48.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 -- 172.21.15.178:0/2888601591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 msgr2=0x7f15780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:48.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15780732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 -- 172.21.15.178:0/2888601591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 msgr2=0x7f15780737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:48.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 0x7f15780737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 -- 172.21.15.178:0/2888601591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570009d00 con 0x7f1578100f70 2023-10-16T14:33:48.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f157759e700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 0x7f15780737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f1576d9d700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f156800e970 tx=0x7f156800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.714+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568009790 con 0x7f1578100f70 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.715+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f156800c7e0 con 0x7f1578100f70 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.715+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1568003c70 con 0x7f1578100f70 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.715+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15781a10a0 con 0x7f1578100f70 2023-10-16T14:33:48.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.715+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15781a15a0 con 0x7f1578100f70 2023-10-16T14:33:48.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.716+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15781084b0 con 0x7f1578100f70 2023-10-16T14:33:48.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.717+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f156800cc50 con 0x7f1578100f70 2023-10-16T14:33:48.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.719+0000 7f155ffff700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 0x7f15600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:48.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.719+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1568014070 con 0x7f1578100f70 2023-10-16T14:33:48.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.719+0000 7f157659c700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 0x7f15600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:48.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.720+0000 7f157659c700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 0x7f15600a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f156c009c60 tx=0x7f156c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:48.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.723+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f15680964a0 con 0x7f1578100f70 2023-10-16T14:33:48.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.929+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f157802d210 con 0x7f15600a1500 2023-10-16T14:33:48.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.930+0000 7f155ffff700 1 -- 172.21.15.178:0/2888601591 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f157802d210 con 0x7f15600a1500 2023-10-16T14:33:48.933 INFO:teuthology.orchestra.run.smithi178.stdout:HOST ADDR LABELS STATUS 2023-10-16T14:33:48.933 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178 172.21.15.178 2023-10-16T14:33:48.934 INFO:teuthology.orchestra.run.smithi178.stdout:smithi186 172.21.15.186 2023-10-16T14:33:48.934 INFO:teuthology.orchestra.run.smithi178.stdout:smithi187 172.21.15.187 2023-10-16T14:33:48.934 INFO:teuthology.orchestra.run.smithi178.stdout:3 hosts in cluster 2023-10-16T14:33:48.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 msgr2=0x7f15600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:48.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 0x7f15600a39c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f156c009c60 tx=0x7f156c009260 comp rx=0 tx=0).stop 2023-10-16T14:33:48.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 msgr2=0x7f1578074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:48.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f156800e970 tx=0x7f156800ec80 comp rx=0 tx=0).stop 2023-10-16T14:33:48.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 shutdown_connections 2023-10-16T14:33:48.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f15600a1500 0x7f15600a39c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1578102b10 0x7f15780737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578100f70 0x7f1578074c50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 --2- 172.21.15.178:0/2888601591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1578102170 0x7f15780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:48.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 >> 172.21.15.178:0/2888601591 conn(0x7f15780fc4d0 msgr2=0x7f15781055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:48.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 shutdown_connections 2023-10-16T14:33:48.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:48.933+0000 7f157d1c6700 1 -- 172.21.15.178:0/2888601591 wait complete. 2023-10-16T14:33:49.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph orch device ls' 2023-10-16T14:33:50.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:49 smithi178 ceph-mon[98495]: pgmap v342: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:50.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:49 smithi178 ceph-mon[98495]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:49 smithi186 ceph-mon[134892]: pgmap v342: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:49 smithi186 ceph-mon[134892]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:50.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:49 smithi187 ceph-mon[135297]: pgmap v342: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-10-16T14:33:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:49 smithi187 ceph-mon[135297]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:50.284 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:51.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:51 smithi178 ceph-mon[98495]: pgmap v343: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2023-10-16T14:33:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:51 smithi186 ceph-mon[134892]: pgmap v343: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2023-10-16T14:33:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:51 smithi187 ceph-mon[135297]: pgmap v343: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2023-10-16T14:33:51.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 -- 172.21.15.178:0/4054626839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 msgr2=0x7fbc941090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:51.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 --2- 172.21.15.178:0/4054626839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc941090d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00b0a0 tx=0x7fbc8c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 -- 172.21.15.178:0/4054626839 shutdown_connections 2023-10-16T14:33:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 --2- 172.21.15.178:0/4054626839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc941090d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 --2- 172.21.15.178:0/4054626839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94101a00 0x7fbc94103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 --2- 172.21.15.178:0/4054626839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94104330 0x7fbc94106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 -- 172.21.15.178:0/4054626839 >> 172.21.15.178:0/4054626839 conn(0x7fbc940fb3f0 msgr2=0x7fbc940fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:51.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 -- 172.21.15.178:0/4054626839 shutdown_connections 2023-10-16T14:33:51.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.936+0000 7fbc98d81700 1 -- 172.21.15.178:0/4054626839 wait complete. 2023-10-16T14:33:51.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.937+0000 7fbc98d81700 1 Processor -- start 2023-10-16T14:33:51.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.937+0000 7fbc98d81700 1 -- start start 2023-10-16T14:33:51.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.937+0000 7fbc98d81700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94101a00 0x7fbc94101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:51.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc98d81700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:51.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc98d81700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc940ffef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:51.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc98d81700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc94100670 con 0x7fbc94106c60 2023-10-16T14:33:51.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc98d81700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941007e0 con 0x7fbc94101a00 2023-10-16T14:33:51.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc98d81700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc941a0dd0 con 0x7fbc94104330 2023-10-16T14:33:51.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:51.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36946/0 (socket says 172.21.15.178:36946) 2023-10-16T14:33:51.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc92d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc940ffef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:51.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 -- 172.21.15.178:0/1050395086 learned_addr learned my addr 172.21.15.178:0/1050395086 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:51.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc9259c700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94101a00 0x7fbc94101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:51.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 -- 172.21.15.178:0/1050395086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94101a00 msgr2=0x7fbc94101360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:51.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94101a00 0x7fbc94101360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 -- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 msgr2=0x7fbc940ffef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:51.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc940ffef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:51.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc91d9b700 1 -- 172.21.15.178:0/1050395086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c009d00 con 0x7fbc94104330 2023-10-16T14:33:51.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.938+0000 7fbc92d9d700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc940ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:51.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc91d9b700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9a0 tx=0x7fbc8800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:51.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88009790 con 0x7fbc94104330 2023-10-16T14:33:51.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc941a10b0 con 0x7fbc94104330 2023-10-16T14:33:51.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc88010430 con 0x7fbc94104330 2023-10-16T14:33:51.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88018680 con 0x7fbc94104330 2023-10-16T14:33:51.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.939+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc941a1600 con 0x7fbc94104330 2023-10-16T14:33:51.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.940+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc940fcf70 con 0x7fbc94104330 2023-10-16T14:33:51.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.942+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbc880105a0 con 0x7fbc94104330 2023-10-16T14:33:51.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.943+0000 7fbc837fe700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 0x7fbc7c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:51.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.943+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbc88014070 con 0x7fbc94104330 2023-10-16T14:33:51.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.944+0000 7fbc9259c700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 0x7fbc7c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:51.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.944+0000 7fbc9259c700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 0x7fbc7c0a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009c60 tx=0x7fbc84009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:51.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:51.946+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbc88096700 con 0x7fbc94104330 2023-10-16T14:33:52.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.156+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc9402d160 con 0x7fbc7c0a1480 2023-10-16T14:33:52.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.158+0000 7fbc837fe700 1 -- 172.21.15.178:0/1050395086 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fbc9402d160 con 0x7fbc7c0a1480 2023-10-16T14:33:52.161 INFO:teuthology.orchestra.run.smithi178.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-10-16T14:33:52.161 INFO:teuthology.orchestra.run.smithi178.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 3m ago Has a FileSystem, LVM detected 2023-10-16T14:33:52.161 INFO:teuthology.orchestra.run.smithi178.stdout:smithi186 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300CB400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-10-16T14:33:52.162 INFO:teuthology.orchestra.run.smithi178.stdout:smithi187 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2023-10-16T14:33:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.160+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 msgr2=0x7fbc7c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:52.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.160+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 0x7fbc7c0a3940 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009c60 tx=0x7fbc84009260 comp rx=0 tx=0).stop 2023-10-16T14:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.160+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 msgr2=0x7fbc940ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.160+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9a0 tx=0x7fbc8800ecb0 comp rx=0 tx=0).stop 2023-10-16T14:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 shutdown_connections 2023-10-16T14:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbc7c0a1480 0x7fbc7c0a3940 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:52.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc94106c60 0x7fbc940ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbc94101a00 0x7fbc94101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 --2- 172.21.15.178:0/1050395086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc94104330 0x7fbc940ff9b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 >> 172.21.15.178:0/1050395086 conn(0x7fbc940fb3f0 msgr2=0x7fbc941026f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 shutdown_connections 2023-10-16T14:33:52.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:52.161+0000 7fbc98d81700 1 -- 172.21.15.178:0/1050395086 wait complete. 2023-10-16T14:33:52.681 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-10-16T14:33:52.691 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:33:52.691 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph fs dump' 2023-10-16T14:33:53.427 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:54.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:53 smithi178 ceph-mon[98495]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:54.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:53 smithi178 ceph-mon[98495]: pgmap v344: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-16T14:33:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:53 smithi186 ceph-mon[134892]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:53 smithi186 ceph-mon[134892]: pgmap v344: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-16T14:33:54.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:53 smithi187 ceph-mon[135297]: from='client.25258 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-10-16T14:33:54.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:53 smithi187 ceph-mon[135297]: pgmap v344: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 5 op/s 2023-10-16T14:33:55.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 -- 172.21.15.178:0/4229610443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 msgr2=0x7f876c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 --2- 172.21.15.178:0/4229610443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c073d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8760009990 tx=0x7f8760009ca0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 -- 172.21.15.178:0/4229610443 shutdown_connections 2023-10-16T14:33:55.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 --2- 172.21.15.178:0/4229610443 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 0x7f876c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 --2- 172.21.15.178:0/4229610443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 --2- 172.21.15.178:0/4229610443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c073d90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 -- 172.21.15.178:0/4229610443 >> 172.21.15.178:0/4229610443 conn(0x7f876c0fc4d0 msgr2=0x7f876c0fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 -- 172.21.15.178:0/4229610443 shutdown_connections 2023-10-16T14:33:55.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.118+0000 7f877292c700 1 -- 172.21.15.178:0/4229610443 wait complete. 2023-10-16T14:33:55.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.119+0000 7f877292c700 1 Processor -- start 2023-10-16T14:33:55.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.119+0000 7f877292c700 1 -- start start 2023-10-16T14:33:55.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.119+0000 7f877292c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:55.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.119+0000 7f877292c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c19d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:55.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f877292c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 0x7f876c1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:55.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f877292c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1a4df0 con 0x7f876c073910 2023-10-16T14:33:55.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f877292c700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1a4f30 con 0x7f876c10d510 2023-10-16T14:33:55.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f877292c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1a50c0 con 0x7f876c074ee0 2023-10-16T14:33:55.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:55.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c19d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:55.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40372/0 (socket says 172.21.15.178:40372) 2023-10-16T14:33:55.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876bfff700 1 -- 172.21.15.178:0/118675684 learned_addr learned my addr 172.21.15.178:0/118675684 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:55.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f8770ec9700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 0x7f876c1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:55.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 -- 172.21.15.178:0/118675684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 msgr2=0x7f876c1a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:55.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 0x7f876c1a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 -- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 msgr2=0x7f876c19d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:55.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876b7fe700 1 -- 172.21.15.178:0/118675684 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600095f0 con 0x7f876c074ee0 2023-10-16T14:33:55.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.120+0000 7f876bfff700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:55.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f876b7fe700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c19d7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f876000b680 tx=0x7f8760004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:55.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876001d070 con 0x7f876c074ee0 2023-10-16T14:33:55.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c1a5340 con 0x7f876c074ee0 2023-10-16T14:33:55.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c1a5750 con 0x7f876c074ee0 2023-10-16T14:33:55.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8760005030 con 0x7f876c074ee0 2023-10-16T14:33:55.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.121+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760004b00 con 0x7f876c074ee0 2023-10-16T14:33:55.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.122+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c103060 con 0x7f876c074ee0 2023-10-16T14:33:55.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.124+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8760005430 con 0x7f876c074ee0 2023-10-16T14:33:55.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.125+0000 7f87697fa700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 0x7f87540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:55.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.125+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f87600ce3d0 con 0x7f876c074ee0 2023-10-16T14:33:55.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.125+0000 7f876bfff700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 0x7f87540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:55.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.126+0000 7f876bfff700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 0x7f87540a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f875c005950 tx=0x7f875c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:55.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.127+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8760097750 con 0x7f876c074ee0 2023-10-16T14:33:55.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.372+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f876c112da0 con 0x7f876c074ee0 2023-10-16T14:33:55.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.373+0000 7f87697fa700 1 -- 172.21.15.178:0/118675684 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+2974 (secure 0 0 0) 0x7f876009ad70 con 0x7f876c074ee0 2023-10-16T14:33:55.382 INFO:teuthology.orchestra.run.smithi178.stdout:e26 2023-10-16T14:33:55.383 INFO:teuthology.orchestra.run.smithi178.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-10-16T14:33:55.383 INFO:teuthology.orchestra.run.smithi178.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-16T14:33:55.383 INFO:teuthology.orchestra.run.smithi178.stdout:legacy client fscid: 1 2023-10-16T14:33:55.383 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:55.384 INFO:teuthology.orchestra.run.smithi178.stdout:Filesystem 'cephfs' (1) 2023-10-16T14:33:55.384 INFO:teuthology.orchestra.run.smithi178.stdout:fs_name cephfs 2023-10-16T14:33:55.384 INFO:teuthology.orchestra.run.smithi178.stdout:epoch 26 2023-10-16T14:33:55.384 INFO:teuthology.orchestra.run.smithi178.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-10-16T14:33:55.384 INFO:teuthology.orchestra.run.smithi178.stdout:created 2023-10-16T14:31:36.554542+0000 2023-10-16T14:33:55.385 INFO:teuthology.orchestra.run.smithi178.stdout:modified 2023-10-16T14:32:15.910283+0000 2023-10-16T14:33:55.385 INFO:teuthology.orchestra.run.smithi178.stdout:tableserver 0 2023-10-16T14:33:55.385 INFO:teuthology.orchestra.run.smithi178.stdout:root 0 2023-10-16T14:33:55.386 INFO:teuthology.orchestra.run.smithi178.stdout:session_timeout 300 2023-10-16T14:33:55.386 INFO:teuthology.orchestra.run.smithi178.stdout:session_autoclose 300 2023-10-16T14:33:55.386 INFO:teuthology.orchestra.run.smithi178.stdout:max_file_size 1099511627776 2023-10-16T14:33:55.386 INFO:teuthology.orchestra.run.smithi178.stdout:max_xattr_size 65536 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:required_client_features {} 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:last_failure 0 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:last_failure_osd_epoch 0 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:max_mds 3 2023-10-16T14:33:55.387 INFO:teuthology.orchestra.run.smithi178.stdout:in 0,1,2 2023-10-16T14:33:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:up {0=24593,1=24550,2=24548} 2023-10-16T14:33:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:failed 2023-10-16T14:33:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:damaged 2023-10-16T14:33:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:stopped 2023-10-16T14:33:55.388 INFO:teuthology.orchestra.run.smithi178.stdout:data_pools [3,4] 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:metadata_pool 2 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:inline_data disabled 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:balancer 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:bal_rank_mask -1 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:standby_count_wanted 0 2023-10-16T14:33:55.389 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b{0:24593} state up:active seq 3 export targets 1 addr [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.390 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c{0:24532} state up:standby-replay seq 1 addr [v2:172.21.15.187:6836/4191845467,v1:172.21.15.187:6837/4191845467] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.390 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.i{1:24550} state up:active seq 13 addr [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.390 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.f{1:24520} state up:standby-replay seq 1 addr [v2:172.21.15.187:6834/2797759968,v1:172.21.15.187:6835/2797759968] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.390 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.h{2:24548} state up:active seq 14 addr [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.390 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.e{2:24515} state up:standby-replay seq 1 addr [v2:172.21.15.186:6834/1810043383,v1:172.21.15.186:6835/1810043383] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:55.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:55.391 INFO:teuthology.orchestra.run.smithi178.stdout:Standby daemons: 2023-10-16T14:33:55.391 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:55.391 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.j{-1:14475} state up:standby seq 1 addr [v2:172.21.15.178:6834/1662555012,v1:172.21.15.178:6835/1662555012] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.g{-1:14535} state up:standby seq 1 addr [v2:172.21.15.178:6838/519971243,v1:172.21.15.178:6839/519971243] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a{-1:14547} state up:standby seq 1 addr [v2:172.21.15.178:6840/2221064860,v1:172.21.15.178:6841/2221064860] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.l{-1:24466} state up:standby seq 1 addr [v2:172.21.15.187:6832/1854237656,v1:172.21.15.187:6833/1854237656] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.k{-1:24479} state up:standby seq 1 addr [v2:172.21.15.186:6832/3571012275,v1:172.21.15.186:6833/3571012275] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.392 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d{-1:24503} state up:standby seq 1 addr [v2:172.21.15.178:6836/3662037973,v1:172.21.15.178:6837/3662037973] compat {c=[1],r=[1],i=[fff]}] 2023-10-16T14:33:55.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 msgr2=0x7f87540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:55.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 0x7f87540a3940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f875c005950 tx=0x7f875c009260 comp rx=0 tx=0).stop 2023-10-16T14:33:55.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 msgr2=0x7f876c19d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:55.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c19d7a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f876000b680 tx=0x7f8760004730 comp rx=0 tx=0).stop 2023-10-16T14:33:55.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 shutdown_connections 2023-10-16T14:33:55.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f87540a1480 0x7f87540a3940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f876c10d510 0x7f876c1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f876c073910 0x7f876c19d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 --2- 172.21.15.178:0/118675684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f876c074ee0 0x7f876c19d7a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:55.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.375+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 >> 172.21.15.178:0/118675684 conn(0x7f876c0fc4d0 msgr2=0x7f876c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:55.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.376+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 shutdown_connections 2023-10-16T14:33:55.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:55.376+0000 7f877292c700 1 -- 172.21.15.178:0/118675684 wait complete. 2023-10-16T14:33:55.395 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:33:55.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- bash -c 'ceph osd dump' 2023-10-16T14:33:55.994 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:55 smithi178 ceph-mon[98495]: pgmap v345: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:55.995 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:55 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/118675684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-16T14:33:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:55 smithi186 ceph-mon[134892]: pgmap v345: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:55 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/118675684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-16T14:33:56.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:55 smithi187 ceph-mon[135297]: pgmap v345: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:56.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:55 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/118675684' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-10-16T14:33:56.666 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:33:58.038 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:57 smithi178 ceph-mon[98495]: pgmap v346: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:57 smithi186 ceph-mon[134892]: pgmap v346: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:57 smithi187 ceph-mon[135297]: pgmap v346: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:33:58.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 -- 172.21.15.178:0/1681370395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4102170 msgr2=0x7f51c41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:58.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 --2- 172.21.15.178:0/1681370395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4102170 0x7f51c41025d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009920 tx=0x7f51b0009c30 comp rx=0 tx=0).stop 2023-10-16T14:33:58.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 -- 172.21.15.178:0/1681370395 shutdown_connections 2023-10-16T14:33:58.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 --2- 172.21.15.178:0/1681370395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 0x7f51c41093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 --2- 172.21.15.178:0/1681370395 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4100f70 0x7f51c4101390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 --2- 172.21.15.178:0/1681370395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4102170 0x7f51c41025d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 -- 172.21.15.178:0/1681370395 >> 172.21.15.178:0/1681370395 conn(0x7f51c40fc4d0 msgr2=0x7f51c40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:58.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 -- 172.21.15.178:0/1681370395 shutdown_connections 2023-10-16T14:33:58.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.290+0000 7f51cb011700 1 -- 172.21.15.178:0/1681370395 wait complete. 2023-10-16T14:33:58.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.291+0000 7f51cb011700 1 Processor -- start 2023-10-16T14:33:58.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 -- start start 2023-10-16T14:33:58.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:58.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:58.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 0x7f51c40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:58.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4073f60 con 0x7f51c4100f70 2023-10-16T14:33:58.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40740d0 con 0x7f51c4102170 2023-10-16T14:33:58.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51cb011700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c41a0d50 con 0x7f51c4102b10 2023-10-16T14:33:58.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c8dad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:58.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:58.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:36202/0 (socket says 172.21.15.178:36202) 2023-10-16T14:33:58.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c8dad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40384/0 (socket says 172.21.15.178:40384) 2023-10-16T14:33:58.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c3fff700 1 -- 172.21.15.178:0/997915843 learned_addr learned my addr 172.21.15.178:0/997915843 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:58.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.292+0000 7f51c95ae700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 0x7f51c40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:58.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 -- 172.21.15.178:0/997915843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 msgr2=0x7f51c40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:58.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 0x7f51c40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 -- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 msgr2=0x7f51c4074c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:58.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 -- 172.21.15.178:0/997915843 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0009520 con 0x7f51c4102170 2023-10-16T14:33:58.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c8dad700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:58.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c3fff700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009230 tx=0x7f51b000bf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:58.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b001c070 con 0x7f51c4102170 2023-10-16T14:33:58.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51b000bba0 con 0x7f51c4102170 2023-10-16T14:33:58.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c41a0fd0 con 0x7f51c4102170 2023-10-16T14:33:58.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0003a80 con 0x7f51c4102170 2023-10-16T14:33:58.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.293+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c41a1380 con 0x7f51c4102170 2023-10-16T14:33:58.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.294+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c4108610 con 0x7f51c4102170 2023-10-16T14:33:58.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.296+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f51b0020030 con 0x7f51c4102170 2023-10-16T14:33:58.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.298+0000 7f51c1ffb700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 0x7f51b40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:58.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.298+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f51b00ce3c0 con 0x7f51c4102170 2023-10-16T14:33:58.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.298+0000 7f51c8dad700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 0x7f51b40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:58.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.298+0000 7f51c8dad700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 0x7f51b40a39c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f51c4101fd0 tx=0x7f51b8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:58.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.300+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f51b00976e0 con 0x7f51c4102170 2023-10-16T14:33:58.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.500+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f51c410ca10 con 0x7f51c4102170 2023-10-16T14:33:58.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.501+0000 7f51c1ffb700 1 -- 172.21.15.178:0/997915843 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) v1 ==== 56+0+5707 (secure 0 0 0) 0x7f51b0005040 con 0x7f51c4102170 2023-10-16T14:33:58.505 INFO:teuthology.orchestra.run.smithi178.stdout:epoch 91 2023-10-16T14:33:58.505 INFO:teuthology.orchestra.run.smithi178.stdout:fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:33:58.505 INFO:teuthology.orchestra.run.smithi178.stdout:created 2023-10-16T14:24:07.673050+0000 2023-10-16T14:33:58.505 INFO:teuthology.orchestra.run.smithi178.stdout:modified 2023-10-16T14:33:08.029307+0000 2023-10-16T14:33:58.505 INFO:teuthology.orchestra.run.smithi178.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-10-16T14:33:58.506 INFO:teuthology.orchestra.run.smithi178.stdout:crush_version 27 2023-10-16T14:33:58.506 INFO:teuthology.orchestra.run.smithi178.stdout:full_ratio 0.95 2023-10-16T14:33:58.506 INFO:teuthology.orchestra.run.smithi178.stdout:backfillfull_ratio 0.9 2023-10-16T14:33:58.506 INFO:teuthology.orchestra.run.smithi178.stdout:nearfull_ratio 0.85 2023-10-16T14:33:58.506 INFO:teuthology.orchestra.run.smithi178.stdout:require_min_compat_client luminous 2023-10-16T14:33:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:min_compat_client luminous 2023-10-16T14:33:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:require_osd_release reef 2023-10-16T14:33:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:stretch_mode_enabled false 2023-10-16T14:33:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-10-16T14:33:58.507 INFO:teuthology.orchestra.run.smithi178.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-10-16T14:33:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-10-16T14:33:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-10-16T14:33:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:max_osd 12 2023-10-16T14:33:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6802/1189154044,v1:172.21.15.178:6803/1189154044] [v2:172.21.15.178:6804/1189154044,v1:172.21.15.178:6805/1189154044] exists,up 9a72b3ff-3836-4ade-969c-531e6616f47e 2023-10-16T14:33:58.508 INFO:teuthology.orchestra.run.smithi178.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6800/4108518172,v1:172.21.15.186:6801/4108518172] [v2:172.21.15.186:6802/4108518172,v1:172.21.15.186:6803/4108518172] exists,up 0811af9a-0421-41e8-be52-d60c284536d6 2023-10-16T14:33:58.509 INFO:teuthology.orchestra.run.smithi178.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6800/2818376874,v1:172.21.15.187:6801/2818376874] [v2:172.21.15.187:6802/2818376874,v1:172.21.15.187:6803/2818376874] exists,up f3786153-58c5-4fec-a44b-d3e5b81e9748 2023-10-16T14:33:58.509 INFO:teuthology.orchestra.run.smithi178.stdout:osd.3 up in weight 1 up_from 26 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6810/4190845983,v1:172.21.15.178:6811/4190845983] [v2:172.21.15.178:6812/4190845983,v1:172.21.15.178:6813/4190845983] exists,up 2b6e4a7c-b16b-4da0-a854-7d054ac16422 2023-10-16T14:33:58.509 INFO:teuthology.orchestra.run.smithi178.stdout:osd.4 up in weight 1 up_from 33 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6808/699064619,v1:172.21.15.186:6809/699064619] [v2:172.21.15.186:6810/699064619,v1:172.21.15.186:6811/699064619] exists,up 47410335-85bd-4755-92ee-1203b0cef71f 2023-10-16T14:33:58.509 INFO:teuthology.orchestra.run.smithi178.stdout:osd.5 up in weight 1 up_from 38 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6808/3921910211,v1:172.21.15.187:6809/3921910211] [v2:172.21.15.187:6810/3921910211,v1:172.21.15.187:6811/3921910211] exists,up 038ae36b-4552-4926-af63-dc0498229ee6 2023-10-16T14:33:58.509 INFO:teuthology.orchestra.run.smithi178.stdout:osd.6 up in weight 1 up_from 46 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6818/113445947,v1:172.21.15.178:6819/113445947] [v2:172.21.15.178:6820/113445947,v1:172.21.15.178:6821/113445947] exists,up 73f9efa5-f93a-48b9-83f7-af7901b11cc3 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6816/4049803928,v1:172.21.15.186:6817/4049803928] [v2:172.21.15.186:6818/4049803928,v1:172.21.15.186:6819/4049803928] exists,up dc91a577-422e-4074-a0f8-35c3be1cac6d 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:osd.8 up in weight 1 up_from 58 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6816/1521942991,v1:172.21.15.187:6817/1521942991] [v2:172.21.15.187:6818/1521942991,v1:172.21.15.187:6819/1521942991] exists,up 40ff8899-9c05-4331-8c78-e0d69b8571e6 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:osd.9 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.178:6826/2115000831,v1:172.21.15.178:6827/2115000831] [v2:172.21.15.178:6828/2115000831,v1:172.21.15.178:6829/2115000831] exists,up 22e44478-f2b6-4a84-a740-9bd31070b250 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:osd.10 up in weight 1 up_from 71 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.186:6824/2093939166,v1:172.21.15.186:6825/2093939166] [v2:172.21.15.186:6826/2093939166,v1:172.21.15.186:6827/2093939166] exists,up 2c36d6fd-c771-438e-916d-793a97beea8a 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:osd.11 up in weight 1 up_from 78 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6824/433464338,v1:172.21.15.187:6825/433464338] [v2:172.21.15.187:6826/433464338,v1:172.21.15.187:6827/433464338] exists,up dc27eb51-d50b-431d-9e60-f53a95da61af 2023-10-16T14:33:58.510 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.0 [5,6] 2023-10-16T14:33:58.511 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.b [2,4] 2023-10-16T14:33:58.511 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.16 [2,4] 2023-10-16T14:33:58.511 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.1a [8,1] 2023-10-16T14:33:58.511 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.1c [8,11] 2023-10-16T14:33:58.511 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 2.1f [7,11] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 3.1 [4,9] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 3.3 [4,9] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 3.21 [5,9] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 4.5 [9,10] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 4.6 [9,11] 2023-10-16T14:33:58.512 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 4.12 [9,10] 2023-10-16T14:33:58.513 INFO:teuthology.orchestra.run.smithi178.stdout:pg_upmap_items 4.1a [5,8] 2023-10-16T14:33:58.513 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:6800/3157107959 expires 2023-10-17T14:25:06.501803+0000 2023-10-16T14:33:58.513 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/2364174490 expires 2023-10-17T14:25:06.501803+0000 2023-10-16T14:33:58.513 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/2438067538 expires 2023-10-17T14:25:06.501803+0000 2023-10-16T14:33:58.513 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:6801/2960529415 expires 2023-10-17T14:24:39.404169+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:6801/3157107959 expires 2023-10-17T14:25:06.501803+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/3729267496 expires 2023-10-17T14:24:39.404169+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/3093526104 expires 2023-10-17T14:24:39.404169+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/1552969994 expires 2023-10-17T14:25:06.501803+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:6800/2960529415 expires 2023-10-17T14:24:39.404169+0000 2023-10-16T14:33:58.514 INFO:teuthology.orchestra.run.smithi178.stdout:blocklist 172.21.15.178:0/3372273467 expires 2023-10-17T14:24:39.404169+0000 2023-10-16T14:33:58.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 msgr2=0x7f51b40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:58.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 0x7f51b40a39c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f51c4101fd0 tx=0x7f51b8009260 comp rx=0 tx=0).stop 2023-10-16T14:33:58.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 msgr2=0x7f51c40732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:58.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009230 tx=0x7f51b000bf80 comp rx=0 tx=0).stop 2023-10-16T14:33:58.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 shutdown_connections 2023-10-16T14:33:58.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f51b40a1500 0x7f51b40a39c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4102b10 0x7f51c40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51c4100f70 0x7f51c4074c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 --2- 172.21.15.178:0/997915843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f51c4102170 0x7f51c40732a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 >> 172.21.15.178:0/997915843 conn(0x7f51c40fc4d0 msgr2=0x7f51c4105750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 shutdown_connections 2023-10-16T14:33:58.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:58.504+0000 7f51cb011700 1 -- 172.21.15.178:0/997915843 wait complete. 2023-10-16T14:33:58.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:58 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/997915843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-16T14:33:59.063 INFO:teuthology.run_tasks:Running task fs.ready... 2023-10-16T14:33:59.076 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:33:59.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:58 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/997915843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-16T14:33:59.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:58 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/997915843' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-10-16T14:33:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/2602381975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102a50 msgr2=0x7fa9a4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/2602381975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102a50 0x7fa9a4109370 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa99c00b0a0 tx=0x7fa99c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/2602381975 shutdown_connections 2023-10-16T14:33:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/2602381975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102a50 0x7fa9a4109370 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/2602381975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/2602381975 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102090 0x7fa9a4102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/2602381975 >> 172.21.15.178:0/2602381975 conn(0x7fa9a40fc430 msgr2=0x7fa9a40fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/2602381975 shutdown_connections 2023-10-16T14:33:59.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.276+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/2602381975 wait complete. 2023-10-16T14:33:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.277+0000 7fa9ac7ff700 1 Processor -- start 2023-10-16T14:33:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.277+0000 7fa9ac7ff700 1 -- start start 2023-10-16T14:33:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a4074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102a50 0x7fa9a40737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a4073f80 con 0x7fa9a4102090 2023-10-16T14:33:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a40740f0 con 0x7fa9a4102a50 2023-10-16T14:33:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9ac7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41a0d10 con 0x7fa9a4100eb0 2023-10-16T14:33:59.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9a9d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a4074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9a9d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43254/0 (socket says 172.21.15.178:43254) 2023-10-16T14:33:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9a9d9a700 1 -- 172.21.15.178:0/3065760418 learned_addr learned my addr 172.21.15.178:0/3065760418 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aad9c700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102a50 0x7fa9a40737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 -- 172.21.15.178:0/3065760418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102a50 msgr2=0x7fa9a40737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102a50 0x7fa9a40737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 -- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 msgr2=0x7fa9a40732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9aa59b700 1 -- 172.21.15.178:0/3065760418 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c009d00 con 0x7fa9a4100eb0 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.278+0000 7fa9a9d9a700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:33:59.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa9aa59b700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a4074c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000e9c0 tx=0x7fa9a000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:59.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0004220 con 0x7fa9a4100eb0 2023-10-16T14:33:59.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a41a0ff0 con 0x7fa9a4100eb0 2023-10-16T14:33:59.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9a000cc80 con 0x7fa9a4100eb0 2023-10-16T14:33:59.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0003be0 con 0x7fa9a4100eb0 2023-10-16T14:33:59.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.279+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a41a1470 con 0x7fa9a4100eb0 2023-10-16T14:33:59.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.280+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a4107d70 con 0x7fa9a4100eb0 2023-10-16T14:33:59.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.282+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa9a0003d40 con 0x7fa9a4100eb0 2023-10-16T14:33:59.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.283+0000 7fa99b7fe700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 0x7fa9900a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.284+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa9a0014070 con 0x7fa9a4100eb0 2023-10-16T14:33:59.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.284+0000 7fa9a9d9a700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 0x7fa9900a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.284+0000 7fa9a9d9a700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 0x7fa9900a39e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa994005950 tx=0x7fa994009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:59.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.285+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa9a00966e0 con 0x7fa9a4100eb0 2023-10-16T14:33:59.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.488+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9a402d2b0 con 0x7fa9a4100eb0 2023-10-16T14:33:59.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.490+0000 7fa99b7fe700 1 -- 172.21.15.178:0/3065760418 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fa9a0099d00 con 0x7fa9a4100eb0 2023-10-16T14:33:59.493 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:33:59.494 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:33:59.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 msgr2=0x7fa9900a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 0x7fa9900a39e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa994005950 tx=0x7fa994009260 comp rx=0 tx=0).stop 2023-10-16T14:33:59.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 msgr2=0x7fa9a4074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a4074c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000e9c0 tx=0x7fa9a000ecd0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 shutdown_connections 2023-10-16T14:33:59.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa9900a1520 0x7fa9900a39e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa9a4102a50 0x7fa9a40737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9a4100eb0 0x7fa9a4074c60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 --2- 172.21.15.178:0/3065760418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9a4102090 0x7fa9a40732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 >> 172.21.15.178:0/3065760418 conn(0x7fa9a40fc430 msgr2=0x7fa9a40fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:59.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 shutdown_connections 2023-10-16T14:33:59.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.493+0000 7fa9ac7ff700 1 -- 172.21.15.178:0/3065760418 wait complete. 2023-10-16T14:33:59.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:33:59.515 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:33:59.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 -- 172.21.15.178:0/2626055788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 msgr2=0x7f1ef0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/2626055788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef0109360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ee800b0a0 tx=0x7f1ee800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 -- 172.21.15.178:0/2626055788 shutdown_connections 2023-10-16T14:33:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/2626055788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef0109360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/2626055788 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0100ea0 0x7f1ef01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/2626055788 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0102080 0x7f1ef0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 -- 172.21.15.178:0/2626055788 >> 172.21.15.178:0/2626055788 conn(0x7f1ef00fc420 msgr2=0x7f1ef00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:59.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 -- 172.21.15.178:0/2626055788 shutdown_connections 2023-10-16T14:33:59.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.716+0000 7f1ef4fff700 1 -- 172.21.15.178:0/2626055788 wait complete. 2023-10-16T14:33:59.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.717+0000 7f1ef4fff700 1 Processor -- start 2023-10-16T14:33:59.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.717+0000 7f1ef4fff700 1 -- start start 2023-10-16T14:33:59.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 0x7f1ef0074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0102080 0x7f1ef00732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef0073f80 con 0x7f1ef0102080 2023-10-16T14:33:59.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00740f0 con 0x7f1ef0100ea0 2023-10-16T14:33:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ef4fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01a0d60 con 0x7f1ef0102a40 2023-10-16T14:33:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eee59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0102080 0x7f1ef00732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54302/0 (socket says 172.21.15.178:54302) 2023-10-16T14:33:59.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 -- 172.21.15.178:0/510711665 learned_addr learned my addr 172.21.15.178:0/510711665 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:33:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ee659c700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 0x7f1ef0074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 -- 172.21.15.178:0/510711665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 msgr2=0x7f1ef0074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 0x7f1ef0074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 -- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0102080 msgr2=0x7f1ef00732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0102080 0x7f1ef00732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1eeed9d700 1 -- 172.21.15.178:0/510711665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8009d00 con 0x7f1ef0102a40 2023-10-16T14:33:59.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.718+0000 7f1ee659c700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 0x7f1ef0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:33:59.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1eeed9d700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:59.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800e040 con 0x7f1ef0102a40 2023-10-16T14:33:59.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef01a0fe0 con 0x7f1ef0102a40 2023-10-16T14:33:59.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ee80094a0 con 0x7f1ef0102a40 2023-10-16T14:33:59.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801b6a0 con 0x7f1ef0102a40 2023-10-16T14:33:59.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.719+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef01a1400 con 0x7f1ef0102a40 2023-10-16T14:33:59.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.720+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef010c760 con 0x7f1ef0102a40 2023-10-16T14:33:59.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.722+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1ee801b800 con 0x7f1ef0102a40 2023-10-16T14:33:59.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.723+0000 7f1ee7fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 0x7f1edc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:33:59.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.723+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1ee80ce170 con 0x7f1ef0102a40 2023-10-16T14:33:59.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.724+0000 7f1ee659c700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 0x7f1edc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:33:59.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.724+0000 7f1ee659c700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 0x7f1edc0a3940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0101bf0 tx=0x7f1ee0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:33:59.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.725+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1ee8097490 con 0x7f1ef0102a40 2023-10-16T14:33:59.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:59 smithi178 ceph-mon[98495]: pgmap v347: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:33:59.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:33:59 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3065760418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:33:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.927+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef0107da0 con 0x7f1ef0102a40 2023-10-16T14:33:59.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.928+0000 7f1ee7fff700 1 -- 172.21.15.178:0/510711665 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ee809aab0 con 0x7f1ef0102a40 2023-10-16T14:33:59.931 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:33:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 msgr2=0x7f1edc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 0x7f1edc0a3940 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0101bf0 tx=0x7f1ee0009260 comp rx=0 tx=0).stop 2023-10-16T14:33:59.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 msgr2=0x7f1ef00737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:33:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8009fd0 tx=0x7f1ee800bce0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 shutdown_connections 2023-10-16T14:33:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1edc0a1480 0x7f1edc0a3940 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ef0102a40 0x7f1ef00737e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1ef0100ea0 0x7f1ef0074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 --2- 172.21.15.178:0/510711665 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ef0102080 0x7f1ef00732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:33:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 >> 172.21.15.178:0/510711665 conn(0x7f1ef00fc420 msgr2=0x7f1ef00fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:33:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 shutdown_connections 2023-10-16T14:33:59.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:33:59.930+0000 7f1ef4fff700 1 -- 172.21.15.178:0/510711665 wait complete. 2023-10-16T14:33:59.952 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:34:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:59 smithi186 ceph-mon[134892]: pgmap v347: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:33:59 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3065760418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:00.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 -- 172.21.15.178:0/2870124617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 msgr2=0x7fc5d41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/2870124617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d41012c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009990 tx=0x7fc5c4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 -- 172.21.15.178:0/2870124617 shutdown_connections 2023-10-16T14:34:00.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/2870124617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102a40 0x7fc5d4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/2870124617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d41012c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.152+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/2870124617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102080 0x7fc5d4102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.153+0000 7fc5d9bff700 1 -- 172.21.15.178:0/2870124617 >> 172.21.15.178:0/2870124617 conn(0x7fc5d40fc420 msgr2=0x7fc5d40fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:00.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.153+0000 7fc5d9bff700 1 -- 172.21.15.178:0/2870124617 shutdown_connections 2023-10-16T14:34:00.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.153+0000 7fc5d9bff700 1 -- 172.21.15.178:0/2870124617 wait complete. 2023-10-16T14:34:00.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 Processor -- start 2023-10-16T14:34:00.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 -- start start 2023-10-16T14:34:00.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102080 0x7fc5d41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102a40 0x7fc5d419e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419e820 con 0x7fc5d4100ea0 2023-10-16T14:34:00.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419e990 con 0x7fc5d4102a40 2023-10-16T14:34:00.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d9bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d419eb50 con 0x7fc5d4102080 2023-10-16T14:34:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43286/0 (socket says 172.21.15.178:43286) 2023-10-16T14:34:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d37fe700 1 -- 172.21.15.178:0/1361722631 learned_addr learned my addr 172.21.15.178:0/1361722631 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d3fff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102a40 0x7fc5d419e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.154+0000 7fc5d2ffd700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102080 0x7fc5d41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 -- 172.21.15.178:0/1361722631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102080 msgr2=0x7fc5d41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102080 0x7fc5d41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 -- 172.21.15.178:0/1361722631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102a40 msgr2=0x7fc5d419e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102a40 0x7fc5d419e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 -- 172.21.15.178:0/1361722631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40095f0 con 0x7fc5d4100ea0 2023-10-16T14:34:00.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d37fe700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005720 tx=0x7fc5c4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:00.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c401d070 con 0x7fc5d4100ea0 2023-10-16T14:34:00.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5c4021410 con 0x7fc5d4100ea0 2023-10-16T14:34:00.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4017650 con 0x7fc5d4100ea0 2023-10-16T14:34:00.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d419edd0 con 0x7fc5d4100ea0 2023-10-16T14:34:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.155+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d419f1a0 con 0x7fc5d4100ea0 2023-10-16T14:34:00.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.156+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d410c760 con 0x7fc5d4100ea0 2023-10-16T14:34:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.158+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc5c4021580 con 0x7fc5d4100ea0 2023-10-16T14:34:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.160+0000 7fc5d0ff9700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 0x7fc5bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.160+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc5c40ce650 con 0x7fc5d4100ea0 2023-10-16T14:34:00.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.160+0000 7fc5d2ffd700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 0x7fc5bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.160+0000 7fc5d2ffd700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 0x7fc5bc0a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5d41981b0 tx=0x7fc5c8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:00.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.162+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc5c4097970 con 0x7fc5d4100ea0 2023-10-16T14:34:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:59 smithi187 ceph-mon[135297]: pgmap v347: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:33:59 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3065760418' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:00.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.365+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5d4107ee0 con 0x7fc5d4100ea0 2023-10-16T14:34:00.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.366+0000 7fc5d0ff9700 1 -- 172.21.15.178:0/1361722631 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5c409af90 con 0x7fc5d4100ea0 2023-10-16T14:34:00.369 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 msgr2=0x7fc5bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 0x7fc5bc0a39c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc5d41981b0 tx=0x7fc5c8009260 comp rx=0 tx=0).stop 2023-10-16T14:34:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 msgr2=0x7fc5d4196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005720 tx=0x7fc5c4009580 comp rx=0 tx=0).stop 2023-10-16T14:34:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 shutdown_connections 2023-10-16T14:34:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc5bc0a1500 0x7fc5bc0a39c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc5d4102a40 0x7fc5d419e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc5d4100ea0 0x7fc5d4196cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 --2- 172.21.15.178:0/1361722631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5d4102080 0x7fc5d41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 >> 172.21.15.178:0/1361722631 conn(0x7fc5d40fc420 msgr2=0x7fc5d40fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 shutdown_connections 2023-10-16T14:34:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.368+0000 7fc5d9bff700 1 -- 172.21.15.178:0/1361722631 wait complete. 2023-10-16T14:34:00.390 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:34:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 -- 172.21.15.178:0/3628439928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 msgr2=0x7f857c106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 --2- 172.21.15.178:0/3628439928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c106720 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8568009990 tx=0x7f8568009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 -- 172.21.15.178:0/3628439928 shutdown_connections 2023-10-16T14:34:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 --2- 172.21.15.178:0/3628439928 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c106c60 0x7f857c109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 --2- 172.21.15.178:0/3628439928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c101a00 0x7f857c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 --2- 172.21.15.178:0/3628439928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c106720 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 -- 172.21.15.178:0/3628439928 >> 172.21.15.178:0/3628439928 conn(0x7f857c0fb370 msgr2=0x7f857c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 -- 172.21.15.178:0/3628439928 shutdown_connections 2023-10-16T14:34:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.587+0000 7f8582fff700 1 -- 172.21.15.178:0/3628439928 wait complete. 2023-10-16T14:34:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 Processor -- start 2023-10-16T14:34:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 -- start start 2023-10-16T14:34:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c106c60 0x7f857c19e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19e780 con 0x7f857c104330 2023-10-16T14:34:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19e8f0 con 0x7f857c101a00 2023-10-16T14:34:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8582fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857c19eab0 con 0x7f857c106c60 2023-10-16T14:34:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f857bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8580d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8580d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:47934/0 (socket says 172.21.15.178:47934) 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f857bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43310/0 (socket says 172.21.15.178:43310) 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.588+0000 7f8580d9b700 1 -- 172.21.15.178:0/3328824326 learned_addr learned my addr 172.21.15.178:0/3328824326 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f858159c700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c106c60 0x7f857c19e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 -- 172.21.15.178:0/3328824326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c106c60 msgr2=0x7f857c19e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c106c60 0x7f857c19e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 -- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 msgr2=0x7f857c197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 -- 172.21.15.178:0/3328824326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85680095f0 con 0x7f857c101a00 2023-10-16T14:34:00.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f857bfff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:00.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8580d9b700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9a0 tx=0x7f857000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570009790 con 0x7f857c101a00 2023-10-16T14:34:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857c19ed90 con 0x7f857c101a00 2023-10-16T14:34:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8570010430 con 0x7f857c101a00 2023-10-16T14:34:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.589+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570018680 con 0x7f857c101a00 2023-10-16T14:34:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.590+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857c19f1e0 con 0x7f857c101a00 2023-10-16T14:34:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.590+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f857c0fcef0 con 0x7f857c101a00 2023-10-16T14:34:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.592+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f85700105a0 con 0x7f857c101a00 2023-10-16T14:34:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.594+0000 7f8579ffb700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 0x7f856c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.594+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8570014070 con 0x7f857c101a00 2023-10-16T14:34:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.594+0000 7f857bfff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 0x7f856c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.594+0000 7f857bfff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 0x7f856c0a3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f856800b2e0 tx=0x7f8568005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.596+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8570096520 con 0x7f857c101a00 2023-10-16T14:34:00.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.799+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f857c02d1a0 con 0x7f857c101a00 2023-10-16T14:34:00.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.800+0000 7f8579ffb700 1 -- 172.21.15.178:0/3328824326 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8570099b40 con 0x7f857c101a00 2023-10-16T14:34:00.803 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:34:00.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.802+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 msgr2=0x7f856c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.802+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 0x7f856c0a3940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f856800b2e0 tx=0x7f8568005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.802+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 msgr2=0x7f857c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:00.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.802+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9a0 tx=0x7f857000ecb0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 shutdown_connections 2023-10-16T14:34:00.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f856c0a1480 0x7f856c0a3940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f857c106c60 0x7f857c19e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f857c101a00 0x7f857c196c40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 --2- 172.21.15.178:0/3328824326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f857c104330 0x7f857c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:00.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 >> 172.21.15.178:0/3328824326 conn(0x7f857c0fb370 msgr2=0x7f857c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:00.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 shutdown_connections 2023-10-16T14:34:00.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:00.803+0000 7f8582fff700 1 -- 172.21.15.178:0/3328824326 wait complete. 2023-10-16T14:34:00.824 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:34:01.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 -- 172.21.15.178:0/3745735409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c073910 msgr2=0x7fb41c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 --2- 172.21.15.178:0/3745735409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c073910 0x7fb41c073d90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb410009990 tx=0x7fb410009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 -- 172.21.15.178:0/3745735409 shutdown_connections 2023-10-16T14:34:01.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 --2- 172.21.15.178:0/3745735409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c108db0 0x7fb41c10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 --2- 172.21.15.178:0/3745735409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 --2- 172.21.15.178:0/3745735409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c073910 0x7fb41c073d90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 -- 172.21.15.178:0/3745735409 >> 172.21.15.178:0/3745735409 conn(0x7fb41c0fc420 msgr2=0x7fb41c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:01.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 -- 172.21.15.178:0/3745735409 shutdown_connections 2023-10-16T14:34:01.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.022+0000 7fb421bff700 1 -- 172.21.15.178:0/3745735409 wait complete. 2023-10-16T14:34:01.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.023+0000 7fb421bff700 1 Processor -- start 2023-10-16T14:34:01.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 -- start start 2023-10-16T14:34:01.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 0x7fb41c198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 0x7fb41c1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a0990 con 0x7fb41c108db0 2023-10-16T14:34:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a0ad0 con 0x7fb41c074ef0 2023-10-16T14:34:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb421bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1a0c60 con 0x7fb41c073910 2023-10-16T14:34:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb41affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb41bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 0x7fb41c1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb41b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 0x7fb41c198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb41affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:47958/0 (socket says 172.21.15.178:47958) 2023-10-16T14:34:01.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.024+0000 7fb41affd700 1 -- 172.21.15.178:0/613218917 learned_addr learned my addr 172.21.15.178:0/613218917 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:01.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 -- 172.21.15.178:0/613218917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 msgr2=0x7fb41c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 -- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 msgr2=0x7fb41c1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 0x7fb41c1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 -- 172.21.15.178:0/613218917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100095f0 con 0x7fb41c073910 2023-10-16T14:34:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41bfff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 0x7fb41c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:01.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb41b7fe700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 0x7fb41c198e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00e9f0 tx=0x7fb40c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00c9c0 con 0x7fb41c073910 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c1a0f40 con 0x7fb41c073910 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb40c010430 con 0x7fb41c073910 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.025+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c018680 con 0x7fb41c073910 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.026+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c1a1370 con 0x7fb41c073910 2023-10-16T14:34:01.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.026+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c107460 con 0x7fb41c073910 2023-10-16T14:34:01.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.029+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb40c010860 con 0x7fb41c073910 2023-10-16T14:34:01.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.030+0000 7fb418ff9700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 0x7fb4040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.030+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb40c014070 con 0x7fb41c073910 2023-10-16T14:34:01.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.030+0000 7fb41affd700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 0x7fb4040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.031+0000 7fb41affd700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 0x7fb4040a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb41000b2e0 tx=0x7fb410005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.032+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb40c096620 con 0x7fb41c073910 2023-10-16T14:34:01.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/510711665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1361722631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/510711665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1361722631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/510711665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1361722631' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:01.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.240+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb41c10ebe0 con 0x7fb41c073910 2023-10-16T14:34:01.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.241+0000 7fb418ff9700 1 -- 172.21.15.178:0/613218917 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb40c026020 con 0x7fb41c073910 2023-10-16T14:34:01.243 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:01.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 msgr2=0x7fb4040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 0x7fb4040a3940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb41000b2e0 tx=0x7fb410005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 msgr2=0x7fb41c198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 0x7fb41c198e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00e9f0 tx=0x7fb40c00ed00 comp rx=0 tx=0).stop 2023-10-16T14:34:01.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 shutdown_connections 2023-10-16T14:34:01.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb4040a1480 0x7fb4040a3940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb41c108db0 0x7fb41c1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb41c073910 0x7fb41c198e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 --2- 172.21.15.178:0/613218917 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb41c074ef0 0x7fb41c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 >> 172.21.15.178:0/613218917 conn(0x7fb41c0fc420 msgr2=0x7fb41c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:01.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 shutdown_connections 2023-10-16T14:34:01.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.243+0000 7fb421bff700 1 -- 172.21.15.178:0/613218917 wait complete. 2023-10-16T14:34:01.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:34:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 -- 172.21.15.178:0/2692390861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88104340 msgr2=0x7fbf881047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/2692390861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88104340 0x7fbf881047a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009920 tx=0x7fbf78009c30 comp rx=0 tx=0).stop 2023-10-16T14:34:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 -- 172.21.15.178:0/2692390861 shutdown_connections 2023-10-16T14:34:01.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/2692390861 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 0x7fbf8810b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/2692390861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103150 0x7fbf88103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/2692390861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88104340 0x7fbf881047a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 -- 172.21.15.178:0/2692390861 >> 172.21.15.178:0/2692390861 conn(0x7fbf880fe6c0 msgr2=0x7fbf88100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:01.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 -- 172.21.15.178:0/2692390861 shutdown_connections 2023-10-16T14:34:01.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.462+0000 7fbf8fdff700 1 -- 172.21.15.178:0/2692390861 wait complete. 2023-10-16T14:34:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.463+0000 7fbf8fdff700 1 Processor -- start 2023-10-16T14:34:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.463+0000 7fbf8fdff700 1 -- start start 2023-10-16T14:34:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88104340 0x7fbf88199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 0x7fbf881a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881a0990 con 0x7fbf88103150 2023-10-16T14:34:01.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881a0ad0 con 0x7fbf88104ce0 2023-10-16T14:34:01.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8fdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881a0c60 con 0x7fbf88104340 2023-10-16T14:34:01.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43346/0 (socket says 172.21.15.178:43346) 2023-10-16T14:34:01.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 -- 172.21.15.178:0/3627549522 learned_addr learned my addr 172.21.15.178:0/3627549522 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:01.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8d39a700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88104340 0x7fbf88199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8e39c700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 0x7fbf881a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 -- 172.21.15.178:0/3627549522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88104340 msgr2=0x7fbf88199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88104340 0x7fbf88199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 -- 172.21.15.178:0/3627549522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 msgr2=0x7fbf881a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 0x7fbf881a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.464+0000 7fbf8db9b700 1 -- 172.21.15.178:0/3627549522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840095f0 con 0x7fbf88103150 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf8db9b700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400ebf0 tx=0x7fbf8400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84009a10 con 0x7fbf88103150 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf8400cc90 con 0x7fbf88103150 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf78009520 con 0x7fbf88103150 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84018750 con 0x7fbf88103150 2023-10-16T14:34:01.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.465+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf881a1220 con 0x7fbf88103150 2023-10-16T14:34:01.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.466+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8810e9c0 con 0x7fbf88103150 2023-10-16T14:34:01.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.468+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fbf840188b0 con 0x7fbf88103150 2023-10-16T14:34:01.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.469+0000 7fbf7effd700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 0x7fbf740a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.469+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbf84014070 con 0x7fbf88103150 2023-10-16T14:34:01.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.470+0000 7fbf8d39a700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 0x7fbf740a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.470+0000 7fbf8d39a700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 0x7fbf740a3a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800fa40 tx=0x7fbf78019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.474+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fbf840968d0 con 0x7fbf88103150 2023-10-16T14:34:01.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.720+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf8810a000 con 0x7fbf88103150 2023-10-16T14:34:01.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.721+0000 7fbf7effd700 1 -- 172.21.15.178:0/3627549522 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7fbf84099ef0 con 0x7fbf88103150 2023-10-16T14:34:01.724 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:01.724 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:34:01.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 msgr2=0x7fbf740a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 0x7fbf740a3a60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800fa40 tx=0x7fbf78019040 comp rx=0 tx=0).stop 2023-10-16T14:34:01.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 msgr2=0x7fbf88198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400ebf0 tx=0x7fbf8400ef00 comp rx=0 tx=0).stop 2023-10-16T14:34:01.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 shutdown_connections 2023-10-16T14:34:01.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbf740a15a0 0x7fbf740a3a60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fbf88104ce0 0x7fbf881a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf88103150 0x7fbf88198e10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 --2- 172.21.15.178:0/3627549522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88104340 0x7fbf88199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 >> 172.21.15.178:0/3627549522 conn(0x7fbf880fe6c0 msgr2=0x7fbf88100870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:01.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 shutdown_connections 2023-10-16T14:34:01.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.724+0000 7fbf8fdff700 1 -- 172.21.15.178:0/3627549522 wait complete. 2023-10-16T14:34:01.729 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:34:01.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:01.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 -- 172.21.15.178:0/3705337376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304103110 msgr2=0x7ff304103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3705337376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304103110 0x7ff304103530 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff3000099c0 tx=0x7ff300009cd0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 -- 172.21.15.178:0/3705337376 shutdown_connections 2023-10-16T14:34:01.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3705337376 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104ca0 0x7ff30410b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3705337376 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304103110 0x7ff304103530 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3705337376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304104300 0x7ff304104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 -- 172.21.15.178:0/3705337376 >> 172.21.15.178:0/3705337376 conn(0x7ff3040fe660 msgr2=0x7ff304100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:01.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 -- 172.21.15.178:0/3705337376 shutdown_connections 2023-10-16T14:34:01.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.941+0000 7ff30bfff700 1 -- 172.21.15.178:0/3705337376 wait complete. 2023-10-16T14:34:01.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 Processor -- start 2023-10-16T14:34:01.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 -- start start 2023-10-16T14:34:01.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 0x7ff304199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 0x7ff3041a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041a08c0 con 0x7ff304104ca0 2023-10-16T14:34:01.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041a0a00 con 0x7ff304104300 2023-10-16T14:34:01.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.942+0000 7ff30bfff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041a0b90 con 0x7ff304103110 2023-10-16T14:34:01.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff30a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 0x7ff3041a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff30a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 0x7ff3041a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43366/0 (socket says 172.21.15.178:43366) 2023-10-16T14:34:01.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff30a59c700 1 -- 172.21.15.178:0/3316878106 learned_addr learned my addr 172.21.15.178:0/3316878106 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:01.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54384/0 (socket says 172.21.15.178:54384) 2023-10-16T14:34:01.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff30959a700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 0x7ff304199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 -- 172.21.15.178:0/3316878106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 msgr2=0x7ff304199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 0x7ff304199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 -- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 msgr2=0x7ff3041a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:01.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 0x7ff3041a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:01.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 -- 172.21.15.178:0/3316878106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3000095f0 con 0x7ff304103110 2023-10-16T14:34:01.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff30959a700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 0x7ff304199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:01.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.943+0000 7ff309d9b700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3000098e0 tx=0x7ff30000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.944+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30001d070 con 0x7ff304103110 2023-10-16T14:34:01.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.944+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3041a0e10 con 0x7ff304103110 2023-10-16T14:34:01.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.944+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff300005510 con 0x7ff304103110 2023-10-16T14:34:01.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.944+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff300004b70 con 0x7ff304103110 2023-10-16T14:34:01.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.944+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041a1180 con 0x7ff304103110 2023-10-16T14:34:01.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.945+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30410a040 con 0x7ff304103110 2023-10-16T14:34:01.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.947+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff300004d10 con 0x7ff304103110 2023-10-16T14:34:01.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.948+0000 7ff2faffd700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 0x7ff2f00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:01.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.948+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff3000cdea0 con 0x7ff304103110 2023-10-16T14:34:01.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.948+0000 7ff30959a700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 0x7ff2f00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:01.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.949+0000 7ff30959a700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 0x7ff2f00a39e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009cb0 tx=0x7ff2f4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:01.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:01.950+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3000971c0 con 0x7ff304103110 2023-10-16T14:34:02.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:01 smithi178 ceph-mon[98495]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:02.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:01 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3328824326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:01 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/613218917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:01 smithi186 ceph-mon[134892]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:02.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:01 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3328824326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:01 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/613218917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.155+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff30410e980 con 0x7ff304103110 2023-10-16T14:34:02.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.157+0000 7ff2faffd700 1 -- 172.21.15.178:0/3316878106 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7ff30009a7e0 con 0x7ff304103110 2023-10-16T14:34:02.160 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:02.161 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:02.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 msgr2=0x7ff2f00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 0x7ff2f00a39e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009cb0 tx=0x7ff2f4009260 comp rx=0 tx=0).stop 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 msgr2=0x7ff304198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff3000098e0 tx=0x7ff30000f690 comp rx=0 tx=0).stop 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 shutdown_connections 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2f00a1520 0x7ff2f00a39e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff304104ca0 0x7ff3041a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff304103110 0x7ff304198d40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 --2- 172.21.15.178:0/3316878106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff304104300 0x7ff304199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 >> 172.21.15.178:0/3316878106 conn(0x7ff3040fe660 msgr2=0x7ff304100900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:02.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 shutdown_connections 2023-10-16T14:34:02.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.159+0000 7ff30bfff700 1 -- 172.21.15.178:0/3316878106 wait complete. 2023-10-16T14:34:02.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:01 smithi187 ceph-mon[135297]: pgmap v348: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:02.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:01 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3328824326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:01 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/613218917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:02.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:34:02.182 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:34:02.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.405+0000 7f00463ff700 1 -- 172.21.15.178:0/1127687559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102a10 msgr2=0x7f0040109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.405+0000 7f00463ff700 1 --2- 172.21.15.178:0/1127687559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102a10 0x7f0040109330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0038009f90 tx=0x7f003800b330 comp rx=0 tx=0).stop 2023-10-16T14:34:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 -- 172.21.15.178:0/1127687559 shutdown_connections 2023-10-16T14:34:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 --2- 172.21.15.178:0/1127687559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102a10 0x7f0040109330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 --2- 172.21.15.178:0/1127687559 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040100e70 0x7f0040101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 --2- 172.21.15.178:0/1127687559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040102050 0x7f00401024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 -- 172.21.15.178:0/1127687559 >> 172.21.15.178:0/1127687559 conn(0x7f00400fc430 msgr2=0x7f00400fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 -- 172.21.15.178:0/1127687559 shutdown_connections 2023-10-16T14:34:02.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.406+0000 7f00463ff700 1 -- 172.21.15.178:0/1127687559 wait complete. 2023-10-16T14:34:02.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 Processor -- start 2023-10-16T14:34:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 -- start start 2023-10-16T14:34:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 0x7f0040198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 0x7f0040199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 0x7f00401a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00401a0a40 con 0x7f0040100e70 2023-10-16T14:34:02.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00401a0b80 con 0x7f0040102a10 2023-10-16T14:34:02.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f00463ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00401a0d10 con 0x7f0040102050 2023-10-16T14:34:02.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f0037fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 0x7f0040198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f0037fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 0x7f0040198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43394/0 (socket says 172.21.15.178:43394) 2023-10-16T14:34:02.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f0037fff700 1 -- 172.21.15.178:0/1139575851 learned_addr learned my addr 172.21.15.178:0/1139575851 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:02.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.407+0000 7f003ffff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 0x7f0040199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f004499c700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 0x7f00401a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 -- 172.21.15.178:0/1139575851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 msgr2=0x7f00401a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 0x7f00401a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 -- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 msgr2=0x7f0040198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 0x7f0040198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 -- 172.21.15.178:0/1139575851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0038009c40 con 0x7f0040102050 2023-10-16T14:34:02.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f004499c700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 0x7f00401a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:02.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003ffff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 0x7f0040199400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0028005fd0 tx=0x7f002800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:02.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028004220 con 0x7f0040102050 2023-10-16T14:34:02.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0028004380 con 0x7f0040102050 2023-10-16T14:34:02.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401a0ff0 con 0x7f0040102050 2023-10-16T14:34:02.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028003b50 con 0x7f0040102050 2023-10-16T14:34:02.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.408+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00401a1470 con 0x7f0040102050 2023-10-16T14:34:02.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.409+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004010c730 con 0x7f0040102050 2023-10-16T14:34:02.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.412+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0028003cd0 con 0x7f0040102050 2023-10-16T14:34:02.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.413+0000 7f003dffb700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 0x7f002c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.413+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0028014070 con 0x7f0040102050 2023-10-16T14:34:02.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.413+0000 7f0037fff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 0x7f002c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.414+0000 7f0037fff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 0x7f002c0a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0030005950 tx=0x7f0030009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:02.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.415+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f0028096630 con 0x7f0040102050 2023-10-16T14:34:02.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.625+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00401a16c0 con 0x7f0040102050 2023-10-16T14:34:02.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.626+0000 7f003dffb700 1 -- 172.21.15.178:0/1139575851 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0028099c50 con 0x7f0040102050 2023-10-16T14:34:02.629 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:34:02.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 msgr2=0x7f002c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 0x7f002c0a3940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0030005950 tx=0x7f0030009260 comp rx=0 tx=0).stop 2023-10-16T14:34:02.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 msgr2=0x7f0040199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 0x7f0040199400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0028005fd0 tx=0x7f002800eca0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 shutdown_connections 2023-10-16T14:34:02.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f002c0a1480 0x7f002c0a3940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0040102a10 0x7f00401a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0040100e70 0x7f0040198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 --2- 172.21.15.178:0/1139575851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0040102050 0x7f0040199400 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 >> 172.21.15.178:0/1139575851 conn(0x7f00400fc430 msgr2=0x7f0040107980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:02.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 shutdown_connections 2023-10-16T14:34:02.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.629+0000 7f00463ff700 1 -- 172.21.15.178:0/1139575851 wait complete. 2023-10-16T14:34:02.651 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:34:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 -- 172.21.15.178:0/3613983232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c102050 msgr2=0x7fc00c1024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3613983232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c102050 0x7fc00c1024d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009990 tx=0x7fbff8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 -- 172.21.15.178:0/3613983232 shutdown_connections 2023-10-16T14:34:02.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3613983232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3613983232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c100e70 0x7fc00c101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3613983232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c102050 0x7fc00c1024d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 -- 172.21.15.178:0/3613983232 >> 172.21.15.178:0/3613983232 conn(0x7fc00c0fc430 msgr2=0x7fc00c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 -- 172.21.15.178:0/3613983232 shutdown_connections 2023-10-16T14:34:02.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.864+0000 7fc0131ff700 1 -- 172.21.15.178:0/3613983232 wait complete. 2023-10-16T14:34:02.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 Processor -- start 2023-10-16T14:34:02.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 -- start start 2023-10-16T14:34:02.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 0x7fc00c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c102050 0x7fc00c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1a09c0 con 0x7fc00c100e70 2023-10-16T14:34:02.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1a0b00 con 0x7fc00c102050 2023-10-16T14:34:02.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.865+0000 7fc0131ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c1a0c90 con 0x7fc00c102a10 2023-10-16T14:34:02.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc010f9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 0x7fc00c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54422/0 (socket says 172.21.15.178:54422) 2023-10-16T14:34:02.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 -- 172.21.15.178:0/3945296774 learned_addr learned my addr 172.21.15.178:0/3945296774 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:02.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc00bfff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c102050 0x7fc00c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 -- 172.21.15.178:0/3945296774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c102050 msgr2=0x7fc00c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c102050 0x7fc00c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 -- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 msgr2=0x7fc00c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:02.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 0x7fc00c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:02.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 -- 172.21.15.178:0/3945296774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff80095f0 con 0x7fc00c102a10 2023-10-16T14:34:02.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc010f9b700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 0x7fc00c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc01179c700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc00400ef30 tx=0x7fc00400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004018070 con 0x7fc00c102a10 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc00400f040 con 0x7fc00c102a10 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.866+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c1a0f10 con 0x7fc00c102a10 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.867+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00400cd00 con 0x7fc00c102a10 2023-10-16T14:34:02.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.867+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c1a1340 con 0x7fc00c102a10 2023-10-16T14:34:02.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.868+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c10c730 con 0x7fc00c102a10 2023-10-16T14:34:02.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.869+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc004003a00 con 0x7fc00c102a10 2023-10-16T14:34:02.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.871+0000 7fc009ffb700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 0x7fbffc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:02.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.871+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0040ccbe0 con 0x7fc00c102a10 2023-10-16T14:34:02.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.871+0000 7fc010f9b700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 0x7fbffc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:02.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.871+0000 7fc010f9b700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 0x7fbffc0a3940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc00c101bc0 tx=0x7fc000009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:02.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:02.874+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc004095f00 con 0x7fc00c102a10 2023-10-16T14:34:03.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3627549522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3316878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:02 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1139575851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3627549522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3316878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:02 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1139575851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:03.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.088+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc00c1a1530 con 0x7fc00c102a10 2023-10-16T14:34:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.089+0000 7fc009ffb700 1 -- 172.21.15.178:0/3945296774 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc004099520 con 0x7fc00c102a10 2023-10-16T14:34:03.093 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 msgr2=0x7fbffc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 0x7fbffc0a3940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc00c101bc0 tx=0x7fc000009260 comp rx=0 tx=0).stop 2023-10-16T14:34:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 msgr2=0x7fc00c1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc00400ef30 tx=0x7fc00400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 shutdown_connections 2023-10-16T14:34:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fbffc0a1480 0x7fbffc0a3940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc00c102a10 0x7fc00c1a03f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc00c100e70 0x7fc00c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 --2- 172.21.15.178:0/3945296774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc00c102050 0x7fc00c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 >> 172.21.15.178:0/3945296774 conn(0x7fc00c0fc430 msgr2=0x7fc00c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 shutdown_connections 2023-10-16T14:34:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.092+0000 7fc0131ff700 1 -- 172.21.15.178:0/3945296774 wait complete. 2023-10-16T14:34:03.114 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:34:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3627549522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3316878106' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:34:03.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:02 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1139575851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:03.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 -- 172.21.15.178:0/4124438128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec10d4a0 msgr2=0x7f7fec10f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/4124438128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec10d4a0 0x7f7fec10f890 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400b0a0 tx=0x7f7fe400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 -- 172.21.15.178:0/4124438128 shutdown_connections 2023-10-16T14:34:03.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/4124438128 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec10d4a0 0x7f7fec10f890 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/4124438128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec074ef0 0x7f7fec073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/4124438128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec073910 0x7f7fec073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.325+0000 7f7ff19ff700 1 -- 172.21.15.178:0/4124438128 >> 172.21.15.178:0/4124438128 conn(0x7f7fec0fc400 msgr2=0x7f7fec0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:03.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.326+0000 7f7ff19ff700 1 -- 172.21.15.178:0/4124438128 shutdown_connections 2023-10-16T14:34:03.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.326+0000 7f7ff19ff700 1 -- 172.21.15.178:0/4124438128 wait complete. 2023-10-16T14:34:03.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.326+0000 7f7ff19ff700 1 Processor -- start 2023-10-16T14:34:03.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 -- start start 2023-10-16T14:34:03.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec074ef0 0x7f7fec19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec10d4a0 0x7f7fec1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1a4e80 con 0x7f7fec10d4a0 2023-10-16T14:34:03.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1a4fc0 con 0x7f7fec073910 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7ff19ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fec1a5150 con 0x7f7fec074ef0 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7feaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7feaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48050/0 (socket says 172.21.15.178:48050) 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7feaffd700 1 -- 172.21.15.178:0/626383270 learned_addr learned my addr 172.21.15.178:0/626383270 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7feb7fe700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec10d4a0 0x7f7fec1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.327+0000 7f7fea7fc700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec074ef0 0x7f7fec19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 -- 172.21.15.178:0/626383270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec074ef0 msgr2=0x7f7fec19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec074ef0 0x7f7fec19d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 -- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec10d4a0 msgr2=0x7f7fec1a48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec10d4a0 0x7f7fec1a48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 -- 172.21.15.178:0/626383270 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4009d00 con 0x7f7fec073910 2023-10-16T14:34:03.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7feaffd700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00e9a0 tx=0x7f7fdc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:03.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc009790 con 0x7f7fec073910 2023-10-16T14:34:03.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fec1a5430 con 0x7f7fec073910 2023-10-16T14:34:03.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fdc010430 con 0x7f7fec073910 2023-10-16T14:34:03.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc018680 con 0x7f7fec073910 2023-10-16T14:34:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.328+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fec1a5990 con 0x7f7fec073910 2023-10-16T14:34:03.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.329+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fec102ff0 con 0x7f7fec073910 2023-10-16T14:34:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.331+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7fdc0105a0 con 0x7f7fec073910 2023-10-16T14:34:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.332+0000 7f7ff09fd700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 0x7f7fd40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.333+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7fdc014070 con 0x7f7fec073910 2023-10-16T14:34:03.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.333+0000 7f7fea7fc700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 0x7f7fd40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.333+0000 7f7fea7fc700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 0x7f7fd40a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7fec074c10 tx=0x7f7fe0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:03.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.339+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7fdc096510 con 0x7f7fec073910 2023-10-16T14:34:03.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.565+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fec112f60 con 0x7f7fec073910 2023-10-16T14:34:03.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.566+0000 7f7ff09fd700 1 -- 172.21.15.178:0/626383270 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fdc099b30 con 0x7f7fec073910 2023-10-16T14:34:03.569 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:34:03.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 msgr2=0x7f7fd40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 0x7f7fd40a3940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7fec074c10 tx=0x7f7fe0009260 comp rx=0 tx=0).stop 2023-10-16T14:34:03.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 msgr2=0x7f7fec19d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00e9a0 tx=0x7f7fdc00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 shutdown_connections 2023-10-16T14:34:03.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7fd40a1480 0x7f7fd40a3940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fec10d4a0 0x7f7fec1a48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fec073910 0x7f7fec19d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 --2- 172.21.15.178:0/626383270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fec074ef0 0x7f7fec19d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.568+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 >> 172.21.15.178:0/626383270 conn(0x7f7fec0fc400 msgr2=0x7f7fec0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:03.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.569+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 shutdown_connections 2023-10-16T14:34:03.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.569+0000 7f7ff19ff700 1 -- 172.21.15.178:0/626383270 wait complete. 2023-10-16T14:34:03.592 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:34:03.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 -- 172.21.15.178:0/2947580474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 msgr2=0x7f96e01024d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2947580474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e01024d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009990 tx=0x7f96d4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 -- 172.21.15.178:0/2947580474 shutdown_connections 2023-10-16T14:34:03.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2947580474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0102a10 0x7f96e0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2947580474 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0100e70 0x7f96e0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2947580474 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e01024d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 -- 172.21.15.178:0/2947580474 >> 172.21.15.178:0/2947580474 conn(0x7f96e00fc3f0 msgr2=0x7f96e00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:03.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 -- 172.21.15.178:0/2947580474 shutdown_connections 2023-10-16T14:34:03.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.794+0000 7f96e4bff700 1 -- 172.21.15.178:0/2947580474 wait complete. 2023-10-16T14:34:03.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 Processor -- start 2023-10-16T14:34:03.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 -- start start 2023-10-16T14:34:03.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 0x7f96e01a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0990 con 0x7f96e0102a10 2023-10-16T14:34:03.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0ad0 con 0x7f96e0102050 2023-10-16T14:34:03.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.795+0000 7f96e4bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0c60 con 0x7f96e0100e70 2023-10-16T14:34:03.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54464/0 (socket says 172.21.15.178:54464) 2023-10-16T14:34:03.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 -- 172.21.15.178:0/2617625431 learned_addr learned my addr 172.21.15.178:0/2617625431 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:03.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96ded9d700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 0x7f96e01a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96ddd9b700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 -- 172.21.15.178:0/2617625431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 msgr2=0x7f96e0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 -- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 msgr2=0x7f96e01a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:03.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 0x7f96e01a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:03.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 -- 172.21.15.178:0/2617625431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d40095f0 con 0x7f96e0100e70 2023-10-16T14:34:03.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96ded9d700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 0x7f96e01a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:03.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.796+0000 7f96de59c700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e970 tx=0x7f96d000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:03.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.797+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0009790 con 0x7f96e0100e70 2023-10-16T14:34:03.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.797+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d00044e0 con 0x7f96e0100e70 2023-10-16T14:34:03.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.797+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e01a0f40 con 0x7f96e0100e70 2023-10-16T14:34:03.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.797+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0003e30 con 0x7f96e0100e70 2023-10-16T14:34:03.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.797+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e01a1370 con 0x7f96e0100e70 2023-10-16T14:34:03.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.798+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e010c730 con 0x7f96e0100e70 2023-10-16T14:34:03.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.800+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f96d0003f90 con 0x7f96e0100e70 2023-10-16T14:34:03.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.801+0000 7f96cf7fe700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 0x7f96c80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:03.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.801+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f96d0014070 con 0x7f96e0100e70 2023-10-16T14:34:03.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.801+0000 7f96ddd9b700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 0x7f96c80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:03.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.802+0000 7f96ddd9b700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 0x7f96c80a39c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d400b260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:03.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:03.803+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f96d0096950 con 0x7f96e0100e70 2023-10-16T14:34:04.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.018+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f96e01a15c0 con 0x7f96e0100e70 2023-10-16T14:34:04.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.019+0000 7f96cf7fe700 1 -- 172.21.15.178:0/2617625431 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f96d0099f70 con 0x7f96e0100e70 2023-10-16T14:34:04.022 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:04.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.021+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 msgr2=0x7f96c80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.021+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 0x7f96c80a39c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006010 tx=0x7f96d400b260 comp rx=0 tx=0).stop 2023-10-16T14:34:04.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.021+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 msgr2=0x7f96e0198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.021+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96d000e970 tx=0x7f96d000ed30 comp rx=0 tx=0).stop 2023-10-16T14:34:04.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.021+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 shutdown_connections 2023-10-16T14:34:04.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1500 0x7f96c80a39c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0102a10 0x7f96e01a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e0100e70 0x7f96e0198e10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 --2- 172.21.15.178:0/2617625431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0102050 0x7f96e0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 >> 172.21.15.178:0/2617625431 conn(0x7f96e00fc3f0 msgr2=0x7f96e00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:04.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 shutdown_connections 2023-10-16T14:34:04.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.022+0000 7f96e4bff700 1 -- 172.21.15.178:0/2617625431 wait complete. 2023-10-16T14:34:04.043 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:03 smithi178 ceph-mon[98495]: pgmap v349: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:04.043 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:03 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3945296774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.043 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:03 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/626383270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.044 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:34:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:03 smithi186 ceph-mon[134892]: pgmap v349: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:03 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3945296774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:03 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/626383270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:03 smithi187 ceph-mon[135297]: pgmap v349: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:03 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3945296774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:03 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/626383270' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:04.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.270+0000 7f5ad3fff700 1 -- 172.21.15.178:0/177760919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad4101760 msgr2=0x7f5ad4103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.270+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/177760919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad4101760 0x7f5ad4103b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009990 tx=0x7f5ac4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 -- 172.21.15.178:0/177760919 shutdown_connections 2023-10-16T14:34:04.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/177760919 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad41069c0 0x7f5ad4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/177760919 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad4101760 0x7f5ad4103b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/177760919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 -- 172.21.15.178:0/177760919 >> 172.21.15.178:0/177760919 conn(0x7f5ad40fb150 msgr2=0x7f5ad40fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:04.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 -- 172.21.15.178:0/177760919 shutdown_connections 2023-10-16T14:34:04.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.271+0000 7f5ad3fff700 1 -- 172.21.15.178:0/177760919 wait complete. 2023-10-16T14:34:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.272+0000 7f5ad3fff700 1 Processor -- start 2023-10-16T14:34:04.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.272+0000 7f5ad3fff700 1 -- start start 2023-10-16T14:34:04.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 0x7f5ad40706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4070c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 0x7f5ad4071c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4072240 con 0x7f5ad4104090 2023-10-16T14:34:04.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40723b0 con 0x7f5ad4101760 2023-10-16T14:34:04.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad3fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4072570 con 0x7f5ad41069c0 2023-10-16T14:34:04.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4070c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 0x7f5ad40706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad27fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4070c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43468/0 (socket says 172.21.15.178:43468) 2023-10-16T14:34:04.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad27fc700 1 -- 172.21.15.178:0/2582802527 learned_addr learned my addr 172.21.15.178:0/2582802527 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:04.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 0x7f5ad4071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 -- 172.21.15.178:0/2582802527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 msgr2=0x7f5ad40706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 0x7f5ad40706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 -- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 msgr2=0x7f5ad4070c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4070c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad37fe700 1 -- 172.21.15.178:0/2582802527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40095f0 con 0x7f5ad41069c0 2023-10-16T14:34:04.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.273+0000 7f5ad2ffd700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 0x7f5ad40706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:04.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5ad37fe700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 0x7f5ad4071c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00efc0 tx=0x7f5acc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:04.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc018070 con 0x7f5ad41069c0 2023-10-16T14:34:04.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5acc00f040 con 0x7f5ad41069c0 2023-10-16T14:34:04.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5acc007bb0 con 0x7f5ad41069c0 2023-10-16T14:34:04.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad4072790 con 0x7f5ad41069c0 2023-10-16T14:34:04.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.274+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad4072c50 con 0x7f5ad41069c0 2023-10-16T14:34:04.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.275+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad40fccd0 con 0x7f5ad41069c0 2023-10-16T14:34:04.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.277+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5acc00c940 con 0x7f5ad41069c0 2023-10-16T14:34:04.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.278+0000 7f5abbfff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 0x7f5abc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.279+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5acc0cd2f0 con 0x7f5ad41069c0 2023-10-16T14:34:04.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.279+0000 7f5ad2ffd700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 0x7f5abc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.280+0000 7f5ad2ffd700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 0x7f5abc0a39c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400b2e0 tx=0x7f5ac4005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:04.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.280+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5acc096610 con 0x7f5ad41069c0 2023-10-16T14:34:04.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.547+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ad4072e70 con 0x7f5ad41069c0 2023-10-16T14:34:04.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.548+0000 7f5abbfff700 1 -- 172.21.15.178:0/2582802527 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7f5acc099c30 con 0x7f5ad41069c0 2023-10-16T14:34:04.551 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:04.551 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:34:04.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 msgr2=0x7f5abc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 0x7f5abc0a39c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400b2e0 tx=0x7f5ac4005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 msgr2=0x7f5ad4071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 0x7f5ad4071c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5acc00efc0 tx=0x7f5acc00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 shutdown_connections 2023-10-16T14:34:04.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5abc0a1500 0x7f5abc0a39c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5ad41069c0 0x7f5ad4071c70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5ad4101760 0x7f5ad40706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 --2- 172.21.15.178:0/2582802527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad4104090 0x7f5ad4070c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 >> 172.21.15.178:0/2582802527 conn(0x7f5ad40fb150 msgr2=0x7f5ad41023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:04.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 shutdown_connections 2023-10-16T14:34:04.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.550+0000 7f5ad3fff700 1 -- 172.21.15.178:0/2582802527 wait complete. 2023-10-16T14:34:04.555 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:34:04.572 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:04.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 -- 172.21.15.178:0/355928068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58102090 msgr2=0x7ffb58102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 --2- 172.21.15.178:0/355928068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58102090 0x7ffb58102510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009990 tx=0x7ffb48009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 -- 172.21.15.178:0/355928068 shutdown_connections 2023-10-16T14:34:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 --2- 172.21.15.178:0/355928068 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 0x7ffb58109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 --2- 172.21.15.178:0/355928068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58100eb0 0x7ffb581012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 --2- 172.21.15.178:0/355928068 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58102090 0x7ffb58102510 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 -- 172.21.15.178:0/355928068 >> 172.21.15.178:0/355928068 conn(0x7ffb580fc430 msgr2=0x7ffb580fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 -- 172.21.15.178:0/355928068 shutdown_connections 2023-10-16T14:34:04.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.787+0000 7ffb605ff700 1 -- 172.21.15.178:0/355928068 wait complete. 2023-10-16T14:34:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.788+0000 7ffb605ff700 1 Processor -- start 2023-10-16T14:34:04.795 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 -- start start 2023-10-16T14:34:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58102090 0x7ffb580732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 0x7ffb580737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.796 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb58073f80 con 0x7ffb58102a50 2023-10-16T14:34:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb580740f0 con 0x7ffb58100eb0 2023-10-16T14:34:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb605ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb581a0bb0 con 0x7ffb58102090 2023-10-16T14:34:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.797 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48108/0 (socket says 172.21.15.178:48108) 2023-10-16T14:34:04.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 -- 172.21.15.178:0/4173296198 learned_addr learned my addr 172.21.15.178:0/4173296198 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:04.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5eb9c700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 0x7ffb580737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5db9a700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58102090 0x7ffb580732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.798 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 -- 172.21.15.178:0/4173296198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58102090 msgr2=0x7ffb580732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58102090 0x7ffb580732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 -- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 msgr2=0x7ffb580737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:04.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 0x7ffb580737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:04.799 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.789+0000 7ffb5e39b700 1 -- 172.21.15.178:0/4173296198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb480095f0 con 0x7ffb58100eb0 2023-10-16T14:34:04.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb5e39b700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e970 tx=0x7ffb5400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:04.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54009790 con 0x7ffb58100eb0 2023-10-16T14:34:04.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb5400eec0 con 0x7ffb58100eb0 2023-10-16T14:34:04.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb581a0e90 con 0x7ffb58100eb0 2023-10-16T14:34:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54003cf0 con 0x7ffb58100eb0 2023-10-16T14:34:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.790+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb581a12b0 con 0x7ffb58100eb0 2023-10-16T14:34:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.791+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb58107db0 con 0x7ffb58100eb0 2023-10-16T14:34:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.793+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffb5400ccd0 con 0x7ffb58100eb0 2023-10-16T14:34:04.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.794+0000 7ffb4f7fe700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 0x7ffb440a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:04.802 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.794+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffb54014070 con 0x7ffb58100eb0 2023-10-16T14:34:04.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.794+0000 7ffb5db9a700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 0x7ffb440a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:04.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.795+0000 7ffb5db9a700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 0x7ffb440a39e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800b2e0 tx=0x7ffb48005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:04.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:04.796+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffb54096940 con 0x7ffb58100eb0 2023-10-16T14:34:05.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.034+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb5810c770 con 0x7ffb58100eb0 2023-10-16T14:34:05.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.036+0000 7ffb4f7fe700 1 -- 172.21.15.178:0/4173296198 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7ffb54099f60 con 0x7ffb58100eb0 2023-10-16T14:34:05.039 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:05.039 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:05.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 msgr2=0x7ffb440a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 0x7ffb440a39e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800b2e0 tx=0x7ffb48005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 msgr2=0x7ffb58074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400e970 tx=0x7ffb5400ec80 comp rx=0 tx=0).stop 2023-10-16T14:34:05.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 shutdown_connections 2023-10-16T14:34:05.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffb440a1520 0x7ffb440a39e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffb58102a50 0x7ffb580737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffb58100eb0 0x7ffb58074c60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 --2- 172.21.15.178:0/4173296198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffb58102090 0x7ffb580732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.038+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 >> 172.21.15.178:0/4173296198 conn(0x7ffb580fc430 msgr2=0x7ffb580fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.039+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 shutdown_connections 2023-10-16T14:34:05.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.039+0000 7ffb605ff700 1 -- 172.21.15.178:0/4173296198 wait complete. 2023-10-16T14:34:05.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2617625431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2582802527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:05.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-10-16T14:31:36.554542+0000', 'modified': '2023-10-16T14:32:15.910283+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24593, 'mds_1': 24550, 'mds_2': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24515': {'gid': 24515, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.186:6835/1810043383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6834', 'nonce': 1810043383}, {'type': 'v1', 'addr': '172.21.15.186:6835', 'nonce': 1810043383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24520': {'gid': 24520, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/2797759968', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 2797759968}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 2797759968}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6837/4191845467', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 4191845467}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 4191845467}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 2, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.186:6837/1524095342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6836', 'nonce': 1524095342}, {'type': 'v1', 'addr': '172.21.15.186:6837', 'nonce': 1524095342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24550': {'gid': 24550, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.187:6839/397706812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 397706812}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 397706812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24593': {'gid': 24593, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.186:6839/2475230127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.186:6838', 'nonce': 2475230127}, {'type': 'v1', 'addr': '172.21.15.186:6839', 'nonce': 2475230127}]}, 'join_fscid': -1, 'export_targets': [1], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T14:34:05.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-10-16T14:34:05.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-10-16T14:34:05.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2617625431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2582802527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:05.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2617625431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2582802527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:05.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.277+0000 7f51231ff700 1 -- 172.21.15.178:0/1389927724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 msgr2=0x7f511c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.277+0000 7f51231ff700 1 --2- 172.21.15.178:0/1389927724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c1012c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5110009990 tx=0x7f5110009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 -- 172.21.15.178:0/1389927724 shutdown_connections 2023-10-16T14:34:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 --2- 172.21.15.178:0/1389927724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102a40 0x7f511c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 --2- 172.21.15.178:0/1389927724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c1012c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 --2- 172.21.15.178:0/1389927724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102080 0x7f511c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 -- 172.21.15.178:0/1389927724 >> 172.21.15.178:0/1389927724 conn(0x7f511c0fc420 msgr2=0x7f511c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 -- 172.21.15.178:0/1389927724 shutdown_connections 2023-10-16T14:34:05.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.278+0000 7f51231ff700 1 -- 172.21.15.178:0/1389927724 wait complete. 2023-10-16T14:34:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 Processor -- start 2023-10-16T14:34:05.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 -- start start 2023-10-16T14:34:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 0x7f511c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 0x7f511c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19e750 con 0x7f511c102080 2023-10-16T14:34:05.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19e890 con 0x7f511c100ea0 2023-10-16T14:34:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f51231ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19ea20 con 0x7f511c102a40 2023-10-16T14:34:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48132/0 (socket says 172.21.15.178:48132) 2023-10-16T14:34:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f5120f9b700 1 -- 172.21.15.178:0/353853763 learned_addr learned my addr 172.21.15.178:0/353853763 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.279+0000 7f511bfff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 0x7f511c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f512179c700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 0x7f511c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 -- 172.21.15.178:0/353853763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 msgr2=0x7f511c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 0x7f511c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 -- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 msgr2=0x7f511c197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 0x7f511c197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 -- 172.21.15.178:0/353853763 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100095f0 con 0x7f511c100ea0 2023-10-16T14:34:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f512179c700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 0x7f511c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f511bfff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 0x7f511c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5120f9b700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5110004710 tx=0x7f5110009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5119ffb700 1 -- 172.21.15.178:0/353853763 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511001d070 con 0x7f511c100ea0 2023-10-16T14:34:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5119ffb700 1 -- 172.21.15.178:0/353853763 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511000fbc0 con 0x7f511c100ea0 2023-10-16T14:34:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f5119ffb700 1 -- 172.21.15.178:0/353853763 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110017650 con 0x7f511c100ea0 2023-10-16T14:34:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c19eca0 con 0x7f511c100ea0 2023-10-16T14:34:05.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.280+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c19f070 con 0x7f511c100ea0 2023-10-16T14:34:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.284+0000 7f5119ffb700 1 -- 172.21.15.178:0/353853763 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5110021410 con 0x7f511c100ea0 2023-10-16T14:34:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.285+0000 7f5119ffb700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 0x7f510c0a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.285+0000 7f5119ffb700 1 -- 172.21.15.178:0/353853763 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f51100ce650 con 0x7f511c100ea0 2023-10-16T14:34:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.285+0000 7f511bfff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 0x7f510c0a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.286+0000 7f511bfff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 0x7f510c0a3a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f511c100bc0 tx=0x7f5108009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 Processor -- start 2023-10-16T14:34:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 -- start start 2023-10-16T14:34:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe70ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fe71a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fde4c20 con 0x56431fe70730 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fdc9c10 con 0x56431fe6fdd0 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5136b3db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fdc7530 con 0x56431fe715d0 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.287+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48140/0 (socket says 172.21.15.178:48140) 2023-10-16T14:34:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 -- 172.21.15.178:0/4032052578 learned_addr learned my addr 172.21.15.178:0/4032052578 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f511bfff700 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe70ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f512179c700 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fe71a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 -- 172.21.15.178:0/4032052578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 msgr2=0x56431fe71a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fe71a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 -- 172.21.15.178:0/4032052578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 msgr2=0x56431fe70ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe70ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 -- 172.21.15.178:0/4032052578 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431fd803a0 con 0x56431fe6fdd0 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5120f9b700 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51100980a0 tx=0x7f5110005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8314e9f597c69377 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5106ffd700 1 -- 172.21.15.178:0/4032052578 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100860f0 con 0x56431fe6fdd0 2023-10-16T14:34:05.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5106ffd700 1 -- 172.21.15.178:0/4032052578 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511009b630 con 0x56431fe6fdd0 2023-10-16T14:34:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.288+0000 7f5106ffd700 1 -- 172.21.15.178:0/4032052578 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511009ab20 con 0x56431fe6fdd0 2023-10-16T14:34:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 -- 172.21.15.178:0/4032052578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 msgr2=0x56431fe701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51100980a0 tx=0x7f5110005a90 comp rx=0 tx=0).stop 2023-10-16T14:34:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 -- 172.21.15.178:0/4032052578 shutdown_connections 2023-10-16T14:34:05.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fe71a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe701f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 --2- 172.21.15.178:0/4032052578 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe70ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 -- 172.21.15.178:0/4032052578 >> 172.21.15.178:0/4032052578 conn(0x56431fd9dd80 msgr2=0x56431fe6af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 -- 172.21.15.178:0/4032052578 shutdown_connections 2023-10-16T14:34:05.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.289+0000 7f5136b3db80 1 -- 172.21.15.178:0/4032052578 wait complete. 2023-10-16T14:34:05.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.290+0000 7f5136b3db80 1 -- start start 2023-10-16T14:34:05.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.290+0000 7f5136b3db80 1 Processor -- start 2023-10-16T14:34:05.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.290+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 0x56431fe7dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.290+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.290+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe715d0 0x56431fe85360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5136b3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fdc2e20 con 0x56431fe6fdd0 2023-10-16T14:34:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5136b3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fd5f6c0 con 0x56431fe715d0 2023-10-16T14:34:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5136b3db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fd5f360 con 0x56431fe70730 2023-10-16T14:34:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5136b3db80 10 client.0 _ll_register_callbacks cb 0x56431fbbc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 0x56431fe7dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 0x56431fe7dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43548/0 (socket says 172.21.15.178:43548) 2023-10-16T14:34:05.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54556/0 (socket says 172.21.15.178:54556) 2023-10-16T14:34:05.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f5120f9b700 1 -- 172.21.15.178:0/2089339488 learned_addr learned my addr 172.21.15.178:0/2089339488 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f512179c700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe715d0 0x56431fe85360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 -- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe715d0 msgr2=0x56431fe85360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe715d0 0x56431fe85360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 -- 172.21.15.178:0/2089339488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 msgr2=0x56431fe7dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 0x56431fe7dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 -- 172.21.15.178:0/2089339488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c19f070 con 0x56431fe70730 2023-10-16T14:34:05.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f511bfff700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5108009810 tx=0x7f51080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18c95b0433f59d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.291+0000 7f50fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:05.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108029670 con 0x56431fe70730 2023-10-16T14:34:05.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56431fdc7ea0 con 0x56431fe70730 2023-10-16T14:34:05.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5108018070 con 0x56431fe70730 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51080297d0 con 0x56431fe70730 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f5136b3db80 10 client.25360 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56431fbedd90 con 0x56431fe70730 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56431fdc7ea0 con 0x56431fe70730 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5108020aa0 con 0x56431fe70730 2023-10-16T14:34:05.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f5136b3db80 10 client.25360 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.292+0000 7f5136b3db80 10 client.25360 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-16T14:34:05.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5136b3db80 20 client.25360 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fe899a0 0x56431fe884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5136b3db80 4 client.25360 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:05.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56431fe3fbc0 con 0x56431fe899a0 2023-10-16T14:34:05.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5120f9b700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fe899a0 0x56431fe884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f5120f9b700 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fe899a0 0x56431fe884c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=95b380e3a0da115c server_cookie=5e29710b6cd4f702 in_seq=0 out_seq=0 2023-10-16T14:34:05.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.293+0000 7f50fbfff700 10 client.25360 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:05.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f50fbfff700 1 -- 172.21.15.178:0/2089339488 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56431fe3fbc0 con 0x56431fe899a0 2023-10-16T14:34:05.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f50fbfff700 10 client.25360 handle_command_reply: tid=0 2023-10-16T14:34:05.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 client.25360 shutdown 2023-10-16T14:34:05.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 msgr2=0x56431fe7e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5108009810 tx=0x7f51080047c0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 shutdown_connections 2023-10-16T14:34:05.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fe899a0 0x56431fe884c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe715d0 0x56431fe85360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe6fdd0 0x56431fe7dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2089339488 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe7e320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f50fbfff700 0 client.25360 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:05.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 >> 172.21.15.178:0/2089339488 conn(0x56431fd9dd80 msgr2=0x56431fe6aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 shutdown_connections 2023-10-16T14:34:05.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.294+0000 7f5136b3db80 1 -- 172.21.15.178:0/2089339488 wait complete. 2023-10-16T14:34:05.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.295+0000 7f5136b3db80 20 client.25360 trim_cache size 0 max 16384 2023-10-16T14:34:05.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.313+0000 7f5136b3db80 1 Processor -- start 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.313+0000 7f5136b3db80 1 -- start start 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe78d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe715d0 0x56431fe85320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fc438d0 con 0x56431fe715d0 2023-10-16T14:34:05.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fe3fbc0 con 0x56431fe6fdd0 2023-10-16T14:34:05.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5136b3db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fdc7ea0 con 0x56431fe70730 2023-10-16T14:34:05.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe78d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54572/0 (socket says 172.21.15.178:54572) 2023-10-16T14:34:05.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe78d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:48152/0 (socket says 172.21.15.178:48152) 2023-10-16T14:34:05.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 -- 172.21.15.178:0/916054794 learned_addr learned my addr 172.21.15.178:0/916054794 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f512179c700 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe715d0 0x56431fe85320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 -- 172.21.15.178:0/916054794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 msgr2=0x56431fe78d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe78d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 -- 172.21.15.178:0/916054794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe715d0 msgr2=0x56431fe85320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe715d0 0x56431fe85320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.314+0000 7f511bfff700 1 -- 172.21.15.178:0/916054794 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431fbd6c10 con 0x56431fe70730 2023-10-16T14:34:05.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f511bfff700 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f510801f4d0 tx=0x7f5108004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=946beccf2162c49f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f51077fe700 1 -- 172.21.15.178:0/916054794 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108012be0 con 0x56431fe70730 2023-10-16T14:34:05.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f51077fe700 1 -- 172.21.15.178:0/916054794 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5108016070 con 0x56431fe70730 2023-10-16T14:34:05.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f51077fe700 1 -- 172.21.15.178:0/916054794 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108005470 con 0x56431fe70730 2023-10-16T14:34:05.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f5136b3db80 1 -- 172.21.15.178:0/916054794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 msgr2=0x56431fe79260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.315+0000 7f5136b3db80 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f510801f4d0 tx=0x7f5108004740 comp rx=0 tx=0).stop 2023-10-16T14:34:05.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 -- 172.21.15.178:0/916054794 shutdown_connections 2023-10-16T14:34:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe715d0 0x56431fe85320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe78d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 --2- 172.21.15.178:0/916054794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe70730 0x56431fe79260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 -- 172.21.15.178:0/916054794 >> 172.21.15.178:0/916054794 conn(0x56431fd9dd80 msgr2=0x56431fe69750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 -- 172.21.15.178:0/916054794 shutdown_connections 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 -- 172.21.15.178:0/916054794 wait complete. 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 -- start start 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.316+0000 7f5136b3db80 1 Processor -- start 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe7aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fe7b630 con 0x56431fe70730 2023-10-16T14:34:05.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fedd900 con 0x56431fe6fdd0 2023-10-16T14:34:05.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431fedda40 con 0x56431fe715d0 2023-10-16T14:34:05.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5136b3db80 10 client.0 _ll_register_callbacks cb 0x56431fbbc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f511bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f5120f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe7aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f511bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43564/0 (socket says 172.21.15.178:43564) 2023-10-16T14:34:05.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f511bfff700 1 -- 172.21.15.178:0/2555935644 learned_addr learned my addr 172.21.15.178:0/2555935644 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54584/0 (socket says 172.21.15.178:54584) 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 -- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 msgr2=0x56431fe7aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe7aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 -- 172.21.15.178:0/2555935644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 msgr2=0x56431fe7b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f512179c700 1 -- 172.21.15.178:0/2555935644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510802e040 con 0x56431fe715d0 2023-10-16T14:34:05.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.317+0000 7f511bfff700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f512179c700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56431fe797f0 tx=0x7f5114007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2ce91894464e8e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f50fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:05.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400e040 con 0x56431fe715d0 2023-10-16T14:34:05.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f511009b630 con 0x56431fe715d0 2023-10-16T14:34:05.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5114004db0 con 0x56431fe715d0 2023-10-16T14:34:05.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.318+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5114003a00 con 0x56431fe715d0 2023-10-16T14:34:05.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.319+0000 7f5136b3db80 10 client.25366 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.319+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56431fc14200 con 0x56431fe715d0 2023-10-16T14:34:05.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.319+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f511009b630 con 0x56431fe715d0 2023-10-16T14:34:05.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.319+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5114005540 con 0x56431fe715d0 2023-10-16T14:34:05.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 10 client.25366 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 10 client.25366 resolve_mds: resolved name 'h' to daemon mds.h 2023-10-16T14:34:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 20 client.25366 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fee2d80 0x56431fedf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 4 client.25366 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:34:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56431fdbbe00 con 0x56431fee2d80 2023-10-16T14:34:05.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5120f9b700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fee2d80 0x56431fedf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f50fbfff700 10 client.25366 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:05.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.320+0000 7f5120f9b700 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fee2d80 0x56431fedf1b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd9b2a1835678443 server_cookie=d0e2999c561ee37e in_seq=0 out_seq=0 2023-10-16T14:34:05.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f50fbfff700 1 -- 172.21.15.178:0/2555935644 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56431fdbbe00 con 0x56431fee2d80 2023-10-16T14:34:05.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f50fbfff700 10 client.25366 handle_command_reply: tid=0 2023-10-16T14:34:05.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 client.25366 shutdown 2023-10-16T14:34:05.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 msgr2=0x56431fedd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56431fe797f0 tx=0x7f5114007ed0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 shutdown_connections 2023-10-16T14:34:05.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x56431fee2d80 0x56431fedf1b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56431fe715d0 0x56431fedd3c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56431fe6fdd0 0x56431fe7aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 --2- 172.21.15.178:0/2555935644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56431fe70730 0x56431fe7b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f50fbfff700 0 client.25366 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 >> 172.21.15.178:0/2555935644 conn(0x56431fd9dd80 msgr2=0x56431fe884c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 shutdown_connections 2023-10-16T14:34:05.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 1 -- 172.21.15.178:0/2555935644 wait complete. 2023-10-16T14:34:05.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.321+0000 7f5136b3db80 20 client.25366 trim_cache size 0 max 16384 2023-10-16T14:34:05.337 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:34:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 2, 2023-10-16T14:34:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24548, 2023-10-16T14:34:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:34:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:34:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:34:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 116.628758809, 2023-10-16T14:34:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:34:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:34:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:34:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 165.06666305799999 2023-10-16T14:34:05.339 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:05.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 msgr2=0x7f510c0a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 0x7f510c0a3a10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f511c100bc0 tx=0x7f5108009260 comp rx=0 tx=0).stop 2023-10-16T14:34:05.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 msgr2=0x7f511c196bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5110004710 tx=0x7f5110009580 comp rx=0 tx=0).stop 2023-10-16T14:34:05.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 shutdown_connections 2023-10-16T14:34:05.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f510c0a1550 0x7f510c0a3a10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f511c102a40 0x7f511c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f511c100ea0 0x7f511c196bd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 --2- 172.21.15.178:0/353853763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f511c102080 0x7f511c197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 >> 172.21.15.178:0/353853763 conn(0x7f511c0fc420 msgr2=0x7f511c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 shutdown_connections 2023-10-16T14:34:05.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.322+0000 7f51231ff700 1 -- 172.21.15.178:0/353853763 wait complete. 2023-10-16T14:34:05.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-10-16T14:34:05.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 -- 172.21.15.178:0/2710859839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840102080 msgr2=0x7f1840102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 --2- 172.21.15.178:0/2710859839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840102080 0x7f1840102500 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1830009990 tx=0x7f1830009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 -- 172.21.15.178:0/2710859839 shutdown_connections 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 --2- 172.21.15.178:0/2710859839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102a40 0x7f1840109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 --2- 172.21.15.178:0/2710859839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840100ea0 0x7f18401012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 --2- 172.21.15.178:0/2710859839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840102080 0x7f1840102500 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 -- 172.21.15.178:0/2710859839 >> 172.21.15.178:0/2710859839 conn(0x7f18400fc420 msgr2=0x7f18400fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 -- 172.21.15.178:0/2710859839 shutdown_connections 2023-10-16T14:34:05.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.572+0000 7f18481ff700 1 -- 172.21.15.178:0/2710859839 wait complete. 2023-10-16T14:34:05.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 Processor -- start 2023-10-16T14:34:05.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 -- start start 2023-10-16T14:34:05.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102080 0x7f1840197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184019e740 con 0x7f1840100ea0 2023-10-16T14:34:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184019e880 con 0x7f1840102080 2023-10-16T14:34:05.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.573+0000 7f18481ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184019ea10 con 0x7f1840102a40 2023-10-16T14:34:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54610/0 (socket says 172.21.15.178:54610) 2023-10-16T14:34:05.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 -- 172.21.15.178:0/1158651117 learned_addr learned my addr 172.21.15.178:0/1158651117 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43578/0 (socket says 172.21.15.178:43578) 2023-10-16T14:34:05.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184579a700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102080 0x7f1840197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 -- 172.21.15.178:0/1158651117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102080 msgr2=0x7f1840197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102080 0x7f1840197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 -- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 msgr2=0x7f1840196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 -- 172.21.15.178:0/1158651117 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18300095f0 con 0x7f1840102a40 2023-10-16T14:34:05.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f1845f9b700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f184679c700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f183800ef30 tx=0x7f183800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.574+0000 7f1836ffd700 1 -- 172.21.15.178:0/1158651117 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838018070 con 0x7f1840102a40 2023-10-16T14:34:05.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.575+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184019ec90 con 0x7f1840102a40 2023-10-16T14:34:05.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.575+0000 7f1836ffd700 1 -- 172.21.15.178:0/1158651117 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183800f040 con 0x7f1840102a40 2023-10-16T14:34:05.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.575+0000 7f1836ffd700 1 -- 172.21.15.178:0/1158651117 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800cd00 con 0x7f1840102a40 2023-10-16T14:34:05.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.575+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184019f150 con 0x7f1840102a40 2023-10-16T14:34:05.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.578+0000 7f1836ffd700 1 -- 172.21.15.178:0/1158651117 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1838003a00 con 0x7f1840102a40 2023-10-16T14:34:05.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 Processor -- start 2023-10-16T14:34:05.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f1836ffd700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 0x7f182c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 -- start start 2023-10-16T14:34:05.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f1836ffd700 1 -- 172.21.15.178:0/1158651117 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f18380ccb70 con 0x7f1840102a40 2023-10-16T14:34:05.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f184579a700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 0x7f182c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb17400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb17e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada86c20 con 0x559eadb16b40 2023-10-16T14:34:05.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada6bc10 con 0x559eadb179e0 2023-10-16T14:34:05.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.579+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada69530 con 0x559eadb161e0 2023-10-16T14:34:05.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f184579a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb17400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54612/0 (socket says 172.21.15.178:54612) 2023-10-16T14:34:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 -- 172.21.15.178:0/3188002508 learned_addr learned my addr 172.21.15.178:0/3188002508 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f184679c700 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb17e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f184579a700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 0x7f182c0a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f183000b2e0 tx=0x7f183000ba30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 -- 172.21.15.178:0/3188002508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 msgr2=0x559eadb17e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb17e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 -- 172.21.15.178:0/3188002508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 msgr2=0x559eadb17400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb17400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 -- 172.21.15.178:0/3188002508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eada223a0 con 0x559eadb161e0 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f184679c700 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb17e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.580+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c010580 tx=0x7f183c010890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1bdca929f88dd11 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f182bfff700 1 -- 172.21.15.178:0/3188002508 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c011640 con 0x559eadb161e0 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f182bfff700 1 -- 172.21.15.178:0/3188002508 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183c015070 con 0x559eadb161e0 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f182bfff700 1 -- 172.21.15.178:0/3188002508 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c019a30 con 0x559eadb161e0 2023-10-16T14:34:05.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3188002508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 msgr2=0x559eadb16600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f183c010580 tx=0x7f183c010890 comp rx=0 tx=0).stop 2023-10-16T14:34:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3188002508 shutdown_connections 2023-10-16T14:34:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb17e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb16600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3188002508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb17400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3188002508 >> 172.21.15.178:0/3188002508 conn(0x559eada3fd80 msgr2=0x559eadb0cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3188002508 shutdown_connections 2023-10-16T14:34:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.581+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3188002508 wait complete. 2023-10-16T14:34:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.582+0000 7f185bb1cb80 1 -- start start 2023-10-16T14:34:05.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.582+0000 7f185bb1cb80 1 Processor -- start 2023-10-16T14:34:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.582+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 0x559eadb20040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 0x559eadb275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada64e20 con 0x559eadb16b40 2023-10-16T14:34:05.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada016c0 con 0x559eadb161e0 2023-10-16T14:34:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada01360 con 0x559eadb179e0 2023-10-16T14:34:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f185bb1cb80 10 client.0 _ll_register_callbacks cb 0x559ead85e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184579a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 0x559eadb20040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 0x559eadb275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184579a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43594/0 (socket says 172.21.15.178:43594) 2023-10-16T14:34:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184579a700 1 -- 172.21.15.178:0/4185656142 learned_addr learned my addr 172.21.15.178:0/4185656142 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 -- 172.21.15.178:0/4185656142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 msgr2=0x559eadb20040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 0x559eadb20040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 -- 172.21.15.178:0/4185656142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 msgr2=0x559eadb20580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 -- 172.21.15.178:0/4185656142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184019f150 con 0x559eadb179e0 2023-10-16T14:34:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f1845f9b700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 0x559eadb20040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184579a700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f184679c700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 0x559eadb275c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1838099850 tx=0x7f1838099580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5326d05965d76243 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.583+0000 7f18297fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838099240 con 0x559eadb179e0 2023-10-16T14:34:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eada69ea0 con 0x559eadb179e0 2023-10-16T14:34:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f183809a250 con 0x559eadb179e0 2023-10-16T14:34:05.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f183800ce60 con 0x559eadb179e0 2023-10-16T14:34:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f185bb1cb80 10 client.25378 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f183c007d40 con 0x559eadb179e0 2023-10-16T14:34:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f18380b6070 con 0x559eadb179e0 2023-10-16T14:34:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f18380a19c0 con 0x559eadb179e0 2023-10-16T14:34:05.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f185bb1cb80 10 client.25378 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.584+0000 7f185bb1cb80 10 client.25378 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-16T14:34:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.585+0000 7f185bb1cb80 20 client.25378 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.585+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb2bbf0 0x559eadb2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.585+0000 7f185bb1cb80 4 client.25378 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:05.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.585+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eadae1bc0 con 0x559eadb2bbf0 2023-10-16T14:34:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.585+0000 7f1845f9b700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb2bbf0 0x559eadb2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.586+0000 7f18297fa700 10 client.25378 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.586+0000 7f1845f9b700 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb2bbf0 0x559eadb2a710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87a069b64a7c5cc server_cookie=a1bafb3c6e9f066d in_seq=0 out_seq=0 2023-10-16T14:34:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.586+0000 7f18297fa700 1 -- 172.21.15.178:0/4185656142 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559eadae1bc0 con 0x559eadb2bbf0 2023-10-16T14:34:05.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.586+0000 7f18297fa700 10 client.25378 handle_command_reply: tid=0 2023-10-16T14:34:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.586+0000 7f185bb1cb80 1 client.25378 shutdown 2023-10-16T14:34:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 msgr2=0x559eadb275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 0x559eadb275c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1838099850 tx=0x7f1838099580 comp rx=0 tx=0).stop 2023-10-16T14:34:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 shutdown_connections 2023-10-16T14:34:05.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb2bbf0 0x559eadb2a710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb179e0 0x559eadb275c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f18297fa700 0 client.25378 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb161e0 0x559eadb20040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4185656142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 >> 172.21.15.178:0/4185656142 conn(0x559eada3fd80 msgr2=0x559eadb0cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 shutdown_connections 2023-10-16T14:34:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4185656142 wait complete. 2023-10-16T14:34:05.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.587+0000 7f185bb1cb80 20 client.25378 trim_cache size 0 max 16384 2023-10-16T14:34:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 Processor -- start 2023-10-16T14:34:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 -- start start 2023-10-16T14:34:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb16b40 0x559eadb27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 0x559eadb1a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ead8e58d0 con 0x559eadb179e0 2023-10-16T14:34:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada60f10 con 0x559eadb16b40 2023-10-16T14:34:05.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.605+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eada23590 con 0x559eadb161e0 2023-10-16T14:34:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54622/0 (socket says 172.21.15.178:54622) 2023-10-16T14:34:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 -- 172.21.15.178:0/3575695132 learned_addr learned my addr 172.21.15.178:0/3575695132 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f184679c700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 0x559eadb1a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f184579a700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb16b40 0x559eadb27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 -- 172.21.15.178:0/3575695132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb16b40 msgr2=0x559eadb27580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb16b40 0x559eadb27580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 -- 172.21.15.178:0/3575695132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 msgr2=0x559eadb1a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 0x559eadb1a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 -- 172.21.15.178:0/3575695132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ead88fd90 con 0x559eadb161e0 2023-10-16T14:34:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f184679c700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 0x559eadb1a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1845f9b700 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0193d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2792787363fc636 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1834ff9700 1 -- 172.21.15.178:0/3575695132 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c005040 con 0x559eadb161e0 2023-10-16T14:34:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1834ff9700 1 -- 172.21.15.178:0/3575695132 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183c005660 con 0x559eadb161e0 2023-10-16T14:34:05.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.606+0000 7f1834ff9700 1 -- 172.21.15.178:0/3575695132 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c019540 con 0x559eadb161e0 2023-10-16T14:34:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3575695132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 msgr2=0x559eadb1ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f183c000c00 tx=0x7f183c0193d0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3575695132 shutdown_connections 2023-10-16T14:34:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb179e0 0x559eadb1a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb1ddd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/3575695132 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb16b40 0x559eadb27580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3575695132 >> 172.21.15.178:0/3575695132 conn(0x559eada3fd80 msgr2=0x559eadb0b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3575695132 shutdown_connections 2023-10-16T14:34:05.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.607+0000 7f185bb1cb80 1 -- 172.21.15.178:0/3575695132 wait complete. 2023-10-16T14:34:05.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 -- start start 2023-10-16T14:34:05.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 Processor -- start 2023-10-16T14:34:05.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb27410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eadb24ad0 con 0x559eadb16b40 2023-10-16T14:34:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eadb15a10 con 0x559eadb179e0 2023-10-16T14:34:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eadb15bd0 con 0x559eadb161e0 2023-10-16T14:34:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f185bb1cb80 10 client.0 _ll_register_callbacks cb 0x7f1838099980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb27410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f1845f9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb27410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54634/0 (socket says 172.21.15.178:54634) 2023-10-16T14:34:05.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f184579a700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f1845f9b700 1 -- 172.21.15.178:0/4240020330 learned_addr learned my addr 172.21.15.178:0/4240020330 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.608+0000 7f184679c700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 -- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 msgr2=0x559eadb27410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb27410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 -- 172.21.15.178:0/4240020330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 msgr2=0x559eadb243f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb243f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 -- 172.21.15.178:0/4240020330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c007d40 con 0x559eadb16b40 2023-10-16T14:34:05.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184679c700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f184579a700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb23eb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f183000b680 tx=0x7f1830009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c447fa05921545 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f18297fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183001d030 con 0x559eadb16b40 2023-10-16T14:34:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f183c015070 con 0x559eadb16b40 2023-10-16T14:34:05.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.609+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1830005700 con 0x559eadb16b40 2023-10-16T14:34:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f183000f670 con 0x559eadb16b40 2023-10-16T14:34:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 10 client.15354 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f183c01e040 con 0x559eadb16b40 2023-10-16T14:34:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f183c015070 con 0x559eadb16b40 2023-10-16T14:34:05.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1830031950 con 0x559eadb16b40 2023-10-16T14:34:05.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 10 client.15354 resolve_mds: resolved name 'i' to daemon mds.i 2023-10-16T14:34:05.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 20 client.15354 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb88de0 0x559eadb85230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 4 client.15354 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:34:05.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559eada5de00 con 0x559eadb88de0 2023-10-16T14:34:05.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.610+0000 7f1845f9b700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb88de0 0x559eadb85230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f1845f9b700 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb88de0 0x559eadb85230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da07e76311b3fbc0 server_cookie=9cfd478c7cd600b0 in_seq=0 out_seq=0 2023-10-16T14:34:05.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f18297fa700 10 client.15354 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:05.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f18297fa700 1 -- 172.21.15.178:0/4240020330 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x559eada5de00 con 0x559eadb88de0 2023-10-16T14:34:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f18297fa700 10 client.15354 handle_command_reply: tid=0 2023-10-16T14:34:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 client.15354 shutdown 2023-10-16T14:34:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 msgr2=0x559eadb23eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb23eb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f183000b680 tx=0x7f1830009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 shutdown_connections 2023-10-16T14:34:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559eadb88de0 0x559eadb85230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x559eadb179e0 0x559eadb243f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559eadb161e0 0x559eadb27410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f18297fa700 0 client.15354 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:05.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 --2- 172.21.15.178:0/4240020330 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x559eadb16b40 0x559eadb23eb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 >> 172.21.15.178:0/4240020330 conn(0x559eada3fd80 msgr2=0x559eadb1a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 shutdown_connections 2023-10-16T14:34:05.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.611+0000 7f185bb1cb80 1 -- 172.21.15.178:0/4240020330 wait complete. 2023-10-16T14:34:05.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f185bb1cb80 20 client.15354 trim_cache size 0 max 16384 2023-10-16T14:34:05.632 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:34:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 1, 2023-10-16T14:34:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24550, 2023-10-16T14:34:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:34:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:34:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:34:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 118.931944284, 2023-10-16T14:34:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:34:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:34:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:34:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 163.17687843799999 2023-10-16T14:34:05.634 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:05.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 msgr2=0x7f182c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 0x7f182c0a3990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f183000b2e0 tx=0x7f183000ba30 comp rx=0 tx=0).stop 2023-10-16T14:34:05.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 msgr2=0x7f184019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f183800ef30 tx=0x7f183800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 shutdown_connections 2023-10-16T14:34:05.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f182c0a14d0 0x7f182c0a3990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1840102a40 0x7f184019e170 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1840100ea0 0x7f1840196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 --2- 172.21.15.178:0/1158651117 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1840102080 0x7f1840197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 >> 172.21.15.178:0/1158651117 conn(0x7f18400fc420 msgr2=0x7f18400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 shutdown_connections 2023-10-16T14:34:05.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.612+0000 7f18481ff700 1 -- 172.21.15.178:0/1158651117 wait complete. 2023-10-16T14:34:05.638 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-16T14:34:05.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.853+0000 7ff8a5fff700 1 -- 172.21.15.178:0/1316740896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 msgr2=0x7ff8a01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.853+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/1316740896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a01012c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff890009990 tx=0x7ff890009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 -- 172.21.15.178:0/1316740896 shutdown_connections 2023-10-16T14:34:05.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/1316740896 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 0x7ff8a0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/1316740896 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a01012c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/1316740896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 -- 172.21.15.178:0/1316740896 >> 172.21.15.178:0/1316740896 conn(0x7ff8a00fc420 msgr2=0x7ff8a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 -- 172.21.15.178:0/1316740896 shutdown_connections 2023-10-16T14:34:05.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.854+0000 7ff8a5fff700 1 -- 172.21.15.178:0/1316740896 wait complete. 2023-10-16T14:34:05.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 Processor -- start 2023-10-16T14:34:05.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 -- start start 2023-10-16T14:34:05.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 0x7ff8a01a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a09a0 con 0x7ff8a0100ea0 2023-10-16T14:34:05.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a0ae0 con 0x7ff8a0102a40 2023-10-16T14:34:05.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff8a5fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a01a0c70 con 0x7ff8a0102080 2023-10-16T14:34:05.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff89effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43628/0 (socket says 172.21.15.178:43628) 2023-10-16T14:34:05.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff89f7fe700 1 -- 172.21.15.178:0/839627954 learned_addr learned my addr 172.21.15.178:0/839627954 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.855+0000 7ff89ffff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 0x7ff8a01a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 -- 172.21.15.178:0/839627954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 msgr2=0x7ff8a0199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 -- 172.21.15.178:0/839627954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 msgr2=0x7ff8a01a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 0x7ff8a01a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 -- 172.21.15.178:0/839627954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898009c40 con 0x7ff8a0100ea0 2023-10-16T14:34:05.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89effd700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff890004710 tx=0x7ff890009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89cff9700 1 -- 172.21.15.178:0/839627954 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89001d070 con 0x7ff8a0100ea0 2023-10-16T14:34:05.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89cff9700 1 -- 172.21.15.178:0/839627954 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff890021410 con 0x7ff8a0100ea0 2023-10-16T14:34:05.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8900095f0 con 0x7ff8a0100ea0 2023-10-16T14:34:05.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff89cff9700 1 -- 172.21.15.178:0/839627954 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890017650 con 0x7ff8a0100ea0 2023-10-16T14:34:05.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.856+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a01a1190 con 0x7ff8a0100ea0 2023-10-16T14:34:05.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.861+0000 7ff89cff9700 1 -- 172.21.15.178:0/839627954 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff890021580 con 0x7ff8a0100ea0 2023-10-16T14:34:05.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 Processor -- start 2023-10-16T14:34:05.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 -- start start 2023-10-16T14:34:05.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff89cff9700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 0x7ff8880a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff89cff9700 1 -- 172.21.15.178:0/839627954 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff8900ce320 con 0x7ff8a0100ea0 2023-10-16T14:34:05.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a58ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a59a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29ccc20 con 0x55a4d2a595d0 2023-10-16T14:34:05.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29b1c10 con 0x55a4d2a58730 2023-10-16T14:34:05.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff89ffff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 0x7ff8880a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.862+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29af530 con 0x55a4d2a57dd0 2023-10-16T14:34:05.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54658/0 (socket says 172.21.15.178:54658) 2023-10-16T14:34:05.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 -- 172.21.15.178:0/3501938581 learned_addr learned my addr 172.21.15.178:0/3501938581 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89ffff700 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a59a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89effd700 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a58ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 -- 172.21.15.178:0/3501938581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 msgr2=0x55a4d2a58ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a58ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 -- 172.21.15.178:0/3501938581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 msgr2=0x55a4d2a59a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a59a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 -- 172.21.15.178:0/3501938581 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4d29683a0 con 0x55a4d2a57dd0 2023-10-16T14:34:05.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff890097e70 tx=0x7ff890097870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad80495acca207d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff89ffff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 0x7ff8880a3a10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff898009fd0 tx=0x7ff8980115a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff885ffb700 1 -- 172.21.15.178:0/3501938581 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89009bb00 con 0x55a4d2a57dd0 2023-10-16T14:34:05.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff885ffb700 1 -- 172.21.15.178:0/3501938581 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff89009b400 con 0x55a4d2a57dd0 2023-10-16T14:34:05.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.863+0000 7ff885ffb700 1 -- 172.21.15.178:0/3501938581 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89009b660 con 0x55a4d2a57dd0 2023-10-16T14:34:05.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/3501938581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 msgr2=0x55a4d2a581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff890097e70 tx=0x7ff890097870 comp rx=0 tx=0).stop 2023-10-16T14:34:05.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/3501938581 shutdown_connections 2023-10-16T14:34:05.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a59a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a581f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/3501938581 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a58ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/3501938581 >> 172.21.15.178:0/3501938581 conn(0x55a4d2985d80 msgr2=0x55a4d2a52f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.864+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/3501938581 shutdown_connections 2023-10-16T14:34:05.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.865+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/3501938581 wait complete. 2023-10-16T14:34:05.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.865+0000 7ff8b99a2b80 1 -- start start 2023-10-16T14:34:05.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.865+0000 7ff8b99a2b80 1 Processor -- start 2023-10-16T14:34:05.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a61870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a68df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29aae20 con 0x55a4d2a595d0 2023-10-16T14:34:05.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29476c0 con 0x55a4d2a57dd0 2023-10-16T14:34:05.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d2947360 con 0x55a4d2a58730 2023-10-16T14:34:05.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff8b99a2b80 10 client.0 _ll_register_callbacks cb 0x55a4d27a47b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a68df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54666/0 (socket says 172.21.15.178:54666) 2023-10-16T14:34:05.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89effd700 1 -- 172.21.15.178:0/1008549873 learned_addr learned my addr 172.21.15.178:0/1008549873 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a61870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 -- 172.21.15.178:0/1008549873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 msgr2=0x55a4d2a61db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 -- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a57dd0 msgr2=0x55a4d2a61870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a61870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89ffff700 1 -- 172.21.15.178:0/1008549873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a01a1190 con 0x55a4d2a595d0 2023-10-16T14:34:05.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.866+0000 7ff89effd700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff89ffff700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a68df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff898011fd0 tx=0x7ff898004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3934f61ea341bff1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff87effd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:05.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800e040 con 0x55a4d2a595d0 2023-10-16T14:34:05.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff898004240 con 0x55a4d2a595d0 2023-10-16T14:34:05.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4d29afea0 con 0x55a4d2a595d0 2023-10-16T14:34:05.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff898004400 con 0x55a4d2a595d0 2023-10-16T14:34:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff8b99a2b80 10 client.15366 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.867+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a4d27d5d90 con 0x55a4d2a595d0 2023-10-16T14:34:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4d29afea0 con 0x55a4d2a595d0 2023-10-16T14:34:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff898007ad0 con 0x55a4d2a595d0 2023-10-16T14:34:05.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 10 client.15366 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:34:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 20 client.15366 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2a6d4e0 0x55a4d2a6bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 4 client.15366 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4d2a27bc0 con 0x55a4d2a6d4e0 2023-10-16T14:34:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.868+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2a6d4e0 0x55a4d2a6bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.869+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2a6d4e0 0x55a4d2a6bef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28f26d25e42bb4a server_cookie=ee7ad1adc8d2bbf in_seq=0 out_seq=0 2023-10-16T14:34:05.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.869+0000 7ff87effd700 10 client.15366 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:05.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.869+0000 7ff87effd700 1 -- 172.21.15.178:0/1008549873 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a4d2a27bc0 con 0x55a4d2a6d4e0 2023-10-16T14:34:05.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.869+0000 7ff87effd700 10 client.15366 handle_command_reply: tid=0 2023-10-16T14:34:05.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.869+0000 7ff8b99a2b80 1 client.15366 shutdown 2023-10-16T14:34:05.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 msgr2=0x55a4d2a68df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a68df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff898011fd0 tx=0x7ff898004b40 comp rx=0 tx=0).stop 2023-10-16T14:34:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 shutdown_connections 2023-10-16T14:34:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2a6d4e0 0x55a4d2a6bef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a595d0 0x55a4d2a68df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a61870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff87effd700 0 client.15366 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/1008549873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a61db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 >> 172.21.15.178:0/1008549873 conn(0x55a4d2985d80 msgr2=0x55a4d2a52ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 shutdown_connections 2023-10-16T14:34:05.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/1008549873 wait complete. 2023-10-16T14:34:05.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.870+0000 7ff8b99a2b80 20 client.15366 trim_cache size 0 max 16384 2023-10-16T14:34:05.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.895+0000 7ff8b99a2b80 1 Processor -- start 2023-10-16T14:34:05.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.895+0000 7ff8b99a2b80 1 -- start start 2023-10-16T14:34:05.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.895+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.895+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a595d0 0x55a4d2a5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d282b8d0 con 0x55a4d2a57dd0 2023-10-16T14:34:05.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d29a6f10 con 0x55a4d2a595d0 2023-10-16T14:34:05.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d2969590 con 0x55a4d2a58730 2023-10-16T14:34:05.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54668/0 (socket says 172.21.15.178:54668) 2023-10-16T14:34:05.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 -- 172.21.15.178:0/4256998756 learned_addr learned my addr 172.21.15.178:0/4256998756 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89ffff700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a595d0 0x55a4d2a5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 -- 172.21.15.178:0/4256998756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a595d0 msgr2=0x55a4d2a5f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a595d0 0x55a4d2a5f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 -- 172.21.15.178:0/4256998756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 msgr2=0x55a4d2a521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 -- 172.21.15.178:0/4256998756 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4d27bec10 con 0x55a4d2a58730 2023-10-16T14:34:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.896+0000 7ff89effd700 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff894004390 tx=0x7ff894011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=254f41dd579dca73 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.897+0000 7ff8867fc700 1 -- 172.21.15.178:0/4256998756 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894016050 con 0x55a4d2a58730 2023-10-16T14:34:05.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.897+0000 7ff8867fc700 1 -- 172.21.15.178:0/4256998756 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff894017040 con 0x55a4d2a58730 2023-10-16T14:34:05.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.897+0000 7ff8867fc700 1 -- 172.21.15.178:0/4256998756 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89401ca50 con 0x55a4d2a58730 2023-10-16T14:34:05.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.897+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/4256998756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 msgr2=0x55a4d2a68d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff894004390 tx=0x7ff894011750 comp rx=0 tx=0).stop 2023-10-16T14:34:05.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/4256998756 shutdown_connections 2023-10-16T14:34:05.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a595d0 0x55a4d2a5f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/4256998756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a58730 0x55a4d2a68d10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/4256998756 >> 172.21.15.178:0/4256998756 conn(0x55a4d2985d80 msgr2=0x55a4d2a60560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/4256998756 shutdown_connections 2023-10-16T14:34:05.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/4256998756 wait complete. 2023-10-16T14:34:05.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 -- start start 2023-10-16T14:34:05.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.898+0000 7ff8b99a2b80 1 Processor -- start 2023-10-16T14:34:05.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a68e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a595d0 0x55a4d2a65490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d2a65b70 con 0x55a4d2a57dd0 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d2a65cb0 con 0x55a4d2a58730 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4d2a65e70 con 0x55a4d2a595d0 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff8b99a2b80 10 client.0 _ll_register_callbacks cb 0x7ff898011950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff89effd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a68e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff89f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43652/0 (socket says 172.21.15.178:43652) 2023-10-16T14:34:05.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff89f7fe700 1 -- 172.21.15.178:0/2887119268 learned_addr learned my addr 172.21.15.178:0/2887119268 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:05.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.899+0000 7ff89ffff700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a595d0 0x55a4d2a65490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 -- 172.21.15.178:0/2887119268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a595d0 msgr2=0x55a4d2a65490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a595d0 0x55a4d2a65490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 -- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 msgr2=0x55a4d2a68e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:05.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a68e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 -- 172.21.15.178:0/2887119268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff894004040 con 0x55a4d2a57dd0 2023-10-16T14:34:05.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89effd700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:05.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff89f7fe700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff890005cd0 tx=0x7ff89000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6516968947914e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:05.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff87effd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89009f360 con 0x55a4d2a57dd0 2023-10-16T14:34:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff89009f550 con 0x55a4d2a57dd0 2023-10-16T14:34:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff89009b400 con 0x55a4d2a57dd0 2023-10-16T14:34:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff890099810 con 0x55a4d2a57dd0 2023-10-16T14:34:05.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff8b99a2b80 10 client.15372 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.900+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a4d27fc200 con 0x55a4d2a57dd0 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff89009b400 con 0x55a4d2a57dd0 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff89009b760 con 0x55a4d2a57dd0 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 10 client.15372 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-16T14:34:05.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 20 client.15372 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2ac5ba0 0x55a4d2ac1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 4 client.15372 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T14:34:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a4d29a3e00 con 0x55a4d2ac5ba0 2023-10-16T14:34:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.901+0000 7ff89effd700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2ac5ba0 0x55a4d2ac1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:05.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff87effd700 10 client.15372 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff89effd700 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2ac5ba0 0x55a4d2ac1fd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f61d7536378bcf3 server_cookie=1cbc51ff2c65d17c in_seq=0 out_seq=0 2023-10-16T14:34:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff87effd700 1 -- 172.21.15.178:0/2887119268 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a4d29a3e00 con 0x55a4d2ac5ba0 2023-10-16T14:34:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff87effd700 10 client.15372 handle_command_reply: tid=0 2023-10-16T14:34:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 client.15372 shutdown 2023-10-16T14:34:05.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 msgr2=0x55a4d2a68900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff890005cd0 tx=0x7ff89000b390 comp rx=0 tx=0).stop 2023-10-16T14:34:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 shutdown_connections 2023-10-16T14:34:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55a4d2ac5ba0 0x55a4d2ac1fd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4d2a595d0 0x55a4d2a65490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a4d2a57dd0 0x55a4d2a68900 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 --2- 172.21.15.178:0/2887119268 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a4d2a58730 0x55a4d2a68e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 >> 172.21.15.178:0/2887119268 conn(0x55a4d2985d80 msgr2=0x55a4d2a5f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff87effd700 0 client.15372 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 shutdown_connections 2023-10-16T14:34:05.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.902+0000 7ff8b99a2b80 1 -- 172.21.15.178:0/2887119268 wait complete. 2023-10-16T14:34:05.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8b99a2b80 20 client.15372 trim_cache size 0 max 16384 2023-10-16T14:34:05.920 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "cluster_fsid": "8c31bde4-6c2f-11ee-8db6-212e2dc638e7", 2023-10-16T14:34:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "whoami": 0, 2023-10-16T14:34:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24593, 2023-10-16T14:34:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "want_state": "up:active", 2023-10-16T14:34:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "state": "up:active", 2023-10-16T14:34:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs", 2023-10-16T14:34:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "rank_uptime": 148.85044424, 2023-10-16T14:34:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "mdsmap_epoch": 26, 2023-10-16T14:34:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch": 91, 2023-10-16T14:34:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_epoch_barrier": 89, 2023-10-16T14:34:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "uptime": 154.99628079499999 2023-10-16T14:34:05.922 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 msgr2=0x7ff8880a3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 0x7ff8880a3a10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff898009fd0 tx=0x7ff8980115a0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 msgr2=0x7ff8a0198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff890004710 tx=0x7ff890009580 comp rx=0 tx=0).stop 2023-10-16T14:34:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 shutdown_connections 2023-10-16T14:34:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8880a1550 0x7ff8880a3a10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff8a0102a40 0x7ff8a01a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8a0100ea0 0x7ff8a0198e20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 --2- 172.21.15.178:0/839627954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8a0102080 0x7ff8a0199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.903+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 >> 172.21.15.178:0/839627954 conn(0x7ff8a00fc420 msgr2=0x7ff8a00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.904+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 shutdown_connections 2023-10-16T14:34:05.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:05.904+0000 7ff8a5fff700 1 -- 172.21.15.178:0/839627954 wait complete. 2023-10-16T14:34:05.926 INFO:teuthology.run_tasks:Running task kclient... 2023-10-16T14:34:05.939 INFO:tasks.kclient:Mounting kernel clients... 2023-10-16T14:34:05.939 INFO:tasks.kclient:config is {'client.0': {}} 2023-10-16T14:34:05.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:05.940 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-10-16T14:34:05.986 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-10-16T14:34:05.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:05.988 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-10-16T14:34:06.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:06.106 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-10-16T14:34:06.156 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: pgmap v350: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:06.156 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.157 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.157 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.157 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.157 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4173296198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:06.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: pgmap v350: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:06.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4173296198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:06.245 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2023-10-16T14:34:06.246 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T14:34:06.246 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:06.247 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph fs ls 2023-10-16T14:34:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: pgmap v350: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:06.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4173296198' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:07.002 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:34:07.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: pgmap v351: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:34:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:34:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.470 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.471 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.471 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: pgmap v351: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:07.471 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:34:07.471 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:34:07.471 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:07.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: pgmap v351: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:34:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:34:07.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:34:08.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 -- 172.21.15.178:0/2973119999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648100f80 msgr2=0x7fd648101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:08.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/2973119999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648100f80 0x7fd648101400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd6380099c0 tx=0x7fd638009cd0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 -- 172.21.15.178:0/2973119999 shutdown_connections 2023-10-16T14:34:08.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/2973119999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648101940 0x7fd64810ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/2973119999 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648106f90 0x7fd648107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/2973119999 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648100f80 0x7fd648101400 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 -- 172.21.15.178:0/2973119999 >> 172.21.15.178:0/2973119999 conn(0x7fd6480780a0 msgr2=0x7fd6480784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:08.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 -- 172.21.15.178:0/2973119999 shutdown_connections 2023-10-16T14:34:08.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.664+0000 7fd64fbc4700 1 -- 172.21.15.178:0/2973119999 wait complete. 2023-10-16T14:34:08.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.665+0000 7fd64fbc4700 1 Processor -- start 2023-10-16T14:34:08.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.665+0000 7fd64fbc4700 1 -- start start 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648101940 0x7fd6481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 0x7fd64819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64819e200 con 0x7fd648106f90 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64819e370 con 0x7fd648101940 2023-10-16T14:34:08.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64fbc4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64819e530 con 0x7fd648100f80 2023-10-16T14:34:08.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64e161700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 0x7fd64819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:08.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:08.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54692/0 (socket says 172.21.15.178:54692) 2023-10-16T14:34:08.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 -- 172.21.15.178:0/1030050436 learned_addr learned my addr 172.21.15.178:0/1030050436 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:08.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d15f700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648101940 0x7fd6481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:08.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 -- 172.21.15.178:0/1030050436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648101940 msgr2=0x7fd6481997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:08.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648101940 0x7fd6481997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 -- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 msgr2=0x7fd64819dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:08.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 0x7fd64819dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64d960700 1 -- 172.21.15.178:0/1030050436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380095f0 con 0x7fd648100f80 2023-10-16T14:34:08.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.666+0000 7fd64e161700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 0x7fd64819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:08.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd64d960700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd64400e9a0 tx=0x7fd64400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:08.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644009790 con 0x7fd648100f80 2023-10-16T14:34:08.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64819e810 con 0x7fd648100f80 2023-10-16T14:34:08.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd644010430 con 0x7fd648100f80 2023-10-16T14:34:08.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644018680 con 0x7fd648100f80 2023-10-16T14:34:08.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.667+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6481a6070 con 0x7fd648100f80 2023-10-16T14:34:08.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.668+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64804ebb0 con 0x7fd648100f80 2023-10-16T14:34:08.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.670+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd6440105a0 con 0x7fd648100f80 2023-10-16T14:34:08.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.671+0000 7fd63effd700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 0x7fd6340a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:08.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.672+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd644014070 con 0x7fd648100f80 2023-10-16T14:34:08.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.672+0000 7fd64d15f700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 0x7fd6340a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:08.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.672+0000 7fd64d15f700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 0x7fd6340a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd638009990 tx=0x7fd638005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:08.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.674+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd6440966a0 con 0x7fd648100f80 2023-10-16T14:34:08.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.919+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd648110ea0 con 0x7fd648100f80 2023-10-16T14:34:08.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.919+0000 7fd63effd700 1 -- 172.21.15.178:0/1030050436 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd644099cc0 con 0x7fd648100f80 2023-10-16T14:34:08.922 INFO:teuthology.orchestra.run.smithi178.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-10-16T14:34:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 msgr2=0x7fd6340a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 0x7fd6340a3940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd638009990 tx=0x7fd638005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 msgr2=0x7fd648199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd64400e9a0 tx=0x7fd64400ecb0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 shutdown_connections 2023-10-16T14:34:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd6340a1480 0x7fd6340a3940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd648106f90 0x7fd64819dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd648100f80 0x7fd648199290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 --2- 172.21.15.178:0/1030050436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd648101940 0x7fd6481997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 >> 172.21.15.178:0/1030050436 conn(0x7fd6480780a0 msgr2=0x7fd648073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 shutdown_connections 2023-10-16T14:34:08.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:08.922+0000 7fd64fbc4700 1 -- 172.21.15.178:0/1030050436 wait complete. 2023-10-16T14:34:09.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T14:34:09.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T14:34:09.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2023-10-16T14:34:09.395 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T14:34:09.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:09.396 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T14:34:09.396 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T14:34:09.396 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T14:34:09.396 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T14:34:09.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:09.397 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2023-10-16T14:34:09.443 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T14:34:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T14:34:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T14:34:09.443 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2023-10-16T14:34:09.444 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:34:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:34:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2023-10-16T14:34:09.445 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-16T14:34:09.445 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:34:09.445 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T14:34:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2023-10-16T14:34:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-16T14:34:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 41617sec preferred_lft 41617sec 2023-10-16T14:34:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::92fb:5c5a:625:714e/64 scope link noprefixroute 2023-10-16T14:34:09.446 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-10-16T14:34:09.448 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T14:34:09.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:09.448 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:34:09.449 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:34:09.449 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2023-10-16T14:34:09.449 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2023-10-16T14:34:09.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2023-10-16T14:34:09.450 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T14:34:09.450 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:34:09.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:09.696 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T14:34:09.766 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-10-16T14:34:09.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:09.769 DEBUG:teuthology.orchestra.run.smithi178:> route 2023-10-16T14:34:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2023-10-16T14:34:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T14:34:09.816 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T14:34:09.817 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T14:34:09.817 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T14:34:09.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:09.819 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:34:09.819 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:34:09.819 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T14:34:09.819 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T14:34:09.820 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T14:34:09.820 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:34:09.959 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:09 smithi178 ceph-mon[98495]: pgmap v352: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:09.960 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:09 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1030050436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:34:10.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.030 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-10-16T14:34:10.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.075 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2023-10-16T14:34:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:09 smithi186 ceph-mon[134892]: pgmap v352: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:09 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1030050436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:34:10.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.160 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:34:10.160 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:34:10.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T14:34:10.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T14:34:10.161 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:34:10.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:09 smithi187 ceph-mon[135297]: pgmap v352: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:09 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1030050436' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-10-16T14:34:10.343 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T14:34:10.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.344 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:34:10.344 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:34:10.344 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T14:34:10.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T14:34:10.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T14:34:10.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T14:34:10.345 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T14:34:10.346 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:34:10.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.589 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-10-16T14:34:10.590 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-10-16T14:34:10.590 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2023-10-16T14:34:10.590 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T14:34:10.591 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-10-16T14:34:10.763 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T14:34:10.764 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:10.821 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T14:34:10.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T14:34:10.823 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:10.914 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-10-16T14:34:10.915 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-10-16T14:34:10.915 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408 2023-10-16T14:34:10.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:10.917 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-16T14:34:11.066 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-16T14:34:11.066 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-10-16T14:34:11.067 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-10-16T14:34:11.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:11 smithi178 ceph-mon[98495]: pgmap v353: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:11.547 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2023-10-16T14:34:11.548 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2023-10-16T14:34:11.548 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-10-16T14:34:11.548 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.178:6789,172.21.15.186:6789,172.21.15.187:6789" 2023-10-16T14:34:11.548 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.178:6789,172.21.15.186:6789,172.21.15.187:6789:/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408 2023-10-16T14:34:11.549 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=8c31bde4-6c2f-11ee-8db6-212e2dc638e7" will pass to kernel 2023-10-16T14:34:11.555 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-16T14:34:11.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:11.556 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:11 smithi186 ceph-mon[134892]: pgmap v353: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:11.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:11.630 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T14:34:11.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:11 smithi187 ceph-mon[135297]: pgmap v353: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:11.717 INFO:teuthology.orchestra.run.smithi178.stdout:client15381 2023-10-16T14:34:11.717 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:34:11.718 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:34:11.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:11.763 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T14:34:11.889 INFO:teuthology.orchestra.run.smithi178.stdout:client15381 2023-10-16T14:34:11.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:11.890 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-10-16T14:34:11.977 INFO:teuthology.orchestra.run.smithi178.stdout:client15381 2023-10-16T14:34:11.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:34:11.978 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7.client15381/status') 2023-10-16T14:34:12.086 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.15381 (3)192.168.0.1:0/763321561 2023-10-16T14:34:12.086 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2023-10-16T14:34:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2023-10-16T14:34:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2023-10-16T14:34:12.087 INFO:teuthology.orchestra.run.smithi178.stderr:69 bytes copied, 5.363e-05 s, 1.3 MB/s 2023-10-16T14:34:12.089 INFO:teuthology.run_tasks:Running task check-counter... 2023-10-16T14:34:12.105 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-10-16T14:34:12.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 -- 172.21.15.178:0/466550903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 msgr2=0x7f3b10104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 --2- 172.21.15.178:0/466550903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10104760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009990 tx=0x7f3b00009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 -- 172.21.15.178:0/466550903 shutdown_connections 2023-10-16T14:34:12.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 --2- 172.21.15.178:0/466550903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10104ca0 0x7f3b1010b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 --2- 172.21.15.178:0/466550903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10103110 0x7f3b10103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 --2- 172.21.15.178:0/466550903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10104760 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 -- 172.21.15.178:0/466550903 >> 172.21.15.178:0/466550903 conn(0x7f3b100fe660 msgr2=0x7f3b10100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 -- 172.21.15.178:0/466550903 shutdown_connections 2023-10-16T14:34:12.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.330+0000 7f3b17bff700 1 -- 172.21.15.178:0/466550903 wait complete. 2023-10-16T14:34:12.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.331+0000 7f3b17bff700 1 Processor -- start 2023-10-16T14:34:12.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.331+0000 7f3b17bff700 1 -- start start 2023-10-16T14:34:12.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10103110 0x7f3b10198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101a0870 con 0x7f3b10104ca0 2023-10-16T14:34:12.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101a09e0 con 0x7f3b10103110 2023-10-16T14:34:12.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b17bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b101a0ba0 con 0x7f3b10104300 2023-10-16T14:34:12.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40760/0 (socket says 172.21.15.178:40760) 2023-10-16T14:34:12.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 -- 172.21.15.178:0/1773060153 learned_addr learned my addr 172.21.15.178:0/1773060153 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1519a700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1599b700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10103110 0x7f3b10198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 -- 172.21.15.178:0/1773060153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 msgr2=0x7f3b10199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 -- 172.21.15.178:0/1773060153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10103110 msgr2=0x7f3b10198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10103110 0x7f3b10198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.332+0000 7f3b1619c700 1 -- 172.21.15.178:0/1773060153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000095f0 con 0x7f3b10104ca0 2023-10-16T14:34:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b1619c700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800ef30 tx=0x7f3b0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:12.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08018070 con 0x7f3b10104ca0 2023-10-16T14:34:12.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b101a0e20 con 0x7f3b10104ca0 2023-10-16T14:34:12.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b0800f040 con 0x7f3b10104ca0 2023-10-16T14:34:12.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0800cd00 con 0x7f3b10104ca0 2023-10-16T14:34:12.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.333+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b101a1240 con 0x7f3b10104ca0 2023-10-16T14:34:12.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.334+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1010e970 con 0x7f3b10104ca0 2023-10-16T14:34:12.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.336+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3b08003a00 con 0x7f3b10104ca0 2023-10-16T14:34:12.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.337+0000 7f3b06ffd700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 0x7f3afc0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.337+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3b080ccc10 con 0x7f3b10104ca0 2023-10-16T14:34:12.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.337+0000 7f3b1599b700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 0x7f3afc0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.338+0000 7f3b1599b700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 0x7f3afc0a39e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b1019a200 tx=0x7f3b0c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:12.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.341+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3b08095f30 con 0x7f3b10104ca0 2023-10-16T14:34:12.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.544+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b10109f20 con 0x7f3b10104ca0 2023-10-16T14:34:12.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.545+0000 7f3b06ffd700 1 -- 172.21.15.178:0/1773060153 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7f3b08099550 con 0x7f3b10104ca0 2023-10-16T14:34:12.548 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:12.549 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:12.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 msgr2=0x7f3afc0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 0x7f3afc0a39e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b1019a200 tx=0x7f3b0c009260 comp rx=0 tx=0).stop 2023-10-16T14:34:12.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 msgr2=0x7f3b101a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800ef30 tx=0x7f3b0800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 shutdown_connections 2023-10-16T14:34:12.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3afc0a1520 0x7f3afc0a39e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b10104ca0 0x7f3b101a02a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3b10103110 0x7f3b10198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 --2- 172.21.15.178:0/1773060153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10104300 0x7f3b10199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 >> 172.21.15.178:0/1773060153 conn(0x7f3b100fe660 msgr2=0x7f3b101007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:12.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 shutdown_connections 2023-10-16T14:34:12.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.548+0000 7f3b17bff700 1 -- 172.21.15.178:0/1773060153 wait complete. 2023-10-16T14:34:12.571 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:34:12.571 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:34:12.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 -- 172.21.15.178:0/2981179072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95800ff7d0 msgr2=0x7f95800ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 --2- 172.21.15.178:0/2981179072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95800ff7d0 0x7f95800ffbf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f95740099c0 tx=0x7f9574009cd0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 -- 172.21.15.178:0/2981179072 shutdown_connections 2023-10-16T14:34:12.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 --2- 172.21.15.178:0/2981179072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9580100af0 0x7f9580100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 --2- 172.21.15.178:0/2981179072 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f95800ff7d0 0x7f95800ffbf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 --2- 172.21.15.178:0/2981179072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100130 0x7f95801005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 -- 172.21.15.178:0/2981179072 >> 172.21.15.178:0/2981179072 conn(0x7f95800fb370 msgr2=0x7f95800fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:12.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 -- 172.21.15.178:0/2981179072 shutdown_connections 2023-10-16T14:34:12.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.770+0000 7f9586fff700 1 -- 172.21.15.178:0/2981179072 wait complete. 2023-10-16T14:34:12.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.771+0000 7f9586fff700 1 Processor -- start 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 -- start start 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95800ff7d0 0x7f9580198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9580100130 0x7f95801992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a08f0 con 0x7f9580100af0 2023-10-16T14:34:12.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a0a30 con 0x7f9580100130 2023-10-16T14:34:12.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9586fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a0bc0 con 0x7f95800ff7d0 2023-10-16T14:34:12.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.782 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40774/0 (socket says 172.21.15.178:40774) 2023-10-16T14:34:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 -- 172.21.15.178:0/375396429 learned_addr learned my addr 172.21.15.178:0/375396429 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f9584d9b700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95800ff7d0 0x7f9580198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f957ffff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9580100130 0x7f95801992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 -- 172.21.15.178:0/375396429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95800ff7d0 msgr2=0x7f9580198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95800ff7d0 0x7f9580198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 -- 172.21.15.178:0/375396429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9580100130 msgr2=0x7f95801992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.772+0000 7f958559c700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9580100130 0x7f95801992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f958559c700 1 -- 172.21.15.178:0/375396429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740095f0 con 0x7f9580100af0 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f958559c700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f957800ef30 tx=0x7f957800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578018070 con 0x7f9580100af0 2023-10-16T14:34:12.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957800f040 con 0x7f9580100af0 2023-10-16T14:34:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95801a0e40 con 0x7f9580100af0 2023-10-16T14:34:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800cd00 con 0x7f9580100af0 2023-10-16T14:34:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.773+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95801a12c0 con 0x7f9580100af0 2023-10-16T14:34:12.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.774+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958010a5b0 con 0x7f9580100af0 2023-10-16T14:34:12.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.776+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9578003a00 con 0x7f9580100af0 2023-10-16T14:34:12.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.777+0000 7f957dffb700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 0x7f95700a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:12.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.778+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f95780ccbc0 con 0x7f9580100af0 2023-10-16T14:34:12.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.778+0000 7f9584d9b700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 0x7f95700a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:12.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.778+0000 7f9584d9b700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 0x7f95700a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f957400b580 tx=0x7f9574005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:12.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.782+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9578095ee0 con 0x7f9580100af0 2023-10-16T14:34:12.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:12 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1773060153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:12.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.983+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f958002cf70 con 0x7f9580100af0 2023-10-16T14:34:12.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.984+0000 7f957dffb700 1 -- 172.21.15.178:0/375396429 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9578099500 con 0x7f9580100af0 2023-10-16T14:34:12.987 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:34:12.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 msgr2=0x7f95700a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 0x7f95700a39c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f957400b580 tx=0x7f9574005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 msgr2=0x7f95801a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:12.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f957800ef30 tx=0x7f957800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 shutdown_connections 2023-10-16T14:34:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95700a1500 0x7f95700a39c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9580100af0 0x7f95801a0320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95800ff7d0 0x7f9580198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 --2- 172.21.15.178:0/375396429 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9580100130 0x7f95801992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:12.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 >> 172.21.15.178:0/375396429 conn(0x7f95800fb370 msgr2=0x7f95800fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:12.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 shutdown_connections 2023-10-16T14:34:12.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:12.986+0000 7f9586fff700 1 -- 172.21.15.178:0/375396429 wait complete. 2023-10-16T14:34:13.006 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:12 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1773060153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:13.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:34:13.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:12 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1773060153' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:13.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 -- 172.21.15.178:0/960098244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 msgr2=0x7f9fc0101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 --2- 172.21.15.178:0/960098244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0101090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0099c0 tx=0x7f9fbc009cd0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 -- 172.21.15.178:0/960098244 shutdown_connections 2023-10-16T14:34:13.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 --2- 172.21.15.178:0/960098244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc01090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 --2- 172.21.15.178:0/960098244 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0101090 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 --2- 172.21.15.178:0/960098244 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 -- 172.21.15.178:0/960098244 >> 172.21.15.178:0/960098244 conn(0x7f9fc00fc1a0 msgr2=0x7f9fc00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:13.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 -- 172.21.15.178:0/960098244 shutdown_connections 2023-10-16T14:34:13.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.209+0000 7f9fc759d700 1 -- 172.21.15.178:0/960098244 wait complete. 2023-10-16T14:34:13.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.210+0000 7f9fc759d700 1 Processor -- start 2023-10-16T14:34:13.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.210+0000 7f9fc759d700 1 -- start start 2023-10-16T14:34:13.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0198460 con 0x7f9fc0100c70 2023-10-16T14:34:13.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc01985d0 con 0x7f9fc0101e60 2023-10-16T14:34:13.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc759d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc0198790 con 0x7f9fc0102800 2023-10-16T14:34:13.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc5d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc5d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37680/0 (socket says 172.21.15.178:37680) 2023-10-16T14:34:13.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc5d9a700 1 -- 172.21.15.178:0/421030777 learned_addr learned my addr 172.21.15.178:0/421030777 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:13.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc6d9c700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 -- 172.21.15.178:0/421030777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 msgr2=0x7f9fc0197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc0197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 -- 172.21.15.178:0/421030777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 msgr2=0x7f9fc0196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 -- 172.21.15.178:0/421030777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fbc0095f0 con 0x7f9fc0100c70 2023-10-16T14:34:13.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc6d9c700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:13.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc5d9a700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:13.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.211+0000 7f9fc659b700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0196a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0098e0 tx=0x7f9fbc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:13.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.212+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc01d070 con 0x7f9fc0100c70 2023-10-16T14:34:13.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.212+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fbc004010 con 0x7f9fc0100c70 2023-10-16T14:34:13.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.212+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc0198a10 con 0x7f9fc0100c70 2023-10-16T14:34:13.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.212+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00fd00 con 0x7f9fc0100c70 2023-10-16T14:34:13.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.212+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc019f230 con 0x7f9fc0100c70 2023-10-16T14:34:13.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.213+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc0108250 con 0x7f9fc0100c70 2023-10-16T14:34:13.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.215+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9fbc00f460 con 0x7f9fc0100c70 2023-10-16T14:34:13.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.216+0000 7f9fb77fe700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 0x7f9fac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.216+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9fbc0ce290 con 0x7f9fc0100c70 2023-10-16T14:34:13.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.216+0000 7f9fc5d9a700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 0x7f9fac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.217+0000 7f9fc5d9a700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 0x7f9fac0a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005950 tx=0x7f9fb000b500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:13.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.218+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9fbc0975b0 con 0x7f9fc0100c70 2023-10-16T14:34:13.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.419+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fc010c4d0 con 0x7f9fc0100c70 2023-10-16T14:34:13.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.419+0000 7f9fb77fe700 1 -- 172.21.15.178:0/421030777 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fbc026020 con 0x7f9fc0100c70 2023-10-16T14:34:13.423 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:13.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 msgr2=0x7f9fac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 0x7f9fac0a39c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005950 tx=0x7f9fb000b500 comp rx=0 tx=0).stop 2023-10-16T14:34:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 msgr2=0x7f9fc0196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0196a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc0098e0 tx=0x7f9fbc005740 comp rx=0 tx=0).stop 2023-10-16T14:34:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 shutdown_connections 2023-10-16T14:34:13.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9fac0a1500 0x7f9fac0a39c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fc0102800 0x7f9fc0197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9fc0100c70 0x7f9fc0196a40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 --2- 172.21.15.178:0/421030777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9fc0101e60 0x7f9fc0196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 >> 172.21.15.178:0/421030777 conn(0x7f9fc00fc1a0 msgr2=0x7f9fc0105390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 shutdown_connections 2023-10-16T14:34:13.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.422+0000 7f9fc759d700 1 -- 172.21.15.178:0/421030777 wait complete. 2023-10-16T14:34:13.444 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:34:13.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 -- 172.21.15.178:0/947144967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098100e80 msgr2=0x7ff0981012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 --2- 172.21.15.178:0/947144967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098100e80 0x7ff0981012a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff088009990 tx=0x7ff088009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 -- 172.21.15.178:0/947144967 shutdown_connections 2023-10-16T14:34:13.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 --2- 172.21.15.178:0/947144967 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 0x7ff098109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 --2- 172.21.15.178:0/947144967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098100e80 0x7ff0981012a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 --2- 172.21.15.178:0/947144967 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098102060 0x7ff0981024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 -- 172.21.15.178:0/947144967 >> 172.21.15.178:0/947144967 conn(0x7ff0980fc400 msgr2=0x7ff0980fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:13.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 -- 172.21.15.178:0/947144967 shutdown_connections 2023-10-16T14:34:13.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.644+0000 7ff09cbff700 1 -- 172.21.15.178:0/947144967 wait complete. 2023-10-16T14:34:13.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 Processor -- start 2023-10-16T14:34:13.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 -- start start 2023-10-16T14:34:13.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 0x7ff098198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 0x7ff0981a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a09b0 con 0x7ff098102060 2023-10-16T14:34:13.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a0af0 con 0x7ff098102a20 2023-10-16T14:34:13.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.645+0000 7ff09cbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0981a0c80 con 0x7ff098100e80 2023-10-16T14:34:13.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff09659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 0x7ff098198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40824/0 (socket says 172.21.15.178:40824) 2023-10-16T14:34:13.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 -- 172.21.15.178:0/857674684 learned_addr learned my addr 172.21.15.178:0/857674684 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:13.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff096d9d700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 0x7ff0981a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 -- 172.21.15.178:0/857674684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 msgr2=0x7ff098198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 0x7ff098198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 -- 172.21.15.178:0/857674684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 msgr2=0x7ff0981a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 0x7ff0981a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 -- 172.21.15.178:0/857674684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0880095f0 con 0x7ff098102060 2023-10-16T14:34:13.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff09659c700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 0x7ff098198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:13.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff095d9b700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e970 tx=0x7ff08c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:13.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c009790 con 0x7ff098102060 2023-10-16T14:34:13.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff08c00c7e0 con 0x7ff098102060 2023-10-16T14:34:13.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0981a0f60 con 0x7ff098102060 2023-10-16T14:34:13.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.646+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c003c70 con 0x7ff098102060 2023-10-16T14:34:13.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.647+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0981a1430 con 0x7ff098102060 2023-10-16T14:34:13.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.647+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff098193530 con 0x7ff098102060 2023-10-16T14:34:13.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.649+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff08c00cc50 con 0x7ff098102060 2023-10-16T14:34:13.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.651+0000 7ff0877fe700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 0x7ff0800a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:13.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.651+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff08c014070 con 0x7ff098102060 2023-10-16T14:34:13.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.651+0000 7ff09659c700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 0x7ff0800a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:13.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.652+0000 7ff09659c700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 0x7ff0800a39c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b2e0 tx=0x7ff088005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:13.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.653+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff08c0965e0 con 0x7ff098102060 2023-10-16T14:34:13.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:13 smithi178 ceph-mon[98495]: pgmap v354: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:13 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/375396429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:13 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/421030777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:13.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.854+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff09802cf70 con 0x7ff098102060 2023-10-16T14:34:13.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.854+0000 7ff0877fe700 1 -- 172.21.15.178:0/857674684 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff08c003dd0 con 0x7ff098102060 2023-10-16T14:34:13.858 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:34:13.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 msgr2=0x7ff0800a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 0x7ff0800a39c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff08800b2e0 tx=0x7ff088005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 msgr2=0x7ff098199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:13.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00e970 tx=0x7ff08c00ec80 comp rx=0 tx=0).stop 2023-10-16T14:34:13.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 shutdown_connections 2023-10-16T14:34:13.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff0800a1500 0x7ff0800a39c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff098102a20 0x7ff0981a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff098100e80 0x7ff098198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 --2- 172.21.15.178:0/857674684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff098102060 0x7ff098199370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:13.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 >> 172.21.15.178:0/857674684 conn(0x7ff0980fc400 msgr2=0x7ff098107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:13.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 shutdown_connections 2023-10-16T14:34:13.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:13.857+0000 7ff09cbff700 1 -- 172.21.15.178:0/857674684 wait complete. 2023-10-16T14:34:13.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:34:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:13 smithi187 ceph-mon[135297]: pgmap v354: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:13 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/375396429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:13 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/421030777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:13 smithi186 ceph-mon[134892]: pgmap v354: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:13 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/375396429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:14.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:13 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/421030777' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:14.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 -- 172.21.15.178:0/2141104471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102060 msgr2=0x7f12401024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 --2- 172.21.15.178:0/2141104471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102060 0x7f12401024e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1234009990 tx=0x7f1234009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 -- 172.21.15.178:0/2141104471 shutdown_connections 2023-10-16T14:34:14.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 --2- 172.21.15.178:0/2141104471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102a20 0x7f1240109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 --2- 172.21.15.178:0/2141104471 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f12401012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 --2- 172.21.15.178:0/2141104471 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102060 0x7f12401024e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 -- 172.21.15.178:0/2141104471 >> 172.21.15.178:0/2141104471 conn(0x7f12400fc400 msgr2=0x7f12400fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:14.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 -- 172.21.15.178:0/2141104471 shutdown_connections 2023-10-16T14:34:14.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.078+0000 7f12463ff700 1 -- 172.21.15.178:0/2141104471 wait complete. 2023-10-16T14:34:14.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 Processor -- start 2023-10-16T14:34:14.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 -- start start 2023-10-16T14:34:14.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102060 0x7f12400732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 0x7f12400737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1240073f80 con 0x7f1240102a20 2023-10-16T14:34:14.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12400740f0 con 0x7f1240100e80 2023-10-16T14:34:14.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f12463ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12401a0c60 con 0x7f1240102060 2023-10-16T14:34:14.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f123ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f124499c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 0x7f12400737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f123ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37722/0 (socket says 172.21.15.178:37722) 2023-10-16T14:34:14.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f123ffff700 1 -- 172.21.15.178:0/706739756 learned_addr learned my addr 172.21.15.178:0/706739756 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:14.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.079+0000 7f123f7fe700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102060 0x7f12400732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 -- 172.21.15.178:0/706739756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102060 msgr2=0x7f12400732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102060 0x7f12400732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 -- 172.21.15.178:0/706739756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 msgr2=0x7f1240074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 -- 172.21.15.178:0/706739756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340095f0 con 0x7f1240102a20 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f123ffff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:14.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f124499c700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 0x7f12400737e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f123800efc0 tx=0x7f123800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1238018070 con 0x7f1240102a20 2023-10-16T14:34:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f123800f040 con 0x7f1240102a20 2023-10-16T14:34:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123800cd40 con 0x7f1240102a20 2023-10-16T14:34:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12401a0ee0 con 0x7f1240102a20 2023-10-16T14:34:14.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.080+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12401a12b0 con 0x7f1240102a20 2023-10-16T14:34:14.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.081+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1240107e80 con 0x7f1240102a20 2023-10-16T14:34:14.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.083+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1238003a00 con 0x7f1240102a20 2023-10-16T14:34:14.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.085+0000 7f123d7fa700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 0x7f12280a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.085+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f12380cd650 con 0x7f1240102a20 2023-10-16T14:34:14.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.085+0000 7f123ffff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 0x7f12280a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.085+0000 7f123ffff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 0x7f12280a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1230005950 tx=0x7f1230009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.087+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1238096970 con 0x7f1240102a20 2023-10-16T14:34:14.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.289+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f124002d090 con 0x7f1240102a20 2023-10-16T14:34:14.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.289+0000 7f123d7fa700 1 -- 172.21.15.178:0/706739756 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1238099f90 con 0x7f1240102a20 2023-10-16T14:34:14.293 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 msgr2=0x7f12280a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 0x7f12280a3940 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1230005950 tx=0x7f1230009260 comp rx=0 tx=0).stop 2023-10-16T14:34:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 msgr2=0x7f12400737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 0x7f12400737e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f123800efc0 tx=0x7f123800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 shutdown_connections 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f12280a1480 0x7f12280a3940 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1240102a20 0x7f12400737e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1240100e80 0x7f1240074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 --2- 172.21.15.178:0/706739756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1240102060 0x7f12400732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 >> 172.21.15.178:0/706739756 conn(0x7f12400fc400 msgr2=0x7f12400fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:14.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 shutdown_connections 2023-10-16T14:34:14.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.292+0000 7f12463ff700 1 -- 172.21.15.178:0/706739756 wait complete. 2023-10-16T14:34:14.315 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:14.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3725388876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 msgr2=0x7f6f28109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3725388876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f28109360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6f2000b0a0 tx=0x7f6f2000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3725388876 shutdown_connections 2023-10-16T14:34:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3725388876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f28109360 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3725388876 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28100ea0 0x7f6f281012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3725388876 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28102080 0x7f6f28102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3725388876 >> 172.21.15.178:0/3725388876 conn(0x7f6f280fc420 msgr2=0x7f6f280fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:14.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3725388876 shutdown_connections 2023-10-16T14:34:14.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.509+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3725388876 wait complete. 2023-10-16T14:34:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.510+0000 7f6f2e3ff700 1 Processor -- start 2023-10-16T14:34:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.510+0000 7f6f2e3ff700 1 -- start start 2023-10-16T14:34:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28100ea0 0x7f6f28198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28102080 0x7f6f281993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f281a09e0 con 0x7f6f28102a40 2023-10-16T14:34:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f281a0b20 con 0x7f6f28102080 2023-10-16T14:34:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2e3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f281a0cb0 con 0x7f6f28100ea0 2023-10-16T14:34:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40862/0 (socket says 172.21.15.178:40862) 2023-10-16T14:34:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 -- 172.21.15.178:0/3618417006 learned_addr learned my addr 172.21.15.178:0/3618417006 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f277fe700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28102080 0x7f6f281993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f27fff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28100ea0 0x7f6f28198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 -- 172.21.15.178:0/3618417006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28100ea0 msgr2=0x7f6f28198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28100ea0 0x7f6f28198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 -- 172.21.15.178:0/3618417006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28102080 msgr2=0x7f6f281993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28102080 0x7f6f281993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.511+0000 7f6f2c99c700 1 -- 172.21.15.178:0/3618417006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c009520 con 0x7f6f28102a40 2023-10-16T14:34:14.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f2c99c700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fd0 tx=0x7f6f2000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2000e040 con 0x7f6f28102a40 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f200094a0 con 0x7f6f28102a40 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f20009d00 con 0x7f6f28102a40 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2001b6c0 con 0x7f6f28102a40 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.512+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f281a11c0 con 0x7f6f28102a40 2023-10-16T14:34:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.513+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2810c7d0 con 0x7f6f28102a40 2023-10-16T14:34:14.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.515+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6f2001d470 con 0x7f6f28102a40 2023-10-16T14:34:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.516+0000 7f6f257fa700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 0x7f6f100a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.517+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6f200cdfb0 con 0x7f6f28102a40 2023-10-16T14:34:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.517+0000 7f6f27fff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 0x7f6f100a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.517+0000 7f6f27fff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 0x7f6f100a39e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f28101bf0 tx=0x7f6f18009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.520+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6f20097330 con 0x7f6f28102a40 2023-10-16T14:34:14.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.724+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f281a1350 con 0x7f6f28102a40 2023-10-16T14:34:14.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.726+0000 7f6f257fa700 1 -- 172.21.15.178:0/3618417006 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7f6f2009a950 con 0x7f6f28102a40 2023-10-16T14:34:14.730 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:14.730 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 msgr2=0x7f6f100a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 0x7f6f100a39e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6f28101bf0 tx=0x7f6f18009260 comp rx=0 tx=0).stop 2023-10-16T14:34:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 msgr2=0x7f6f281a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fd0 tx=0x7f6f2000bce0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 shutdown_connections 2023-10-16T14:34:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6f100a1520 0x7f6f100a39e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f28102a40 0x7f6f281a0410 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f28100ea0 0x7f6f28198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 --2- 172.21.15.178:0/3618417006 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6f28102080 0x7f6f281993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 >> 172.21.15.178:0/3618417006 conn(0x7f6f280fc420 msgr2=0x7f6f281078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 shutdown_connections 2023-10-16T14:34:14.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.728+0000 7f6f2e3ff700 1 -- 172.21.15.178:0/3618417006 wait complete. 2023-10-16T14:34:14.751 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:34:14.753 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:34:14.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:14 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/857674684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:14.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:14 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/706739756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:14.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 -- 172.21.15.178:0/3423894510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 msgr2=0x7f8298109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 --2- 172.21.15.178:0/3423894510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f8298109360 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f829000b0a0 tx=0x7f829000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 -- 172.21.15.178:0/3423894510 shutdown_connections 2023-10-16T14:34:14.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 --2- 172.21.15.178:0/3423894510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f8298109360 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 --2- 172.21.15.178:0/3423894510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 0x7f82981012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 --2- 172.21.15.178:0/3423894510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 0x7f8298102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 -- 172.21.15.178:0/3423894510 >> 172.21.15.178:0/3423894510 conn(0x7f82980fc420 msgr2=0x7f82980fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:14.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 -- 172.21.15.178:0/3423894510 shutdown_connections 2023-10-16T14:34:14.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.950+0000 7f829fbff700 1 -- 172.21.15.178:0/3423894510 wait complete. 2023-10-16T14:34:14.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 Processor -- start 2023-10-16T14:34:14.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 -- start start 2023-10-16T14:34:14.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 0x7f8298196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 0x7f8298197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.951+0000 7f829fbff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829819e870 con 0x7f8298102a40 2023-10-16T14:34:14.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829fbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829819e9e0 con 0x7f8298102080 2023-10-16T14:34:14.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829fbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829819eba0 con 0x7f8298100ea0 2023-10-16T14:34:14.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40874/0 (socket says 172.21.15.178:40874) 2023-10-16T14:34:14.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 -- 172.21.15.178:0/169000254 learned_addr learned my addr 172.21.15.178:0/169000254 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:14.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829d99b700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 0x7f8298196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829d19a700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 0x7f8298197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 -- 172.21.15.178:0/169000254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 msgr2=0x7f8298196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 0x7f8298196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 -- 172.21.15.178:0/169000254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 msgr2=0x7f8298197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:14.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 0x7f8298197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:14.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 -- 172.21.15.178:0/169000254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009d00 con 0x7f8298102a40 2023-10-16T14:34:14.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.952+0000 7f829e19c700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8290009fd0 tx=0x7f829000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.953+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000e040 con 0x7f8298102a40 2023-10-16T14:34:14.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.953+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829819ee20 con 0x7f8298102a40 2023-10-16T14:34:14.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.953+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8290003e10 con 0x7f8298102a40 2023-10-16T14:34:14.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.953+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829001b6a0 con 0x7f8298102a40 2023-10-16T14:34:14.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.953+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829819f190 con 0x7f8298102a40 2023-10-16T14:34:14.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.954+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829810c760 con 0x7f8298102a40 2023-10-16T14:34:14.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.956+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f829001d470 con 0x7f8298102a40 2023-10-16T14:34:14.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.957+0000 7f828effd700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 0x7f82840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:14.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.957+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f82900ce1f0 con 0x7f8298102a40 2023-10-16T14:34:14.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.957+0000 7f829d99b700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 0x7f82840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:14.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.958+0000 7f829d99b700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 0x7f82840a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8298198200 tx=0x7f829400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:14.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:14.961+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8290097510 con 0x7f8298102a40 2023-10-16T14:34:15.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:14 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/857674684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:15.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:14 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/706739756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:14 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/857674684' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:15.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:14 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/706739756' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:15.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.161+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8298107e00 con 0x7f8298102a40 2023-10-16T14:34:15.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.161+0000 7f828effd700 1 -- 172.21.15.178:0/169000254 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f829009ab30 con 0x7f8298102a40 2023-10-16T14:34:15.165 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:34:15.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.163+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 msgr2=0x7f82840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.163+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 0x7f82840a3940 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8298198200 tx=0x7f829400b410 comp rx=0 tx=0).stop 2023-10-16T14:34:15.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 msgr2=0x7f829819e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8290009fd0 tx=0x7f829000bce0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 shutdown_connections 2023-10-16T14:34:15.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f82840a1480 0x7f82840a3940 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8298102a40 0x7f829819e2a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8298100ea0 0x7f8298196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 --2- 172.21.15.178:0/169000254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8298102080 0x7f8298197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 >> 172.21.15.178:0/169000254 conn(0x7f82980fc420 msgr2=0x7f82980fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:15.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 shutdown_connections 2023-10-16T14:34:15.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.164+0000 7f829fbff700 1 -- 172.21.15.178:0/169000254 wait complete. 2023-10-16T14:34:15.186 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:34:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 -- 172.21.15.178:0/2308941834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 msgr2=0x7ff61c102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 --2- 172.21.15.178:0/2308941834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c102500 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff6100098f0 tx=0x7ff610009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 -- 172.21.15.178:0/2308941834 shutdown_connections 2023-10-16T14:34:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 --2- 172.21.15.178:0/2308941834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 --2- 172.21.15.178:0/2308941834 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 0x7ff61c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 --2- 172.21.15.178:0/2308941834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c102500 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 -- 172.21.15.178:0/2308941834 >> 172.21.15.178:0/2308941834 conn(0x7ff61c0fc420 msgr2=0x7ff61c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:15.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 -- 172.21.15.178:0/2308941834 shutdown_connections 2023-10-16T14:34:15.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.384+0000 7ff6217ff700 1 -- 172.21.15.178:0/2308941834 wait complete. 2023-10-16T14:34:15.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.386+0000 7ff6217ff700 1 Processor -- start 2023-10-16T14:34:15.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.386+0000 7ff6217ff700 1 -- start start 2023-10-16T14:34:15.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.386+0000 7ff6217ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 0x7ff61c196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.386+0000 7ff6217ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.386+0000 7ff6217ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff6217ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e820 con 0x7ff61c102a40 2023-10-16T14:34:15.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff6217ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19e990 con 0x7ff61c100ea0 2023-10-16T14:34:15.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff6217ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c19eb50 con 0x7ff61c102080 2023-10-16T14:34:15.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 0x7ff61c196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40896/0 (socket says 172.21.15.178:40896) 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 -- 172.21.15.178:0/161235420 learned_addr learned my addr 172.21.15.178:0/161235420 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61a7fc700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 -- 172.21.15.178:0/161235420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 msgr2=0x7ff61c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.396 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 -- 172.21.15.178:0/161235420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 msgr2=0x7ff61c196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 0x7ff61c196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 -- 172.21.15.178:0/161235420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610009520 con 0x7ff61c102a40 2023-10-16T14:34:15.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.387+0000 7ff61b7fe700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c110 tx=0x7ff61400c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:15.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.388+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff614017070 con 0x7ff61c102a40 2023-10-16T14:34:15.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.388+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c19edd0 con 0x7ff61c102a40 2023-10-16T14:34:15.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.388+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61400e050 con 0x7ff61c102a40 2023-10-16T14:34:15.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.388+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140078f0 con 0x7ff61c102a40 2023-10-16T14:34:15.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.388+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c19f250 con 0x7ff61c102a40 2023-10-16T14:34:15.399 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.389+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c10c760 con 0x7ff61c102a40 2023-10-16T14:34:15.399 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.391+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff614007a50 con 0x7ff61c102a40 2023-10-16T14:34:15.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.392+0000 7ff603fff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 0x7ff6040a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.392+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff6140cca80 con 0x7ff61c102a40 2023-10-16T14:34:15.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.393+0000 7ff61affd700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 0x7ff6040a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.393+0000 7ff61affd700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 0x7ff6040a39c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff61c197b80 tx=0x7ff60c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:15.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.396+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff614095d70 con 0x7ff61c102a40 2023-10-16T14:34:15.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.598+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff61c107f10 con 0x7ff61c102a40 2023-10-16T14:34:15.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.598+0000 7ff603fff700 1 -- 172.21.15.178:0/161235420 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff614099390 con 0x7ff61c102a40 2023-10-16T14:34:15.603 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:15.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 msgr2=0x7ff6040a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 0x7ff6040a39c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff61c197b80 tx=0x7ff60c009260 comp rx=0 tx=0).stop 2023-10-16T14:34:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 msgr2=0x7ff61c19e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff61400c110 tx=0x7ff61400c420 comp rx=0 tx=0).stop 2023-10-16T14:34:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 shutdown_connections 2023-10-16T14:34:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff6040a1500 0x7ff6040a39c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff61c102a40 0x7ff61c19e250 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff61c100ea0 0x7ff61c196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 --2- 172.21.15.178:0/161235420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff61c102080 0x7ff61c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 >> 172.21.15.178:0/161235420 conn(0x7ff61c0fc420 msgr2=0x7ff61c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:15.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 shutdown_connections 2023-10-16T14:34:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.601+0000 7ff6217ff700 1 -- 172.21.15.178:0/161235420 wait complete. 2023-10-16T14:34:15.627 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:34:15.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 -- 172.21.15.178:0/2407743119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84102a20 msgr2=0x7fcb84109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 --2- 172.21.15.178:0/2407743119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84102a20 0x7fcb84109340 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c00b0a0 tx=0x7fcb7c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 -- 172.21.15.178:0/2407743119 shutdown_connections 2023-10-16T14:34:15.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 --2- 172.21.15.178:0/2407743119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84102a20 0x7fcb84109340 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 --2- 172.21.15.178:0/2407743119 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84100e80 0x7fcb841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 --2- 172.21.15.178:0/2407743119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102060 0x7fcb841024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 -- 172.21.15.178:0/2407743119 >> 172.21.15.178:0/2407743119 conn(0x7fcb840fc400 msgr2=0x7fcb840fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:15.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 -- 172.21.15.178:0/2407743119 shutdown_connections 2023-10-16T14:34:15.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.825+0000 7fcb899ff700 1 -- 172.21.15.178:0/2407743119 wait complete. 2023-10-16T14:34:15.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 Processor -- start 2023-10-16T14:34:15.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 -- start start 2023-10-16T14:34:15.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84100e80 0x7fcb84198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84102060 0x7fcb84199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a0aa0 con 0x7fcb84102a20 2023-10-16T14:34:15.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a0be0 con 0x7fcb84102060 2023-10-16T14:34:15.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.826+0000 7fcb899ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb841a0d70 con 0x7fcb84100e80 2023-10-16T14:34:15.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40924/0 (socket says 172.21.15.178:40924) 2023-10-16T14:34:15.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 -- 172.21.15.178:0/1848531153 learned_addr learned my addr 172.21.15.178:0/1848531153 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:15.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb82ffd700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84100e80 0x7fcb84198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb827fc700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84102060 0x7fcb84199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 -- 172.21.15.178:0/1848531153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84100e80 msgr2=0x7fcb84198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84100e80 0x7fcb84198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 -- 172.21.15.178:0/1848531153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84102060 msgr2=0x7fcb84199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:15.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84102060 0x7fcb84199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:15.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 -- 172.21.15.178:0/1848531153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c009d00 con 0x7fcb84102a20 2023-10-16T14:34:15.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb837fe700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009fd0 tx=0x7fcb7c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:15.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.827+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c00e040 con 0x7fcb84102a20 2023-10-16T14:34:15.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.828+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb7c003e10 con 0x7fcb84102a20 2023-10-16T14:34:15.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.828+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb841a0ff0 con 0x7fcb84102a20 2023-10-16T14:34:15.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.828+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c01b6a0 con 0x7fcb84102a20 2023-10-16T14:34:15.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.828+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb841a1410 con 0x7fcb84102a20 2023-10-16T14:34:15.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.829+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb84193530 con 0x7fcb84102a20 2023-10-16T14:34:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.831+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcb7c01d470 con 0x7fcb84102a20 2023-10-16T14:34:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.832+0000 7fcb889fd700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 0x7fcb6c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.832+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fcb7c0ce110 con 0x7fcb84102a20 2023-10-16T14:34:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.832+0000 7fcb82ffd700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 0x7fcb6c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.833+0000 7fcb82ffd700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 0x7fcb6c0a39c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb84101bd0 tx=0x7fcb7400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:15.836+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcb7c097430 con 0x7fcb84102a20 2023-10-16T14:34:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:15 smithi187 ceph-mon[135297]: pgmap v355: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:34:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:15 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3618417006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:15 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/169000254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:15.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:15 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/161235420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.036+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb8402d020 con 0x7fcb84102a20 2023-10-16T14:34:16.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.036+0000 7fcb889fd700 1 -- 172.21.15.178:0/1848531153 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb7c09aa50 con 0x7fcb84102a20 2023-10-16T14:34:16.040 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:34:16.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 msgr2=0x7fcb6c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 0x7fcb6c0a39c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcb84101bd0 tx=0x7fcb7400b410 comp rx=0 tx=0).stop 2023-10-16T14:34:16.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 msgr2=0x7fcb841a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009fd0 tx=0x7fcb7c00bce0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 shutdown_connections 2023-10-16T14:34:16.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcb6c0a1500 0x7fcb6c0a39c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb84102a20 0x7fcb841a04d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcb84100e80 0x7fcb84198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 --2- 172.21.15.178:0/1848531153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcb84102060 0x7fcb84199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 >> 172.21.15.178:0/1848531153 conn(0x7fcb840fc400 msgr2=0x7fcb841079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:16.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 shutdown_connections 2023-10-16T14:34:16.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.039+0000 7fcb899ff700 1 -- 172.21.15.178:0/1848531153 wait complete. 2023-10-16T14:34:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:15 smithi178 ceph-mon[98495]: pgmap v355: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:34:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:15 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3618417006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:15 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/169000254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:15 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/161235420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.063 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:34:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:15 smithi186 ceph-mon[134892]: pgmap v355: 129 pgs: 129 active+clean; 655 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:34:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:15 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3618417006' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:15 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/169000254' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:15 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/161235420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 -- 172.21.15.178:0/2916399902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c074e90 msgr2=0x7ff12c0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 --2- 172.21.15.178:0/2916399902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c074e90 0x7ff12c0732e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1200098f0 tx=0x7ff120009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 -- 172.21.15.178:0/2916399902 shutdown_connections 2023-10-16T14:34:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 --2- 172.21.15.178:0/2916399902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c108db0 0x7ff12c10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 --2- 172.21.15.178:0/2916399902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c074e90 0x7ff12c0732e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 --2- 172.21.15.178:0/2916399902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c0738b0 0x7ff12c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 -- 172.21.15.178:0/2916399902 >> 172.21.15.178:0/2916399902 conn(0x7ff12c0fc430 msgr2=0x7ff12c0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 -- 172.21.15.178:0/2916399902 shutdown_connections 2023-10-16T14:34:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.259+0000 7ff132bff700 1 -- 172.21.15.178:0/2916399902 wait complete. 2023-10-16T14:34:16.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 Processor -- start 2023-10-16T14:34:16.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 -- start start 2023-10-16T14:34:16.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c0738b0 0x7ff12c198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c074e90 0x7ff12c199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a09c0 con 0x7ff12c108db0 2023-10-16T14:34:16.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a0b00 con 0x7ff12c0738b0 2023-10-16T14:34:16.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.260+0000 7ff132bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1a0c90 con 0x7ff12c074e90 2023-10-16T14:34:16.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40950/0 (socket says 172.21.15.178:40950) 2023-10-16T14:34:16.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 -- 172.21.15.178:0/1240927808 learned_addr learned my addr 172.21.15.178:0/1240927808 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:16.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff12bfff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c074e90 0x7ff12c199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13099b700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c0738b0 0x7ff12c198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 -- 172.21.15.178:0/1240927808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c074e90 msgr2=0x7ff12c199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c074e90 0x7ff12c199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 -- 172.21.15.178:0/1240927808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c0738b0 msgr2=0x7ff12c198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c0738b0 0x7ff12c198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 -- 172.21.15.178:0/1240927808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff120009520 con 0x7ff12c108db0 2023-10-16T14:34:16.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff13119c700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff12400ef30 tx=0x7ff12400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:16.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.261+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124018070 con 0x7ff12c108db0 2023-10-16T14:34:16.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.262+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c1a0f10 con 0x7ff12c108db0 2023-10-16T14:34:16.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.262+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12400f040 con 0x7ff12c108db0 2023-10-16T14:34:16.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.262+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400ccf0 con 0x7ff12c108db0 2023-10-16T14:34:16.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.262+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1a1340 con 0x7ff12c108db0 2023-10-16T14:34:16.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.263+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c1074b0 con 0x7ff12c108db0 2023-10-16T14:34:16.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.265+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff124003a00 con 0x7ff12c108db0 2023-10-16T14:34:16.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.266+0000 7ff129ffb700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 0x7ff11c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.266+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff1240ccc00 con 0x7ff12c108db0 2023-10-16T14:34:16.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.266+0000 7ff13099b700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 0x7ff11c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.267+0000 7ff13099b700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 0x7ff11c0a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff120005e90 tx=0x7ff120019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:16.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.270+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff124095f20 con 0x7ff12c108db0 2023-10-16T14:34:16.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.469+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff12c02cf70 con 0x7ff12c108db0 2023-10-16T14:34:16.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.470+0000 7ff129ffb700 1 -- 172.21.15.178:0/1240927808 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff124099540 con 0x7ff12c108db0 2023-10-16T14:34:16.474 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:16.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 msgr2=0x7ff11c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 0x7ff11c0a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff120005e90 tx=0x7ff120019040 comp rx=0 tx=0).stop 2023-10-16T14:34:16.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 msgr2=0x7ff12c1a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff12400ef30 tx=0x7ff12400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 shutdown_connections 2023-10-16T14:34:16.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff11c0a1480 0x7ff11c0a3940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff12c108db0 0x7ff12c1a03f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff12c0738b0 0x7ff12c198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 --2- 172.21.15.178:0/1240927808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c074e90 0x7ff12c199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.472+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 >> 172.21.15.178:0/1240927808 conn(0x7ff12c0fc430 msgr2=0x7ff12c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:16.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.473+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 shutdown_connections 2023-10-16T14:34:16.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.473+0000 7ff132bff700 1 -- 172.21.15.178:0/1240927808 wait complete. 2023-10-16T14:34:16.495 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-10-16T14:34:16.496 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:34:16.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 -- 172.21.15.178:0/2683672680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20100ea0 msgr2=0x7f2b201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 --2- 172.21.15.178:0/2683672680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20100ea0 0x7f2b201012c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009990 tx=0x7f2b10009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 -- 172.21.15.178:0/2683672680 shutdown_connections 2023-10-16T14:34:16.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 --2- 172.21.15.178:0/2683672680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102a40 0x7f2b20109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 --2- 172.21.15.178:0/2683672680 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20100ea0 0x7f2b201012c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 --2- 172.21.15.178:0/2683672680 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20102080 0x7f2b20102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 -- 172.21.15.178:0/2683672680 >> 172.21.15.178:0/2683672680 conn(0x7f2b200fc420 msgr2=0x7f2b200fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:16.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 -- 172.21.15.178:0/2683672680 shutdown_connections 2023-10-16T14:34:16.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.692+0000 7f2b24bff700 1 -- 172.21.15.178:0/2683672680 wait complete. 2023-10-16T14:34:16.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 Processor -- start 2023-10-16T14:34:16.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 -- start start 2023-10-16T14:34:16.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 0x7f2b20198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102080 0x7f2b201992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b201a08f0 con 0x7f2b20102a40 2023-10-16T14:34:16.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b201a0a30 con 0x7f2b20100ea0 2023-10-16T14:34:16.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.693+0000 7f2b24bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b201a0bc0 con 0x7f2b20102080 2023-10-16T14:34:16.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40968/0 (socket says 172.21.15.178:40968) 2023-10-16T14:34:16.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 -- 172.21.15.178:0/1510386399 learned_addr learned my addr 172.21.15.178:0/1510386399 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:16.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1e59c700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 0x7f2b20198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1dd9b700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102080 0x7f2b201992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 -- 172.21.15.178:0/1510386399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102080 msgr2=0x7f2b201992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102080 0x7f2b201992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 -- 172.21.15.178:0/1510386399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 msgr2=0x7f2b20198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 0x7f2b20198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 -- 172.21.15.178:0/1510386399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009c40 con 0x7f2b20102a40 2023-10-16T14:34:16.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1e59c700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 0x7f2b20198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b1ed9d700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c370 tx=0x7f2b1800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18017070 con 0x7f2b20102a40 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.694+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1800f040 con 0x7f2b20102a40 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.695+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b100095f0 con 0x7f2b20102a40 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.695+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800e050 con 0x7f2b20102a40 2023-10-16T14:34:16.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.695+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b201a11c0 con 0x7f2b20102a40 2023-10-16T14:34:16.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.696+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2010c760 con 0x7f2b20102a40 2023-10-16T14:34:16.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.697+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2b1800cb60 con 0x7f2b20102a40 2023-10-16T14:34:16.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.699+0000 7f2b0f7fe700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 0x7f2b080a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:16.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.699+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2b180cca00 con 0x7f2b20102a40 2023-10-16T14:34:16.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.699+0000 7f2b1e59c700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 0x7f2b080a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:16.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.700+0000 7f2b1e59c700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 0x7f2b080a3a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b2e0 tx=0x7f2b10005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:16.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.703+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f2b18095c90 con 0x7f2b20102a40 2023-10-16T14:34:16.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:16 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1848531153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:16 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1240927808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:16 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1848531153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:16 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1240927808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:16.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.945+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b201a1350 con 0x7f2b20102a40 2023-10-16T14:34:16.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.946+0000 7f2b0f7fe700 1 -- 172.21.15.178:0/1510386399 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7f2b180992b0 con 0x7f2b20102a40 2023-10-16T14:34:16.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:16.950 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:34:16.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 msgr2=0x7f2b080a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 0x7f2b080a3a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b2e0 tx=0x7f2b10005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 msgr2=0x7f2b201a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:16.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800c370 tx=0x7f2b1800c680 comp rx=0 tx=0).stop 2023-10-16T14:34:16.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 shutdown_connections 2023-10-16T14:34:16.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f2b080a15a0 0x7f2b080a3a60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b20102a40 0x7f2b201a0320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2b20100ea0 0x7f2b20198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 --2- 172.21.15.178:0/1510386399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b20102080 0x7f2b201992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:16.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 >> 172.21.15.178:0/1510386399 conn(0x7f2b200fc420 msgr2=0x7f2b200fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:16.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 shutdown_connections 2023-10-16T14:34:16.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:16.949+0000 7f2b24bff700 1 -- 172.21.15.178:0/1510386399 wait complete. 2023-10-16T14:34:16.955 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:34:16.972 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-10-16T14:34:16.972 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-10-16T14:34:16.972 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:34:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:16 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1848531153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:17.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:16 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1240927808' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:17.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.169+0000 7fe116bff700 1 -- 172.21.15.178:0/1319603724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 msgr2=0x7fe1101012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.169+0000 7fe116bff700 1 --2- 172.21.15.178:0/1319603724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe1101012c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe104009990 tx=0x7fe104009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 -- 172.21.15.178:0/1319603724 shutdown_connections 2023-10-16T14:34:17.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 --2- 172.21.15.178:0/1319603724 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe110109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 --2- 172.21.15.178:0/1319603724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe1101012c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 --2- 172.21.15.178:0/1319603724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 0x7fe110102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 -- 172.21.15.178:0/1319603724 >> 172.21.15.178:0/1319603724 conn(0x7fe1100fc420 msgr2=0x7fe1100fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:17.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 -- 172.21.15.178:0/1319603724 shutdown_connections 2023-10-16T14:34:17.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.170+0000 7fe116bff700 1 -- 172.21.15.178:0/1319603724 wait complete. 2023-10-16T14:34:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.171+0000 7fe116bff700 1 Processor -- start 2023-10-16T14:34:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.171+0000 7fe116bff700 1 -- start start 2023-10-16T14:34:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 0x7fe1101970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe11019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019e6f0 con 0x7fe110100ea0 2023-10-16T14:34:17.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019e830 con 0x7fe110102a40 2023-10-16T14:34:17.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe116bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11019e9c0 con 0x7fe110102080 2023-10-16T14:34:17.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40984/0 (socket says 172.21.15.178:40984) 2023-10-16T14:34:17.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 -- 172.21.15.178:0/1618232978 learned_addr learned my addr 172.21.15.178:0/1618232978 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:17.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe10ffff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 0x7fe1101970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11519c700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe11019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 -- 172.21.15.178:0/1618232978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 msgr2=0x7fe1101970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 0x7fe1101970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 -- 172.21.15.178:0/1618232978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 msgr2=0x7fe11019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe11019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11499b700 1 -- 172.21.15.178:0/1618232978 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040095f0 con 0x7fe110100ea0 2023-10-16T14:34:17.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.172+0000 7fe11519c700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe11019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:17.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe11499b700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe104004710 tx=0x7fe104009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:17.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10401d070 con 0x7fe110100ea0 2023-10-16T14:34:17.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10400fbc0 con 0x7fe110100ea0 2023-10-16T14:34:17.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104017650 con 0x7fe110100ea0 2023-10-16T14:34:17.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11019ec40 con 0x7fe110100ea0 2023-10-16T14:34:17.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.173+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11019f010 con 0x7fe110100ea0 2023-10-16T14:34:17.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.174+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11010c760 con 0x7fe110100ea0 2023-10-16T14:34:17.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.176+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe104021410 con 0x7fe110100ea0 2023-10-16T14:34:17.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.177+0000 7fe10dffb700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 0x7fe1000a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.178+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe1040ce6b0 con 0x7fe110100ea0 2023-10-16T14:34:17.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.178+0000 7fe10ffff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 0x7fe1000a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.178+0000 7fe10ffff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 0x7fe1000a39e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe110100bc0 tx=0x7fe0fc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:17.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.181+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fe1040979d0 con 0x7fe110100ea0 2023-10-16T14:34:17.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.380+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1100612f0 con 0x7fe1000a1520 2023-10-16T14:34:17.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.394+0000 7fe10dffb700 1 -- 172.21.15.178:0/1618232978 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427827 (secure 0 0 0) 0x7fe1100612f0 con 0x7fe1000a1520 2023-10-16T14:34:17.404 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:17.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 msgr2=0x7fe1000a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 0x7fe1000a39e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe110100bc0 tx=0x7fe0fc009260 comp rx=0 tx=0).stop 2023-10-16T14:34:17.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 msgr2=0x7fe110196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe104004710 tx=0x7fe104009580 comp rx=0 tx=0).stop 2023-10-16T14:34:17.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 shutdown_connections 2023-10-16T14:34:17.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe1000a1520 0x7fe1000a39e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe110102a40 0x7fe11019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe110100ea0 0x7fe110196b70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 --2- 172.21.15.178:0/1618232978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe110102080 0x7fe1101970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.399+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 >> 172.21.15.178:0/1618232978 conn(0x7fe1100fc420 msgr2=0x7fe1100fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:17.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.400+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 shutdown_connections 2023-10-16T14:34:17.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.400+0000 7fe116bff700 1 -- 172.21.15.178:0/1618232978 wait complete. 2023-10-16T14:34:17.410 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:34:17.423 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2023-10-16T14:34:16.630146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672714,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":904,"num_read_kb":584,"num_write":361,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338336,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603488,"statfs":{"total":1151940427776,"available":1151593971712,"internally_reserved":0,"allocated":14483456,"data_stored":8068243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2484,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":39,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637229+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:36.637229+0000","last_peered":"2023-10-16T14:33:36.637229+0000","last_clean":"2023-10-16T14:33:36.637229+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637229+0000","last_undegraded":"2023-10-16T14:33:36.637229+0000","last_fullsized":"2023-10-16T14:33:36.637229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641049+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:36.641049+0000","last_peered":"2023-10-16T14:33:36.641049+0000","last_clean":"2023-10-16T14:33:36.641049+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:36.641049+0000","last_undegraded":"2023-10-16T14:33:36.641049+0000","last_fullsized":"2023-10-16T14:33:36.641049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636867+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:36.636867+0000","last_peered":"2023-10-16T14:33:36.636867+0000","last_clean":"2023-10-16T14:33:36.636867+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:36.636867+0000","last_undegraded":"2023-10-16T14:33:36.636867+0000","last_fullsized":"2023-10-16T14:33:36.636867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638334+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:36.638334+0000","last_peered":"2023-10-16T14:33:36.638334+0000","last_clean":"2023-10-16T14:33:36.638334+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:36.638334+0000","last_undegraded":"2023-10-16T14:33:36.638334+0000","last_fullsized":"2023-10-16T14:33:36.638334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636004+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:36.636004+0000","last_peered":"2023-10-16T14:33:36.636004+0000","last_clean":"2023-10-16T14:33:36.636004+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:36.636004+0000","last_undegraded":"2023-10-16T14:33:36.636004+0000","last_fullsized":"2023-10-16T14:33:36.636004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642338+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:33:36.642338+0000","last_peered":"2023-10-16T14:33:36.642338+0000","last_clean":"2023-10-16T14:33:36.642338+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:33:36.642338+0000","last_undegraded":"2023-10-16T14:33:36.642338+0000","last_fullsized":"2023-10-16T14:33:36.642338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627444+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:36.627444+0000","last_peered":"2023-10-16T14:33:36.627444+0000","last_clean":"2023-10-16T14:33:36.627444+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:36.627444+0000","last_undegraded":"2023-10-16T14:33:36.627444+0000","last_fullsized":"2023-10-16T14:33:36.627444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639455+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:36.639455+0000","last_peered":"2023-10-16T14:33:36.639455+0000","last_clean":"2023-10-16T14:33:36.639455+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:36.639455+0000","last_undegraded":"2023-10-16T14:33:36.639455+0000","last_fullsized":"2023-10-16T14:33:36.639455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626801+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:33:36.626801+0000","last_peered":"2023-10-16T14:33:36.626801+0000","last_clean":"2023-10-16T14:33:36.626801+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:33:36.626801+0000","last_undegraded":"2023-10-16T14:33:36.626801+0000","last_fullsized":"2023-10-16T14:33:36.626801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642427+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:33:36.642427+0000","last_peered":"2023-10-16T14:33:36.642427+0000","last_clean":"2023-10-16T14:33:36.642427+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:33:36.642427+0000","last_undegraded":"2023-10-16T14:33:36.642427+0000","last_fullsized":"2023-10-16T14:33:36.642427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637246+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:36.637246+0000","last_peered":"2023-10-16T14:33:36.637246+0000","last_clean":"2023-10-16T14:33:36.637246+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:36.637246+0000","last_undegraded":"2023-10-16T14:33:36.637246+0000","last_fullsized":"2023-10-16T14:33:36.637246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639803+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:36.639803+0000","last_peered":"2023-10-16T14:33:36.639803+0000","last_clean":"2023-10-16T14:33:36.639803+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:36.639803+0000","last_undegraded":"2023-10-16T14:33:36.639803+0000","last_fullsized":"2023-10-16T14:33:36.639803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641688+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:36.641688+0000","last_peered":"2023-10-16T14:33:36.641688+0000","last_clean":"2023-10-16T14:33:36.641688+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:36.641688+0000","last_undegraded":"2023-10-16T14:33:36.641688+0000","last_fullsized":"2023-10-16T14:33:36.641688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641600+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:36.641600+0000","last_peered":"2023-10-16T14:33:36.641600+0000","last_clean":"2023-10-16T14:33:36.641600+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:36.641600+0000","last_undegraded":"2023-10-16T14:33:36.641600+0000","last_fullsized":"2023-10-16T14:33:36.641600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641196+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:36.641196+0000","last_peered":"2023-10-16T14:33:36.641196+0000","last_clean":"2023-10-16T14:33:36.641196+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:36.641196+0000","last_undegraded":"2023-10-16T14:33:36.641196+0000","last_fullsized":"2023-10-16T14:33:36.641196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637421+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:36.637421+0000","last_peered":"2023-10-16T14:33:36.637421+0000","last_clean":"2023-10-16T14:33:36.637421+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:36.637421+0000","last_undegraded":"2023-10-16T14:33:36.637421+0000","last_fullsized":"2023-10-16T14:33:36.637421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637900+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:36.637900+0000","last_peered":"2023-10-16T14:33:36.637900+0000","last_clean":"2023-10-16T14:33:36.637900+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:36.637900+0000","last_undegraded":"2023-10-16T14:33:36.637900+0000","last_fullsized":"2023-10-16T14:33:36.637900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621257+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:36.621257+0000","last_peered":"2023-10-16T14:33:36.621257+0000","last_clean":"2023-10-16T14:33:36.621257+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:36.621257+0000","last_undegraded":"2023-10-16T14:33:36.621257+0000","last_fullsized":"2023-10-16T14:33:36.621257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290036+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:08.290036+0000","last_peered":"2023-10-16T14:33:08.290036+0000","last_clean":"2023-10-16T14:33:08.290036+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:08.290036+0000","last_undegraded":"2023-10-16T14:33:08.290036+0000","last_fullsized":"2023-10-16T14:33:08.290036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044284+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:08.044284+0000","last_peered":"2023-10-16T14:33:08.044284+0000","last_clean":"2023-10-16T14:33:08.044284+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:08.044284+0000","last_undegraded":"2023-10-16T14:33:08.044284+0000","last_fullsized":"2023-10-16T14:33:08.044284+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638964+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:36.638964+0000","last_peered":"2023-10-16T14:33:36.638964+0000","last_clean":"2023-10-16T14:33:36.638964+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:36.638964+0000","last_undegraded":"2023-10-16T14:33:36.638964+0000","last_fullsized":"2023-10-16T14:33:36.638964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.039726+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:08.039726+0000","last_peered":"2023-10-16T14:33:08.039726+0000","last_clean":"2023-10-16T14:33:08.039726+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:08.039726+0000","last_undegraded":"2023-10-16T14:33:08.039726+0000","last_fullsized":"2023-10-16T14:33:08.039726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038574+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:08.038574+0000","last_peered":"2023-10-16T14:33:08.038574+0000","last_clean":"2023-10-16T14:33:08.038574+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:08.038574+0000","last_undegraded":"2023-10-16T14:33:08.038574+0000","last_fullsized":"2023-10-16T14:33:08.038574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636879+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:36.636879+0000","last_peered":"2023-10-16T14:33:36.636879+0000","last_clean":"2023-10-16T14:33:36.636879+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:36.636879+0000","last_undegraded":"2023-10-16T14:33:36.636879+0000","last_fullsized":"2023-10-16T14:33:36.636879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040318+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:08.040318+0000","last_peered":"2023-10-16T14:33:08.040318+0000","last_clean":"2023-10-16T14:33:08.040318+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:08.040318+0000","last_undegraded":"2023-10-16T14:33:08.040318+0000","last_fullsized":"2023-10-16T14:33:08.040318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033698+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:08.033698+0000","last_peered":"2023-10-16T14:33:08.033698+0000","last_clean":"2023-10-16T14:33:08.033698+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:08.033698+0000","last_undegraded":"2023-10-16T14:33:08.033698+0000","last_fullsized":"2023-10-16T14:33:08.033698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638148+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:33:36.638148+0000","last_peered":"2023-10-16T14:33:36.638148+0000","last_clean":"2023-10-16T14:33:36.638148+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:33:36.638148+0000","last_undegraded":"2023-10-16T14:33:36.638148+0000","last_fullsized":"2023-10-16T14:33:36.638148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.418372+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.418372+0000","last_peered":"2023-10-16T14:33:08.418372+0000","last_clean":"2023-10-16T14:33:08.418372+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.418372+0000","last_undegraded":"2023-10-16T14:33:08.418372+0000","last_fullsized":"2023-10-16T14:33:08.418372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635989+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:36.635989+0000","last_peered":"2023-10-16T14:33:36.635989+0000","last_clean":"2023-10-16T14:33:36.635989+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:36.635989+0000","last_undegraded":"2023-10-16T14:33:36.635989+0000","last_fullsized":"2023-10-16T14:33:36.635989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290088+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:08.290088+0000","last_peered":"2023-10-16T14:33:08.290088+0000","last_clean":"2023-10-16T14:33:08.290088+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:08.290088+0000","last_undegraded":"2023-10-16T14:33:08.290088+0000","last_fullsized":"2023-10-16T14:33:08.290088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'12","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:13.904131+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:34:13.904131+0000","last_peered":"2023-10-16T14:34:13.904131+0000","last_clean":"2023-10-16T14:34:13.904131+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:34:13.904131+0000","last_undegraded":"2023-10-16T14:34:13.904131+0000","last_fullsized":"2023-10-16T14:34:13.904131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":17,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.618324+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:36.618324+0000","last_peered":"2023-10-16T14:33:36.618324+0000","last_clean":"2023-10-16T14:33:36.618324+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:36.618324+0000","last_undegraded":"2023-10-16T14:33:36.618324+0000","last_fullsized":"2023-10-16T14:33:36.618324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290297+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:08.290297+0000","last_peered":"2023-10-16T14:33:08.290297+0000","last_clean":"2023-10-16T14:33:08.290297+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:08.290297+0000","last_undegraded":"2023-10-16T14:33:08.290297+0000","last_fullsized":"2023-10-16T14:33:08.290297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037660+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:33:08.037660+0000","last_peered":"2023-10-16T14:33:08.037660+0000","last_clean":"2023-10-16T14:33:08.037660+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:33:08.037660+0000","last_undegraded":"2023-10-16T14:33:08.037660+0000","last_fullsized":"2023-10-16T14:33:08.037660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641111+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:36.641111+0000","last_peered":"2023-10-16T14:33:36.641111+0000","last_clean":"2023-10-16T14:33:36.641111+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:36.641111+0000","last_undegraded":"2023-10-16T14:33:36.641111+0000","last_fullsized":"2023-10-16T14:33:36.641111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044215+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:08.044215+0000","last_peered":"2023-10-16T14:33:08.044215+0000","last_clean":"2023-10-16T14:33:08.044215+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:08.044215+0000","last_undegraded":"2023-10-16T14:33:08.044215+0000","last_fullsized":"2023-10-16T14:33:08.044215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416188+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:08.416188+0000","last_peered":"2023-10-16T14:33:08.416188+0000","last_clean":"2023-10-16T14:33:08.416188+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:08.416188+0000","last_undegraded":"2023-10-16T14:33:08.416188+0000","last_fullsized":"2023-10-16T14:33:08.416188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638397+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:36.638397+0000","last_peered":"2023-10-16T14:33:36.638397+0000","last_clean":"2023-10-16T14:33:36.638397+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:36.638397+0000","last_undegraded":"2023-10-16T14:33:36.638397+0000","last_fullsized":"2023-10-16T14:33:36.638397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052948+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:08.052948+0000","last_peered":"2023-10-16T14:33:08.052948+0000","last_clean":"2023-10-16T14:33:08.052948+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:08.052948+0000","last_undegraded":"2023-10-16T14:33:08.052948+0000","last_fullsized":"2023-10-16T14:33:08.052948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044478+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:08.044478+0000","last_peered":"2023-10-16T14:33:08.044478+0000","last_clean":"2023-10-16T14:33:08.044478+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:08.044478+0000","last_undegraded":"2023-10-16T14:33:08.044478+0000","last_fullsized":"2023-10-16T14:33:08.044478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640406+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:36.640406+0000","last_peered":"2023-10-16T14:33:36.640406+0000","last_clean":"2023-10-16T14:33:36.640406+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:36.640406+0000","last_undegraded":"2023-10-16T14:33:36.640406+0000","last_fullsized":"2023-10-16T14:33:36.640406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038135+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:08.038135+0000","last_peered":"2023-10-16T14:33:08.038135+0000","last_clean":"2023-10-16T14:33:08.038135+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:08.038135+0000","last_undegraded":"2023-10-16T14:33:08.038135+0000","last_fullsized":"2023-10-16T14:33:08.038135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.542711+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:34:11.542711+0000","last_peered":"2023-10-16T14:34:11.542711+0000","last_clean":"2023-10-16T14:34:11.542711+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:34:11.542711+0000","last_undegraded":"2023-10-16T14:34:11.542711+0000","last_fullsized":"2023-10-16T14:34:11.542711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638618+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:33:36.638618+0000","last_peered":"2023-10-16T14:33:36.638618+0000","last_clean":"2023-10-16T14:33:36.638618+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:33:36.638618+0000","last_undegraded":"2023-10-16T14:33:36.638618+0000","last_fullsized":"2023-10-16T14:33:36.638618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043503+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:08.043503+0000","last_peered":"2023-10-16T14:33:08.043503+0000","last_clean":"2023-10-16T14:33:08.043503+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:08.043503+0000","last_undegraded":"2023-10-16T14:33:08.043503+0000","last_fullsized":"2023-10-16T14:33:08.043503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041325+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:08.041325+0000","last_peered":"2023-10-16T14:33:08.041325+0000","last_clean":"2023-10-16T14:33:08.041325+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:08.041325+0000","last_undegraded":"2023-10-16T14:33:08.041325+0000","last_fullsized":"2023-10-16T14:33:08.041325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.643248+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:36.643248+0000","last_peered":"2023-10-16T14:33:36.643248+0000","last_clean":"2023-10-16T14:33:36.643248+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:36.643248+0000","last_undegraded":"2023-10-16T14:33:36.643248+0000","last_fullsized":"2023-10-16T14:33:36.643248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040462+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:08.040462+0000","last_peered":"2023-10-16T14:33:08.040462+0000","last_clean":"2023-10-16T14:33:08.040462+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:08.040462+0000","last_undegraded":"2023-10-16T14:33:08.040462+0000","last_fullsized":"2023-10-16T14:33:08.040462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043844+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:08.043844+0000","last_peered":"2023-10-16T14:33:08.043844+0000","last_clean":"2023-10-16T14:33:08.043844+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:08.043844+0000","last_undegraded":"2023-10-16T14:33:08.043844+0000","last_fullsized":"2023-10-16T14:33:08.043844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641090+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:33:36.641090+0000","last_peered":"2023-10-16T14:33:36.641090+0000","last_clean":"2023-10-16T14:33:36.641090+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:33:36.641090+0000","last_undegraded":"2023-10-16T14:33:36.641090+0000","last_fullsized":"2023-10-16T14:33:36.641090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.045902+0000","last_change":"2023-10-16T14:33:08.045902+0000","last_active":"2023-10-16T14:33:08.045902+0000","last_peered":"2023-10-16T14:33:08.045902+0000","last_clean":"2023-10-16T14:33:08.045902+0000","last_became_active":"2023-10-16T14:33:08.045481+0000","last_became_peered":"2023-10-16T14:33:08.045481+0000","last_unstale":"2023-10-16T14:33:08.045902+0000","last_undegraded":"2023-10-16T14:33:08.045902+0000","last_fullsized":"2023-10-16T14:33:08.045902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:18.091442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":314,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.899607+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:34:11.899607+0000","last_peered":"2023-10-16T14:34:11.899607+0000","last_clean":"2023-10-16T14:34:11.899607+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:34:11.899607+0000","last_undegraded":"2023-10-16T14:34:11.899607+0000","last_fullsized":"2023-10-16T14:34:11.899607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65477,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":192,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.629796+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:36.629796+0000","last_peered":"2023-10-16T14:33:36.629796+0000","last_clean":"2023-10-16T14:33:36.629796+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:36.629796+0000","last_undegraded":"2023-10-16T14:33:36.629796+0000","last_fullsized":"2023-10-16T14:33:36.629796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044920+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:08.044920+0000","last_peered":"2023-10-16T14:33:08.044920+0000","last_clean":"2023-10-16T14:33:08.044920+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:08.044920+0000","last_undegraded":"2023-10-16T14:33:08.044920+0000","last_fullsized":"2023-10-16T14:33:08.044920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290160+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:08.290160+0000","last_peered":"2023-10-16T14:33:08.290160+0000","last_clean":"2023-10-16T14:33:08.290160+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:08.290160+0000","last_undegraded":"2023-10-16T14:33:08.290160+0000","last_fullsized":"2023-10-16T14:33:08.290160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.632140+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:36.632140+0000","last_peered":"2023-10-16T14:33:36.632140+0000","last_clean":"2023-10-16T14:33:36.632140+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:36.632140+0000","last_undegraded":"2023-10-16T14:33:36.632140+0000","last_fullsized":"2023-10-16T14:33:36.632140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044018+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:08.044018+0000","last_peered":"2023-10-16T14:33:08.044018+0000","last_clean":"2023-10-16T14:33:08.044018+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:08.044018+0000","last_undegraded":"2023-10-16T14:33:08.044018+0000","last_fullsized":"2023-10-16T14:33:08.044018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044199+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:08.044199+0000","last_peered":"2023-10-16T14:33:08.044199+0000","last_clean":"2023-10-16T14:33:08.044199+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:08.044199+0000","last_undegraded":"2023-10-16T14:33:08.044199+0000","last_fullsized":"2023-10-16T14:33:08.044199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642241+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:36.642241+0000","last_peered":"2023-10-16T14:33:36.642241+0000","last_clean":"2023-10-16T14:33:36.642241+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:36.642241+0000","last_undegraded":"2023-10-16T14:33:36.642241+0000","last_fullsized":"2023-10-16T14:33:36.642241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043967+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:08.043967+0000","last_peered":"2023-10-16T14:33:08.043967+0000","last_clean":"2023-10-16T14:33:08.043967+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:08.043967+0000","last_undegraded":"2023-10-16T14:33:08.043967+0000","last_fullsized":"2023-10-16T14:33:08.043967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:12.900730+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:34:12.900730+0000","last_peered":"2023-10-16T14:34:12.900730+0000","last_clean":"2023-10-16T14:34:12.900730+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:34:12.900730+0000","last_undegraded":"2023-10-16T14:34:12.900730+0000","last_fullsized":"2023-10-16T14:34:12.900730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637695+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:36.637695+0000","last_peered":"2023-10-16T14:33:36.637695+0000","last_clean":"2023-10-16T14:33:36.637695+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:36.637695+0000","last_undegraded":"2023-10-16T14:33:36.637695+0000","last_fullsized":"2023-10-16T14:33:36.637695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416657+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:08.416657+0000","last_peered":"2023-10-16T14:33:08.416657+0000","last_clean":"2023-10-16T14:33:08.416657+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:08.416657+0000","last_undegraded":"2023-10-16T14:33:08.416657+0000","last_fullsized":"2023-10-16T14:33:08.416657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033313+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:08.033313+0000","last_peered":"2023-10-16T14:33:08.033313+0000","last_clean":"2023-10-16T14:33:08.033313+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:08.033313+0000","last_undegraded":"2023-10-16T14:33:08.033313+0000","last_fullsized":"2023-10-16T14:33:08.033313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640793+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:36.640793+0000","last_peered":"2023-10-16T14:33:36.640793+0000","last_clean":"2023-10-16T14:33:36.640793+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:36.640793+0000","last_undegraded":"2023-10-16T14:33:36.640793+0000","last_fullsized":"2023-10-16T14:33:36.640793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043832+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:08.043832+0000","last_peered":"2023-10-16T14:33:08.043832+0000","last_clean":"2023-10-16T14:33:08.043832+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:08.043832+0000","last_undegraded":"2023-10-16T14:33:08.043832+0000","last_fullsized":"2023-10-16T14:33:08.043832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033787+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:08.033787+0000","last_peered":"2023-10-16T14:33:08.033787+0000","last_clean":"2023-10-16T14:33:08.033787+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:08.033787+0000","last_undegraded":"2023-10-16T14:33:08.033787+0000","last_fullsized":"2023-10-16T14:33:08.033787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640908+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:36.640908+0000","last_peered":"2023-10-16T14:33:36.640908+0000","last_clean":"2023-10-16T14:33:36.640908+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:36.640908+0000","last_undegraded":"2023-10-16T14:33:36.640908+0000","last_fullsized":"2023-10-16T14:33:36.640908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033930+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:08.033930+0000","last_peered":"2023-10-16T14:33:08.033930+0000","last_clean":"2023-10-16T14:33:08.033930+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:08.033930+0000","last_undegraded":"2023-10-16T14:33:08.033930+0000","last_fullsized":"2023-10-16T14:33:08.033930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043926+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:08.043926+0000","last_peered":"2023-10-16T14:33:08.043926+0000","last_clean":"2023-10-16T14:33:08.043926+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:08.043926+0000","last_undegraded":"2023-10-16T14:33:08.043926+0000","last_fullsized":"2023-10-16T14:33:08.043926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638858+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:33:36.638858+0000","last_peered":"2023-10-16T14:33:36.638858+0000","last_clean":"2023-10-16T14:33:36.638858+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:33:36.638858+0000","last_undegraded":"2023-10-16T14:33:36.638858+0000","last_fullsized":"2023-10-16T14:33:36.638858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416348+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:08.416348+0000","last_peered":"2023-10-16T14:33:08.416348+0000","last_clean":"2023-10-16T14:33:08.416348+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:08.416348+0000","last_undegraded":"2023-10-16T14:33:08.416348+0000","last_fullsized":"2023-10-16T14:33:08.416348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041272+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:08.041272+0000","last_peered":"2023-10-16T14:33:08.041272+0000","last_clean":"2023-10-16T14:33:08.041272+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:08.041272+0000","last_undegraded":"2023-10-16T14:33:08.041272+0000","last_fullsized":"2023-10-16T14:33:08.041272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637932+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:33:36.637932+0000","last_peered":"2023-10-16T14:33:36.637932+0000","last_clean":"2023-10-16T14:33:36.637932+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:33:36.637932+0000","last_undegraded":"2023-10-16T14:33:36.637932+0000","last_fullsized":"2023-10-16T14:33:36.637932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416540+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:08.416540+0000","last_peered":"2023-10-16T14:33:08.416540+0000","last_clean":"2023-10-16T14:33:08.416540+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:08.416540+0000","last_undegraded":"2023-10-16T14:33:08.416540+0000","last_fullsized":"2023-10-16T14:33:08.416540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627499+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:36.627499+0000","last_peered":"2023-10-16T14:33:36.627499+0000","last_clean":"2023-10-16T14:33:36.627499+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:36.627499+0000","last_undegraded":"2023-10-16T14:33:36.627499+0000","last_fullsized":"2023-10-16T14:33:36.627499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:14.903045+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:34:14.903045+0000","last_peered":"2023-10-16T14:34:14.903045+0000","last_clean":"2023-10-16T14:34:14.903045+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:34:14.903045+0000","last_undegraded":"2023-10-16T14:34:14.903045+0000","last_fullsized":"2023-10-16T14:34:14.903045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050766+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050766+0000","last_peered":"2023-10-16T14:33:08.050766+0000","last_clean":"2023-10-16T14:33:08.050766+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050766+0000","last_undegraded":"2023-10-16T14:33:08.050766+0000","last_fullsized":"2023-10-16T14:33:08.050766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040458+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:08.040458+0000","last_peered":"2023-10-16T14:33:08.040458+0000","last_clean":"2023-10-16T14:33:08.040458+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:08.040458+0000","last_undegraded":"2023-10-16T14:33:08.040458+0000","last_fullsized":"2023-10-16T14:33:08.040458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040909+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:08.040909+0000","last_peered":"2023-10-16T14:33:08.040909+0000","last_clean":"2023-10-16T14:33:08.040909+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:08.040909+0000","last_undegraded":"2023-10-16T14:33:08.040909+0000","last_fullsized":"2023-10-16T14:33:08.040909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044956+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:08.044956+0000","last_peered":"2023-10-16T14:33:08.044956+0000","last_clean":"2023-10-16T14:33:08.044956+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:08.044956+0000","last_undegraded":"2023-10-16T14:33:08.044956+0000","last_fullsized":"2023-10-16T14:33:08.044956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627304+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:36.627304+0000","last_peered":"2023-10-16T14:33:36.627304+0000","last_clean":"2023-10-16T14:33:36.627304+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:36.627304+0000","last_undegraded":"2023-10-16T14:33:36.627304+0000","last_fullsized":"2023-10-16T14:33:36.627304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040976+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:08.040976+0000","last_peered":"2023-10-16T14:33:08.040976+0000","last_clean":"2023-10-16T14:33:08.040976+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:08.040976+0000","last_undegraded":"2023-10-16T14:33:08.040976+0000","last_fullsized":"2023-10-16T14:33:08.040976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043860+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:08.043860+0000","last_peered":"2023-10-16T14:33:08.043860+0000","last_clean":"2023-10-16T14:33:08.043860+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:08.043860+0000","last_undegraded":"2023-10-16T14:33:08.043860+0000","last_fullsized":"2023-10-16T14:33:08.043860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033618+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:08.033618+0000","last_peered":"2023-10-16T14:33:08.033618+0000","last_clean":"2023-10-16T14:33:08.033618+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:08.033618+0000","last_undegraded":"2023-10-16T14:33:08.033618+0000","last_fullsized":"2023-10-16T14:33:08.033618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626193+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:36.626193+0000","last_peered":"2023-10-16T14:33:36.626193+0000","last_clean":"2023-10-16T14:33:36.626193+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:36.626193+0000","last_undegraded":"2023-10-16T14:33:36.626193+0000","last_fullsized":"2023-10-16T14:33:36.626193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037942+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:08.037942+0000","last_peered":"2023-10-16T14:33:08.037942+0000","last_clean":"2023-10-16T14:33:08.037942+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:08.037942+0000","last_undegraded":"2023-10-16T14:33:08.037942+0000","last_fullsized":"2023-10-16T14:33:08.037942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:40.913037+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:40.913037+0000","last_peered":"2023-10-16T14:33:40.913037+0000","last_clean":"2023-10-16T14:33:40.913037+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:40.913037+0000","last_undegraded":"2023-10-16T14:33:40.913037+0000","last_fullsized":"2023-10-16T14:33:40.913037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638429+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:36.638429+0000","last_peered":"2023-10-16T14:33:36.638429+0000","last_clean":"2023-10-16T14:33:36.638429+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:36.638429+0000","last_undegraded":"2023-10-16T14:33:36.638429+0000","last_fullsized":"2023-10-16T14:33:36.638429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.289756+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:08.289756+0000","last_peered":"2023-10-16T14:33:08.289756+0000","last_clean":"2023-10-16T14:33:08.289756+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:08.289756+0000","last_undegraded":"2023-10-16T14:33:08.289756+0000","last_fullsized":"2023-10-16T14:33:08.289756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041225+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:08.041225+0000","last_peered":"2023-10-16T14:33:08.041225+0000","last_clean":"2023-10-16T14:33:08.041225+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:08.041225+0000","last_undegraded":"2023-10-16T14:33:08.041225+0000","last_fullsized":"2023-10-16T14:33:08.041225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641382+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:36.641382+0000","last_peered":"2023-10-16T14:33:36.641382+0000","last_clean":"2023-10-16T14:33:36.641382+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:36.641382+0000","last_undegraded":"2023-10-16T14:33:36.641382+0000","last_fullsized":"2023-10-16T14:33:36.641382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040831+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:08.040831+0000","last_peered":"2023-10-16T14:33:08.040831+0000","last_clean":"2023-10-16T14:33:08.040831+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:08.040831+0000","last_undegraded":"2023-10-16T14:33:08.040831+0000","last_fullsized":"2023-10-16T14:33:08.040831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454545+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:33:08.454545+0000","last_peered":"2023-10-16T14:33:08.454545+0000","last_clean":"2023-10-16T14:33:08.454545+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:33:08.454545+0000","last_undegraded":"2023-10-16T14:33:08.454545+0000","last_fullsized":"2023-10-16T14:33:08.454545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.622286+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:36.622286+0000","last_peered":"2023-10-16T14:33:36.622286+0000","last_clean":"2023-10-16T14:33:36.622286+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:36.622286+0000","last_undegraded":"2023-10-16T14:33:36.622286+0000","last_fullsized":"2023-10-16T14:33:36.622286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044222+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:08.044222+0000","last_peered":"2023-10-16T14:33:08.044222+0000","last_clean":"2023-10-16T14:33:08.044222+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:08.044222+0000","last_undegraded":"2023-10-16T14:33:08.044222+0000","last_fullsized":"2023-10-16T14:33:08.044222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038020+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:08.038020+0000","last_peered":"2023-10-16T14:33:08.038020+0000","last_clean":"2023-10-16T14:33:08.038020+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:08.038020+0000","last_undegraded":"2023-10-16T14:33:08.038020+0000","last_fullsized":"2023-10-16T14:33:08.038020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637193+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:36.637193+0000","last_peered":"2023-10-16T14:33:36.637193+0000","last_clean":"2023-10-16T14:33:36.637193+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637193+0000","last_undegraded":"2023-10-16T14:33:36.637193+0000","last_fullsized":"2023-10-16T14:33:36.637193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040273+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:08.040273+0000","last_peered":"2023-10-16T14:33:08.040273+0000","last_clean":"2023-10-16T14:33:08.040273+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:08.040273+0000","last_undegraded":"2023-10-16T14:33:08.040273+0000","last_fullsized":"2023-10-16T14:33:08.040273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:48.668945+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:48.668945+0000","last_peered":"2023-10-16T14:33:48.668945+0000","last_clean":"2023-10-16T14:33:48.668945+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:48.668945+0000","last_undegraded":"2023-10-16T14:33:48.668945+0000","last_fullsized":"2023-10-16T14:33:48.668945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640506+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:36.640506+0000","last_peered":"2023-10-16T14:33:36.640506+0000","last_clean":"2023-10-16T14:33:36.640506+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:36.640506+0000","last_undegraded":"2023-10-16T14:33:36.640506+0000","last_fullsized":"2023-10-16T14:33:36.640506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041196+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:08.041196+0000","last_peered":"2023-10-16T14:33:08.041196+0000","last_clean":"2023-10-16T14:33:08.041196+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:08.041196+0000","last_undegraded":"2023-10-16T14:33:08.041196+0000","last_fullsized":"2023-10-16T14:33:08.041196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033443+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:08.033443+0000","last_peered":"2023-10-16T14:33:08.033443+0000","last_clean":"2023-10-16T14:33:08.033443+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:08.033443+0000","last_undegraded":"2023-10-16T14:33:08.033443+0000","last_fullsized":"2023-10-16T14:33:08.033443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635668+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:36.635668+0000","last_peered":"2023-10-16T14:33:36.635668+0000","last_clean":"2023-10-16T14:33:36.635668+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:36.635668+0000","last_undegraded":"2023-10-16T14:33:36.635668+0000","last_fullsized":"2023-10-16T14:33:36.635668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454661+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:33:08.454661+0000","last_peered":"2023-10-16T14:33:08.454661+0000","last_clean":"2023-10-16T14:33:08.454661+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:33:08.454661+0000","last_undegraded":"2023-10-16T14:33:08.454661+0000","last_fullsized":"2023-10-16T14:33:08.454661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290521+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:08.290521+0000","last_peered":"2023-10-16T14:33:08.290521+0000","last_clean":"2023-10-16T14:33:08.290521+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:08.290521+0000","last_undegraded":"2023-10-16T14:33:08.290521+0000","last_fullsized":"2023-10-16T14:33:08.290521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637090+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:36.637090+0000","last_peered":"2023-10-16T14:33:36.637090+0000","last_clean":"2023-10-16T14:33:36.637090+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:36.637090+0000","last_undegraded":"2023-10-16T14:33:36.637090+0000","last_fullsized":"2023-10-16T14:33:36.637090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052803+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:08.052803+0000","last_peered":"2023-10-16T14:33:08.052803+0000","last_clean":"2023-10-16T14:33:08.052803+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:08.052803+0000","last_undegraded":"2023-10-16T14:33:08.052803+0000","last_fullsized":"2023-10-16T14:33:08.052803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044786+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:08.044786+0000","last_peered":"2023-10-16T14:33:08.044786+0000","last_clean":"2023-10-16T14:33:08.044786+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:08.044786+0000","last_undegraded":"2023-10-16T14:33:08.044786+0000","last_fullsized":"2023-10-16T14:33:08.044786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635462+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:36.635462+0000","last_peered":"2023-10-16T14:33:36.635462+0000","last_clean":"2023-10-16T14:33:36.635462+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:36.635462+0000","last_undegraded":"2023-10-16T14:33:36.635462+0000","last_fullsized":"2023-10-16T14:33:36.635462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044128+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:08.044128+0000","last_peered":"2023-10-16T14:33:08.044128+0000","last_clean":"2023-10-16T14:33:08.044128+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:08.044128+0000","last_undegraded":"2023-10-16T14:33:08.044128+0000","last_fullsized":"2023-10-16T14:33:08.044128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416659+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:08.416659+0000","last_peered":"2023-10-16T14:33:08.416659+0000","last_clean":"2023-10-16T14:33:08.416659+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:08.416659+0000","last_undegraded":"2023-10-16T14:33:08.416659+0000","last_fullsized":"2023-10-16T14:33:08.416659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631478+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:36.631478+0000","last_peered":"2023-10-16T14:33:36.631478+0000","last_clean":"2023-10-16T14:33:36.631478+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:36.631478+0000","last_undegraded":"2023-10-16T14:33:36.631478+0000","last_fullsized":"2023-10-16T14:33:36.631478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.625624+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:36.625624+0000","last_peered":"2023-10-16T14:33:36.625624+0000","last_clean":"2023-10-16T14:33:36.625624+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:36.625624+0000","last_undegraded":"2023-10-16T14:33:36.625624+0000","last_fullsized":"2023-10-16T14:33:36.625624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641472+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:36.641472+0000","last_peered":"2023-10-16T14:33:36.641472+0000","last_clean":"2023-10-16T14:33:36.641472+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:36.641472+0000","last_undegraded":"2023-10-16T14:33:36.641472+0000","last_fullsized":"2023-10-16T14:33:36.641472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636199+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:36.636199+0000","last_peered":"2023-10-16T14:33:36.636199+0000","last_clean":"2023-10-16T14:33:36.636199+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:36.636199+0000","last_undegraded":"2023-10-16T14:33:36.636199+0000","last_fullsized":"2023-10-16T14:33:36.636199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641467+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:36.641467+0000","last_peered":"2023-10-16T14:33:36.641467+0000","last_clean":"2023-10-16T14:33:36.641467+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:36.641467+0000","last_undegraded":"2023-10-16T14:33:36.641467+0000","last_fullsized":"2023-10-16T14:33:36.641467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639422+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:36.639422+0000","last_peered":"2023-10-16T14:33:36.639422+0000","last_clean":"2023-10-16T14:33:36.639422+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:36.639422+0000","last_undegraded":"2023-10-16T14:33:36.639422+0000","last_fullsized":"2023-10-16T14:33:36.639422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621053+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:36.621053+0000","last_peered":"2023-10-16T14:33:36.621053+0000","last_clean":"2023-10-16T14:33:36.621053+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:36.621053+0000","last_undegraded":"2023-10-16T14:33:36.621053+0000","last_fullsized":"2023-10-16T14:33:36.621053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626446+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:36.626446+0000","last_peered":"2023-10-16T14:33:36.626446+0000","last_clean":"2023-10-16T14:33:36.626446+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:36.626446+0000","last_undegraded":"2023-10-16T14:33:36.626446+0000","last_fullsized":"2023-10-16T14:33:36.626446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642770+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:36.642770+0000","last_peered":"2023-10-16T14:33:36.642770+0000","last_clean":"2023-10-16T14:33:36.642770+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:36.642770+0000","last_undegraded":"2023-10-16T14:33:36.642770+0000","last_fullsized":"2023-10-16T14:33:36.642770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636872+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:36.636872+0000","last_peered":"2023-10-16T14:33:36.636872+0000","last_clean":"2023-10-16T14:33:36.636872+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:36.636872+0000","last_undegraded":"2023-10-16T14:33:36.636872+0000","last_fullsized":"2023-10-16T14:33:36.636872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637998+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:36.637998+0000","last_peered":"2023-10-16T14:33:36.637998+0000","last_clean":"2023-10-16T14:33:36.637998+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:36.637998+0000","last_undegraded":"2023-10-16T14:33:36.637998+0000","last_fullsized":"2023-10-16T14:33:36.637998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642106+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:33:36.642106+0000","last_peered":"2023-10-16T14:33:36.642106+0000","last_clean":"2023-10-16T14:33:36.642106+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:33:36.642106+0000","last_undegraded":"2023-10-16T14:33:36.642106+0000","last_fullsized":"2023-10-16T14:33:36.642106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639941+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:36.639941+0000","last_peered":"2023-10-16T14:33:36.639941+0000","last_clean":"2023-10-16T14:33:36.639941+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:36.639941+0000","last_undegraded":"2023-10-16T14:33:36.639941+0000","last_fullsized":"2023-10-16T14:33:36.639941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631870+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:36.631870+0000","last_peered":"2023-10-16T14:33:36.631870+0000","last_clean":"2023-10-16T14:33:36.631870+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:36.631870+0000","last_undegraded":"2023-10-16T14:33:36.631870+0000","last_fullsized":"2023-10-16T14:33:36.631870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82100,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":778,"num_read_kb":475,"num_write":178,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449133,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97899999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.56999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.56399999999999995,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.436},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.81200000000000006,"5min":0.871,"15min":0.871},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.62,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.72299999999999998},"min":{"1min":0.59199999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.65100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.70899999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0189999999999999,"5min":1.256,"15min":1.256},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.70799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79700000000000004,"15min":0.78400000000000003},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.75900000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.996,"5min":1.214,"15min":1.214},"last":0.78200000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678064,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1724416,"data_stored":1163249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.997,"15min":0.997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498}]},{"osd":2,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82299999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52000000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.64000000000000001,"15min":0.65300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.83399999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77200000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93200000000000005,"5min":1.01,"15min":1.01},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73799999999999999,"15min":0.75},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":1.016,"15min":1.016},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.92400000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.49099999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.83099999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95699999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.752},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.55900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1120000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85599999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96499999999999997,"5min":1.038,"15min":1.038},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77200000000000002},"min":{"1min":0.59399999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.048,"15min":1.048},"last":0.78900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841590,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.51500000000000001,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59399999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.58899999999999997},"min":{"1min":0.34899999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61099999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69699999999999995,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91100000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.72699999999999998,"15min":0.79900000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81599999999999995,"5min":2.319,"15min":2.319},"last":0.80800000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.749,"15min":0.82899999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78500000000000003,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":1.849,"15min":1.849},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.68200000000000005,"15min":0.76200000000000001},"min":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.73899999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.753,"15min":0.82099999999999995},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88900000000000001,"5min":1.879,"15min":1.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.753,"15min":0.81599999999999995},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84299999999999997,"15min":0.93000000000000005},"min":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83299999999999996,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79500000000000004,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.877,"15min":0.97199999999999998},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1060000000000001,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.82599999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.84999999999999998,"15min":0.92700000000000005},"min":{"1min":0.57699999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.89200000000000002,"5min":1.944,"15min":1.944},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.83599999999999997,"15min":0.91100000000000003},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.86299999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.095,"5min":1.986,"15min":1.986},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.139,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.79200000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83099999999999996,"15min":0.82999999999999996},"min":{"1min":0.73499999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.016,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.93400000000000005,"15min":0.95699999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0429999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.67500000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103227,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.317,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67300000000000004,"5min":0.87,"15min":0.87},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.505},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59999999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67100000000000004},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67400000000000004,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.875,"5min":1.165,"15min":1.165},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.42499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81999999999999995,"5min":1.087,"15min":1.087},"last":0.39300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.47899999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":1.02,"15min":1.02},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.623}]},{"osd":7,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91400000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.78000000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.76700000000000002},"min":{"1min":0.55400000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64200000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.746,"15min":0.69899999999999995},"min":{"1min":0.66300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.006,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.075,"15min":1.075},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81599999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1113909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.625,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80700000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":2.589,"15min":2.589},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.432,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88700000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.63400000000000001},"min":{"1min":0.50700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.878,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84499999999999997,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.82199999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73299999999999998,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.96199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.879},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.86799999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.83699999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.60399999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.60499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.44800000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.497},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85699999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.748}]},{"osd":1,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81799999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83899999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.46000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75,"15min":0.73199999999999998},"min":{"1min":0.59199999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90800000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.64200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0880000000000001,"5min":1.232,"15min":1.232},"last":0.81000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.036,"5min":1.107,"15min":1.107},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93200000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0700000000000001,"5min":1.133,"15min":1.133},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0029999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":1.004,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.1220000000000001,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.90600000000000003},"min":{"1min":0.65900000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.105,"5min":1.157,"15min":1.157},"last":0.83999999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.85999999999999999},"min":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1060000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.4119999999999999,"15min":1.4990000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0680000000000001,"5min":11.394,"15min":11.394},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":566934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.34200000000000003},"max":{"1min":0.83299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.625,"15min":0.55100000000000005},"min":{"1min":0.55400000000000005,"5min":0.376,"15min":0.376},"max":{"1min":0.96599999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.60899999999999999,"15min":0.55100000000000005},"min":{"1min":0.499,"5min":0.309,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:34:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.502},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.53700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.42999999999999999},"min":{"1min":0.40500000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45700000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66700000000000004,"15min":0.625},"min":{"1min":0.57899999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.032,"5min":1.181,"15min":1.181},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.67200000000000004,"15min":0.59899999999999998},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":2.528,"15min":2.528},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.72699999999999998,"15min":0.68799999999999994},"min":{"1min":0.626,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.624},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.76600000000000001,"15min":0.71099999999999997},"min":{"1min":0.59699999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85999999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.56899999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0089999999999999,"5min":1.012,"15min":1.012},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.72099999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0129999999999999,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.72399999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81999999999999995,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77500000000000002,"15min":0.77200000000000002},"min":{"1min":0.47199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97599999999999998,"5min":1.054,"15min":1.054},"last":0.84199999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.89800000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.89700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.44700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"min":{"1min":0.55000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.141},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.60299999999999998},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95999999999999996,"5min":1.202,"15min":1.202},"last":0.88100000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.877,"15min":0.877},"last":0.435},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.754},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.629,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73599999999999999,"15min":0.66500000000000004},"min":{"1min":0.59699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.101,"15min":1.101},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.51900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.624,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92000000000000004,"5min":1.298,"15min":1.298},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.46600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.03,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85599999999999998,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0700000000000001,"5min":1.274,"15min":1.274},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.59899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.216},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79900000000000004,"15min":0.86499999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89600000000000002,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0509999999999999,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":0.95799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.70299999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.75900000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1379999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1}]},{"osd":11,"last update":"Mon Oct 16 14:33:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.95999999999999996},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.115,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":7.2480000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":581570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.372,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78900000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.52700000000000002},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.32700000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":1.022,"15min":1.022},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.50900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.874},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.36399999999999999},"max":{"1min":0.70399999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.49199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.45700000000000002},"min":{"1min":0.33600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.38,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1180000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84699999999999998,"15min":0.83899999999999997},"min":{"1min":0.71499999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.93000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.59999999999999998},"min":{"1min":0.61599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.64000000000000001,"15min":0.57799999999999996},"min":{"1min":0.56899999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70599999999999996,"15min":0.68300000000000005},"min":{"1min":0.63200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.747},"min":{"1min":0.74199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.97899999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.625,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.79700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.075,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.95299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93000000000000005,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.75900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.433},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48999999999999999,"15min":0.45800000000000002},"min":{"1min":0.34200000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.498}]},{"osd":1,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80300000000000005,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.60399999999999998},"min":{"1min":0.59199999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92500000000000004,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.83999999999999997,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.38400000000000001},"max":{"1min":0.91900000000000004,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:33:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":0.79700000000000004,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.443},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.72699999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.407,"5min":0.83499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":10.314,"5min":10.314,"15min":10.314},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.627},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91700000000000004,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73299999999999998,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.88500000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.80400000000000005,"15min":0.76400000000000001},"min":{"1min":0.61199999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.624},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.56399999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.45200000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.751},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.68400000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.91900000000000004,"15min":0.90500000000000003},"min":{"1min":0.71899999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.53600000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.871,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":0.99299999999999999,"15min":0.90300000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":10.361000000000001,"5min":10.361000000000001,"15min":10.361000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41599999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.66200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.433,"15min":0.40500000000000003},"min":{"1min":0.34399999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.71799999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.752,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81899999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"min":{"1min":0.57099999999999995,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98499999999999999,"5min":1.002,"15min":1.002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.56200000000000006},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.91400000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64100000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.49099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73599999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96399999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.81999999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97799999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.69599999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.83799999999999997,"15min":0.83299999999999996},"min":{"1min":0.66200000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.68899999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74199999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.79400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88600000000000001,"5min":1.425,"15min":1.425},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.40000000000000002},"min":{"1min":0.38500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.58099999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.436,"15min":0.40999999999999998},"min":{"1min":0.32200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95899999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82899999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64800000000000002,"15min":0.58399999999999996},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.629,"15min":0.56599999999999995},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.49299999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.52200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:33:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.196},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67300000000000004,"15min":0.64700000000000002},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.64100000000000001},"min":{"1min":0.56499999999999995,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.54600000000000004,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71599999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.66500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.629,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81299999999999994}]},{"osd":11,"last update":"Mon Oct 16 14:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.034,"5min":1.103,"15min":1.103},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.66300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":14489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:34:17.473 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:34:17.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 -- 172.21.15.178:0/304920484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 msgr2=0x7f41ec102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 --2- 172.21.15.178:0/304920484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec102500 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41dc009990 tx=0x7f41dc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 -- 172.21.15.178:0/304920484 shutdown_connections 2023-10-16T14:34:17.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 --2- 172.21.15.178:0/304920484 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec102a40 0x7f41ec109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 --2- 172.21.15.178:0/304920484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec100ea0 0x7f41ec1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 --2- 172.21.15.178:0/304920484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec102500 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 -- 172.21.15.178:0/304920484 >> 172.21.15.178:0/304920484 conn(0x7f41ec0fc460 msgr2=0x7f41ec0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:17.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 -- 172.21.15.178:0/304920484 shutdown_connections 2023-10-16T14:34:17.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.671+0000 7f41f43ff700 1 -- 172.21.15.178:0/304920484 wait complete. 2023-10-16T14:34:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.672+0000 7f41f43ff700 1 Processor -- start 2023-10-16T14:34:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.672+0000 7f41f43ff700 1 -- start start 2023-10-16T14:34:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec100ea0 0x7f41ec198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1a08f0 con 0x7f41ec102080 2023-10-16T14:34:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1a0a30 con 0x7f41ec100ea0 2023-10-16T14:34:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f43ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ec1a0bc0 con 0x7f41ec102a40 2023-10-16T14:34:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f299c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40996/0 (socket says 172.21.15.178:40996) 2023-10-16T14:34:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f299c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:33832/0 (socket says 172.21.15.178:33832) 2023-10-16T14:34:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 -- 172.21.15.178:0/2245140106 learned_addr learned my addr 172.21.15.178:0/2245140106 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f219b700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec100ea0 0x7f41ec198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 -- 172.21.15.178:0/2245140106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 msgr2=0x7f41ec1a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 -- 172.21.15.178:0/2245140106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec100ea0 msgr2=0x7f41ec198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec100ea0 0x7f41ec198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f199a700 1 -- 172.21.15.178:0/2245140106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0095f0 con 0x7f41ec102080 2023-10-16T14:34:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.673+0000 7f41f299c700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41f199a700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004710 tx=0x7f41dc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:17.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc01d070 con 0x7f41ec102080 2023-10-16T14:34:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41dc021410 con 0x7f41ec102080 2023-10-16T14:34:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41f43ff700 1 -- 172.21.15.178:0/2245140106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ec1a0e40 con 0x7f41ec102080 2023-10-16T14:34:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc017650 con 0x7f41ec102080 2023-10-16T14:34:17.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.674+0000 7f41f43ff700 1 -- 172.21.15.178:0/2245140106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ec1a11b0 con 0x7f41ec102080 2023-10-16T14:34:17.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.675+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d40052f0 con 0x7f41ec102080 2023-10-16T14:34:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.677+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f41dc021580 con 0x7f41ec102080 2023-10-16T14:34:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.678+0000 7f41e37fe700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 0x7f41d80a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.678+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f41dc0ce4a0 con 0x7f41ec102080 2023-10-16T14:34:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.679+0000 7f41f219b700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 0x7f41d80a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:17.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.679+0000 7f41f219b700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 0x7f41d80a39e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41ec101bf0 tx=0x7f41e8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:17.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.682+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f41dc0977c0 con 0x7f41ec102080 2023-10-16T14:34:17.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:17 smithi178 ceph-mon[98495]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-16T14:34:17.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:17 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1510386399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:17.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.879+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41d4000bc0 con 0x7f41d80a1520 2023-10-16T14:34:17.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.892+0000 7f41e37fe700 1 -- 172.21.15.178:0/2245140106 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427827 (secure 0 0 0) 0x7f41d4000bc0 con 0x7f41d80a1520 2023-10-16T14:34:17.901 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:17.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 msgr2=0x7f41d80a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 0x7f41d80a39e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41ec101bf0 tx=0x7f41e8009260 comp rx=0 tx=0).stop 2023-10-16T14:34:17.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 msgr2=0x7f41ec1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:17.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f41dc004710 tx=0x7f41dc009580 comp rx=0 tx=0).stop 2023-10-16T14:34:17.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 shutdown_connections 2023-10-16T14:34:17.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f41d80a1520 0x7f41d80a39e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41ec102a40 0x7f41ec1a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f41ec100ea0 0x7f41ec198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 --2- 172.21.15.178:0/2245140106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41ec102080 0x7f41ec1992b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:17.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 >> 172.21.15.178:0/2245140106 conn(0x7f41ec0fc460 msgr2=0x7f41ec0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:17.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 shutdown_connections 2023-10-16T14:34:17.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:17.898+0000 7f41e17fa700 1 -- 172.21.15.178:0/2245140106 wait complete. 2023-10-16T14:34:17.907 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:34:17.921 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2023-10-16T14:34:16.630146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672714,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":904,"num_read_kb":584,"num_write":361,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338336,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603488,"statfs":{"total":1151940427776,"available":1151593971712,"internally_reserved":0,"allocated":14483456,"data_stored":8068243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2484,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":39,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637229+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:36.637229+0000","last_peered":"2023-10-16T14:33:36.637229+0000","last_clean":"2023-10-16T14:33:36.637229+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637229+0000","last_undegraded":"2023-10-16T14:33:36.637229+0000","last_fullsized":"2023-10-16T14:33:36.637229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641049+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:36.641049+0000","last_peered":"2023-10-16T14:33:36.641049+0000","last_clean":"2023-10-16T14:33:36.641049+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:36.641049+0000","last_undegraded":"2023-10-16T14:33:36.641049+0000","last_fullsized":"2023-10-16T14:33:36.641049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636867+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:36.636867+0000","last_peered":"2023-10-16T14:33:36.636867+0000","last_clean":"2023-10-16T14:33:36.636867+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:36.636867+0000","last_undegraded":"2023-10-16T14:33:36.636867+0000","last_fullsized":"2023-10-16T14:33:36.636867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638334+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:36.638334+0000","last_peered":"2023-10-16T14:33:36.638334+0000","last_clean":"2023-10-16T14:33:36.638334+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:36.638334+0000","last_undegraded":"2023-10-16T14:33:36.638334+0000","last_fullsized":"2023-10-16T14:33:36.638334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636004+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:36.636004+0000","last_peered":"2023-10-16T14:33:36.636004+0000","last_clean":"2023-10-16T14:33:36.636004+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:36.636004+0000","last_undegraded":"2023-10-16T14:33:36.636004+0000","last_fullsized":"2023-10-16T14:33:36.636004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642338+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:33:36.642338+0000","last_peered":"2023-10-16T14:33:36.642338+0000","last_clean":"2023-10-16T14:33:36.642338+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:33:36.642338+0000","last_undegraded":"2023-10-16T14:33:36.642338+0000","last_fullsized":"2023-10-16T14:33:36.642338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627444+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:36.627444+0000","last_peered":"2023-10-16T14:33:36.627444+0000","last_clean":"2023-10-16T14:33:36.627444+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:36.627444+0000","last_undegraded":"2023-10-16T14:33:36.627444+0000","last_fullsized":"2023-10-16T14:33:36.627444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639455+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:36.639455+0000","last_peered":"2023-10-16T14:33:36.639455+0000","last_clean":"2023-10-16T14:33:36.639455+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:36.639455+0000","last_undegraded":"2023-10-16T14:33:36.639455+0000","last_fullsized":"2023-10-16T14:33:36.639455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626801+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:33:36.626801+0000","last_peered":"2023-10-16T14:33:36.626801+0000","last_clean":"2023-10-16T14:33:36.626801+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:33:36.626801+0000","last_undegraded":"2023-10-16T14:33:36.626801+0000","last_fullsized":"2023-10-16T14:33:36.626801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642427+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:33:36.642427+0000","last_peered":"2023-10-16T14:33:36.642427+0000","last_clean":"2023-10-16T14:33:36.642427+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:33:36.642427+0000","last_undegraded":"2023-10-16T14:33:36.642427+0000","last_fullsized":"2023-10-16T14:33:36.642427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637246+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:36.637246+0000","last_peered":"2023-10-16T14:33:36.637246+0000","last_clean":"2023-10-16T14:33:36.637246+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:36.637246+0000","last_undegraded":"2023-10-16T14:33:36.637246+0000","last_fullsized":"2023-10-16T14:33:36.637246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639803+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:36.639803+0000","last_peered":"2023-10-16T14:33:36.639803+0000","last_clean":"2023-10-16T14:33:36.639803+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:36.639803+0000","last_undegraded":"2023-10-16T14:33:36.639803+0000","last_fullsized":"2023-10-16T14:33:36.639803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641688+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:36.641688+0000","last_peered":"2023-10-16T14:33:36.641688+0000","last_clean":"2023-10-16T14:33:36.641688+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:36.641688+0000","last_undegraded":"2023-10-16T14:33:36.641688+0000","last_fullsized":"2023-10-16T14:33:36.641688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641600+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:36.641600+0000","last_peered":"2023-10-16T14:33:36.641600+0000","last_clean":"2023-10-16T14:33:36.641600+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:36.641600+0000","last_undegraded":"2023-10-16T14:33:36.641600+0000","last_fullsized":"2023-10-16T14:33:36.641600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641196+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:36.641196+0000","last_peered":"2023-10-16T14:33:36.641196+0000","last_clean":"2023-10-16T14:33:36.641196+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:36.641196+0000","last_undegraded":"2023-10-16T14:33:36.641196+0000","last_fullsized":"2023-10-16T14:33:36.641196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637421+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:36.637421+0000","last_peered":"2023-10-16T14:33:36.637421+0000","last_clean":"2023-10-16T14:33:36.637421+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:36.637421+0000","last_undegraded":"2023-10-16T14:33:36.637421+0000","last_fullsized":"2023-10-16T14:33:36.637421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637900+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:36.637900+0000","last_peered":"2023-10-16T14:33:36.637900+0000","last_clean":"2023-10-16T14:33:36.637900+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:36.637900+0000","last_undegraded":"2023-10-16T14:33:36.637900+0000","last_fullsized":"2023-10-16T14:33:36.637900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621257+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:36.621257+0000","last_peered":"2023-10-16T14:33:36.621257+0000","last_clean":"2023-10-16T14:33:36.621257+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:36.621257+0000","last_undegraded":"2023-10-16T14:33:36.621257+0000","last_fullsized":"2023-10-16T14:33:36.621257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290036+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:08.290036+0000","last_peered":"2023-10-16T14:33:08.290036+0000","last_clean":"2023-10-16T14:33:08.290036+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:08.290036+0000","last_undegraded":"2023-10-16T14:33:08.290036+0000","last_fullsized":"2023-10-16T14:33:08.290036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044284+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:08.044284+0000","last_peered":"2023-10-16T14:33:08.044284+0000","last_clean":"2023-10-16T14:33:08.044284+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:08.044284+0000","last_undegraded":"2023-10-16T14:33:08.044284+0000","last_fullsized":"2023-10-16T14:33:08.044284+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638964+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:36.638964+0000","last_peered":"2023-10-16T14:33:36.638964+0000","last_clean":"2023-10-16T14:33:36.638964+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:36.638964+0000","last_undegraded":"2023-10-16T14:33:36.638964+0000","last_fullsized":"2023-10-16T14:33:36.638964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.039726+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:08.039726+0000","last_peered":"2023-10-16T14:33:08.039726+0000","last_clean":"2023-10-16T14:33:08.039726+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:08.039726+0000","last_undegraded":"2023-10-16T14:33:08.039726+0000","last_fullsized":"2023-10-16T14:33:08.039726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038574+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:08.038574+0000","last_peered":"2023-10-16T14:33:08.038574+0000","last_clean":"2023-10-16T14:33:08.038574+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:08.038574+0000","last_undegraded":"2023-10-16T14:33:08.038574+0000","last_fullsized":"2023-10-16T14:33:08.038574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636879+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:36.636879+0000","last_peered":"2023-10-16T14:33:36.636879+0000","last_clean":"2023-10-16T14:33:36.636879+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:36.636879+0000","last_undegraded":"2023-10-16T14:33:36.636879+0000","last_fullsized":"2023-10-16T14:33:36.636879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040318+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:08.040318+0000","last_peered":"2023-10-16T14:33:08.040318+0000","last_clean":"2023-10-16T14:33:08.040318+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:08.040318+0000","last_undegraded":"2023-10-16T14:33:08.040318+0000","last_fullsized":"2023-10-16T14:33:08.040318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033698+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:08.033698+0000","last_peered":"2023-10-16T14:33:08.033698+0000","last_clean":"2023-10-16T14:33:08.033698+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:08.033698+0000","last_undegraded":"2023-10-16T14:33:08.033698+0000","last_fullsized":"2023-10-16T14:33:08.033698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638148+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:33:36.638148+0000","last_peered":"2023-10-16T14:33:36.638148+0000","last_clean":"2023-10-16T14:33:36.638148+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:33:36.638148+0000","last_undegraded":"2023-10-16T14:33:36.638148+0000","last_fullsized":"2023-10-16T14:33:36.638148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.418372+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.418372+0000","last_peered":"2023-10-16T14:33:08.418372+0000","last_clean":"2023-10-16T14:33:08.418372+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.418372+0000","last_undegraded":"2023-10-16T14:33:08.418372+0000","last_fullsized":"2023-10-16T14:33:08.418372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635989+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:36.635989+0000","last_peered":"2023-10-16T14:33:36.635989+0000","last_clean":"2023-10-16T14:33:36.635989+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:36.635989+0000","last_undegraded":"2023-10-16T14:33:36.635989+0000","last_fullsized":"2023-10-16T14:33:36.635989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290088+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:08.290088+0000","last_peered":"2023-10-16T14:33:08.290088+0000","last_clean":"2023-10-16T14:33:08.290088+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:08.290088+0000","last_undegraded":"2023-10-16T14:33:08.290088+0000","last_fullsized":"2023-10-16T14:33:08.290088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'12","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:13.904131+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:34:13.904131+0000","last_peered":"2023-10-16T14:34:13.904131+0000","last_clean":"2023-10-16T14:34:13.904131+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:34:13.904131+0000","last_undegraded":"2023-10-16T14:34:13.904131+0000","last_fullsized":"2023-10-16T14:34:13.904131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":17,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.618324+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:36.618324+0000","last_peered":"2023-10-16T14:33:36.618324+0000","last_clean":"2023-10-16T14:33:36.618324+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:36.618324+0000","last_undegraded":"2023-10-16T14:33:36.618324+0000","last_fullsized":"2023-10-16T14:33:36.618324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290297+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:08.290297+0000","last_peered":"2023-10-16T14:33:08.290297+0000","last_clean":"2023-10-16T14:33:08.290297+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:08.290297+0000","last_undegraded":"2023-10-16T14:33:08.290297+0000","last_fullsized":"2023-10-16T14:33:08.290297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037660+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:33:08.037660+0000","last_peered":"2023-10-16T14:33:08.037660+0000","last_clean":"2023-10-16T14:33:08.037660+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:33:08.037660+0000","last_undegraded":"2023-10-16T14:33:08.037660+0000","last_fullsized":"2023-10-16T14:33:08.037660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641111+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:36.641111+0000","last_peered":"2023-10-16T14:33:36.641111+0000","last_clean":"2023-10-16T14:33:36.641111+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:36.641111+0000","last_undegraded":"2023-10-16T14:33:36.641111+0000","last_fullsized":"2023-10-16T14:33:36.641111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044215+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:08.044215+0000","last_peered":"2023-10-16T14:33:08.044215+0000","last_clean":"2023-10-16T14:33:08.044215+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:08.044215+0000","last_undegraded":"2023-10-16T14:33:08.044215+0000","last_fullsized":"2023-10-16T14:33:08.044215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416188+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:08.416188+0000","last_peered":"2023-10-16T14:33:08.416188+0000","last_clean":"2023-10-16T14:33:08.416188+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:08.416188+0000","last_undegraded":"2023-10-16T14:33:08.416188+0000","last_fullsized":"2023-10-16T14:33:08.416188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638397+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:36.638397+0000","last_peered":"2023-10-16T14:33:36.638397+0000","last_clean":"2023-10-16T14:33:36.638397+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:36.638397+0000","last_undegraded":"2023-10-16T14:33:36.638397+0000","last_fullsized":"2023-10-16T14:33:36.638397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052948+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:08.052948+0000","last_peered":"2023-10-16T14:33:08.052948+0000","last_clean":"2023-10-16T14:33:08.052948+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:08.052948+0000","last_undegraded":"2023-10-16T14:33:08.052948+0000","last_fullsized":"2023-10-16T14:33:08.052948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044478+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:08.044478+0000","last_peered":"2023-10-16T14:33:08.044478+0000","last_clean":"2023-10-16T14:33:08.044478+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:08.044478+0000","last_undegraded":"2023-10-16T14:33:08.044478+0000","last_fullsized":"2023-10-16T14:33:08.044478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640406+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:36.640406+0000","last_peered":"2023-10-16T14:33:36.640406+0000","last_clean":"2023-10-16T14:33:36.640406+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:36.640406+0000","last_undegraded":"2023-10-16T14:33:36.640406+0000","last_fullsized":"2023-10-16T14:33:36.640406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038135+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:08.038135+0000","last_peered":"2023-10-16T14:33:08.038135+0000","last_clean":"2023-10-16T14:33:08.038135+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:08.038135+0000","last_undegraded":"2023-10-16T14:33:08.038135+0000","last_fullsized":"2023-10-16T14:33:08.038135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.542711+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:34:11.542711+0000","last_peered":"2023-10-16T14:34:11.542711+0000","last_clean":"2023-10-16T14:34:11.542711+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:34:11.542711+0000","last_undegraded":"2023-10-16T14:34:11.542711+0000","last_fullsized":"2023-10-16T14:34:11.542711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638618+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:33:36.638618+0000","last_peered":"2023-10-16T14:33:36.638618+0000","last_clean":"2023-10-16T14:33:36.638618+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:33:36.638618+0000","last_undegraded":"2023-10-16T14:33:36.638618+0000","last_fullsized":"2023-10-16T14:33:36.638618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043503+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:08.043503+0000","last_peered":"2023-10-16T14:33:08.043503+0000","last_clean":"2023-10-16T14:33:08.043503+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:08.043503+0000","last_undegraded":"2023-10-16T14:33:08.043503+0000","last_fullsized":"2023-10-16T14:33:08.043503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041325+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:08.041325+0000","last_peered":"2023-10-16T14:33:08.041325+0000","last_clean":"2023-10-16T14:33:08.041325+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:08.041325+0000","last_undegraded":"2023-10-16T14:33:08.041325+0000","last_fullsized":"2023-10-16T14:33:08.041325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.643248+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:36.643248+0000","last_peered":"2023-10-16T14:33:36.643248+0000","last_clean":"2023-10-16T14:33:36.643248+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:36.643248+0000","last_undegraded":"2023-10-16T14:33:36.643248+0000","last_fullsized":"2023-10-16T14:33:36.643248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040462+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:08.040462+0000","last_peered":"2023-10-16T14:33:08.040462+0000","last_clean":"2023-10-16T14:33:08.040462+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:08.040462+0000","last_undegraded":"2023-10-16T14:33:08.040462+0000","last_fullsized":"2023-10-16T14:33:08.040462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043844+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:08.043844+0000","last_peered":"2023-10-16T14:33:08.043844+0000","last_clean":"2023-10-16T14:33:08.043844+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:08.043844+0000","last_undegraded":"2023-10-16T14:33:08.043844+0000","last_fullsized":"2023-10-16T14:33:08.043844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641090+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:33:36.641090+0000","last_peered":"2023-10-16T14:33:36.641090+0000","last_clean":"2023-10-16T14:33:36.641090+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:33:36.641090+0000","last_undegraded":"2023-10-16T14:33:36.641090+0000","last_fullsized":"2023-10-16T14:33:36.641090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.045902+0000","last_change":"2023-10-16T14:33:08.045902+0000","last_active":"2023-10-16T14:33:08.045902+0000","last_peered":"2023-10-16T14:33:08.045902+0000","last_clean":"2023-10-16T14:33:08.045902+0000","last_became_active":"2023-10-16T14:33:08.045481+0000","last_became_peered":"2023-10-16T14:33:08.045481+0000","last_unstale":"2023-10-16T14:33:08.045902+0000","last_undegraded":"2023-10-16T14:33:08.045902+0000","last_fullsized":"2023-10-16T14:33:08.045902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:18.091442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":314,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.899607+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:34:11.899607+0000","last_peered":"2023-10-16T14:34:11.899607+0000","last_clean":"2023-10-16T14:34:11.899607+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:34:11.899607+0000","last_undegraded":"2023-10-16T14:34:11.899607+0000","last_fullsized":"2023-10-16T14:34:11.899607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65477,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":192,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.629796+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:36.629796+0000","last_peered":"2023-10-16T14:33:36.629796+0000","last_clean":"2023-10-16T14:33:36.629796+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:36.629796+0000","last_undegraded":"2023-10-16T14:33:36.629796+0000","last_fullsized":"2023-10-16T14:33:36.629796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044920+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:08.044920+0000","last_peered":"2023-10-16T14:33:08.044920+0000","last_clean":"2023-10-16T14:33:08.044920+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:08.044920+0000","last_undegraded":"2023-10-16T14:33:08.044920+0000","last_fullsized":"2023-10-16T14:33:08.044920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290160+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:08.290160+0000","last_peered":"2023-10-16T14:33:08.290160+0000","last_clean":"2023-10-16T14:33:08.290160+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:08.290160+0000","last_undegraded":"2023-10-16T14:33:08.290160+0000","last_fullsized":"2023-10-16T14:33:08.290160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.632140+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:36.632140+0000","last_peered":"2023-10-16T14:33:36.632140+0000","last_clean":"2023-10-16T14:33:36.632140+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:36.632140+0000","last_undegraded":"2023-10-16T14:33:36.632140+0000","last_fullsized":"2023-10-16T14:33:36.632140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044018+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:08.044018+0000","last_peered":"2023-10-16T14:33:08.044018+0000","last_clean":"2023-10-16T14:33:08.044018+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:08.044018+0000","last_undegraded":"2023-10-16T14:33:08.044018+0000","last_fullsized":"2023-10-16T14:33:08.044018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044199+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:08.044199+0000","last_peered":"2023-10-16T14:33:08.044199+0000","last_clean":"2023-10-16T14:33:08.044199+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:08.044199+0000","last_undegraded":"2023-10-16T14:33:08.044199+0000","last_fullsized":"2023-10-16T14:33:08.044199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642241+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:36.642241+0000","last_peered":"2023-10-16T14:33:36.642241+0000","last_clean":"2023-10-16T14:33:36.642241+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:36.642241+0000","last_undegraded":"2023-10-16T14:33:36.642241+0000","last_fullsized":"2023-10-16T14:33:36.642241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043967+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:08.043967+0000","last_peered":"2023-10-16T14:33:08.043967+0000","last_clean":"2023-10-16T14:33:08.043967+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:08.043967+0000","last_undegraded":"2023-10-16T14:33:08.043967+0000","last_fullsized":"2023-10-16T14:33:08.043967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:12.900730+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:34:12.900730+0000","last_peered":"2023-10-16T14:34:12.900730+0000","last_clean":"2023-10-16T14:34:12.900730+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:34:12.900730+0000","last_undegraded":"2023-10-16T14:34:12.900730+0000","last_fullsized":"2023-10-16T14:34:12.900730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637695+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:36.637695+0000","last_peered":"2023-10-16T14:33:36.637695+0000","last_clean":"2023-10-16T14:33:36.637695+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:36.637695+0000","last_undegraded":"2023-10-16T14:33:36.637695+0000","last_fullsized":"2023-10-16T14:33:36.637695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416657+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:08.416657+0000","last_peered":"2023-10-16T14:33:08.416657+0000","last_clean":"2023-10-16T14:33:08.416657+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:08.416657+0000","last_undegraded":"2023-10-16T14:33:08.416657+0000","last_fullsized":"2023-10-16T14:33:08.416657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033313+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:08.033313+0000","last_peered":"2023-10-16T14:33:08.033313+0000","last_clean":"2023-10-16T14:33:08.033313+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:08.033313+0000","last_undegraded":"2023-10-16T14:33:08.033313+0000","last_fullsized":"2023-10-16T14:33:08.033313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640793+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:36.640793+0000","last_peered":"2023-10-16T14:33:36.640793+0000","last_clean":"2023-10-16T14:33:36.640793+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:36.640793+0000","last_undegraded":"2023-10-16T14:33:36.640793+0000","last_fullsized":"2023-10-16T14:33:36.640793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043832+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:08.043832+0000","last_peered":"2023-10-16T14:33:08.043832+0000","last_clean":"2023-10-16T14:33:08.043832+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:08.043832+0000","last_undegraded":"2023-10-16T14:33:08.043832+0000","last_fullsized":"2023-10-16T14:33:08.043832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033787+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:08.033787+0000","last_peered":"2023-10-16T14:33:08.033787+0000","last_clean":"2023-10-16T14:33:08.033787+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:08.033787+0000","last_undegraded":"2023-10-16T14:33:08.033787+0000","last_fullsized":"2023-10-16T14:33:08.033787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640908+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:36.640908+0000","last_peered":"2023-10-16T14:33:36.640908+0000","last_clean":"2023-10-16T14:33:36.640908+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:36.640908+0000","last_undegraded":"2023-10-16T14:33:36.640908+0000","last_fullsized":"2023-10-16T14:33:36.640908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033930+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:08.033930+0000","last_peered":"2023-10-16T14:33:08.033930+0000","last_clean":"2023-10-16T14:33:08.033930+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:08.033930+0000","last_undegraded":"2023-10-16T14:33:08.033930+0000","last_fullsized":"2023-10-16T14:33:08.033930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043926+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:08.043926+0000","last_peered":"2023-10-16T14:33:08.043926+0000","last_clean":"2023-10-16T14:33:08.043926+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:08.043926+0000","last_undegraded":"2023-10-16T14:33:08.043926+0000","last_fullsized":"2023-10-16T14:33:08.043926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638858+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:33:36.638858+0000","last_peered":"2023-10-16T14:33:36.638858+0000","last_clean":"2023-10-16T14:33:36.638858+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:33:36.638858+0000","last_undegraded":"2023-10-16T14:33:36.638858+0000","last_fullsized":"2023-10-16T14:33:36.638858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416348+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:08.416348+0000","last_peered":"2023-10-16T14:33:08.416348+0000","last_clean":"2023-10-16T14:33:08.416348+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:08.416348+0000","last_undegraded":"2023-10-16T14:33:08.416348+0000","last_fullsized":"2023-10-16T14:33:08.416348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041272+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:08.041272+0000","last_peered":"2023-10-16T14:33:08.041272+0000","last_clean":"2023-10-16T14:33:08.041272+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:08.041272+0000","last_undegraded":"2023-10-16T14:33:08.041272+0000","last_fullsized":"2023-10-16T14:33:08.041272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637932+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:33:36.637932+0000","last_peered":"2023-10-16T14:33:36.637932+0000","last_clean":"2023-10-16T14:33:36.637932+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:33:36.637932+0000","last_undegraded":"2023-10-16T14:33:36.637932+0000","last_fullsized":"2023-10-16T14:33:36.637932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416540+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:08.416540+0000","last_peered":"2023-10-16T14:33:08.416540+0000","last_clean":"2023-10-16T14:33:08.416540+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:08.416540+0000","last_undegraded":"2023-10-16T14:33:08.416540+0000","last_fullsized":"2023-10-16T14:33:08.416540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627499+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:36.627499+0000","last_peered":"2023-10-16T14:33:36.627499+0000","last_clean":"2023-10-16T14:33:36.627499+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:36.627499+0000","last_undegraded":"2023-10-16T14:33:36.627499+0000","last_fullsized":"2023-10-16T14:33:36.627499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:14.903045+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:34:14.903045+0000","last_peered":"2023-10-16T14:34:14.903045+0000","last_clean":"2023-10-16T14:34:14.903045+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:34:14.903045+0000","last_undegraded":"2023-10-16T14:34:14.903045+0000","last_fullsized":"2023-10-16T14:34:14.903045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050766+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050766+0000","last_peered":"2023-10-16T14:33:08.050766+0000","last_clean":"2023-10-16T14:33:08.050766+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050766+0000","last_undegraded":"2023-10-16T14:33:08.050766+0000","last_fullsized":"2023-10-16T14:33:08.050766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040458+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:08.040458+0000","last_peered":"2023-10-16T14:33:08.040458+0000","last_clean":"2023-10-16T14:33:08.040458+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:08.040458+0000","last_undegraded":"2023-10-16T14:33:08.040458+0000","last_fullsized":"2023-10-16T14:33:08.040458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040909+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:08.040909+0000","last_peered":"2023-10-16T14:33:08.040909+0000","last_clean":"2023-10-16T14:33:08.040909+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:08.040909+0000","last_undegraded":"2023-10-16T14:33:08.040909+0000","last_fullsized":"2023-10-16T14:33:08.040909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044956+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:08.044956+0000","last_peered":"2023-10-16T14:33:08.044956+0000","last_clean":"2023-10-16T14:33:08.044956+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:08.044956+0000","last_undegraded":"2023-10-16T14:33:08.044956+0000","last_fullsized":"2023-10-16T14:33:08.044956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627304+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:36.627304+0000","last_peered":"2023-10-16T14:33:36.627304+0000","last_clean":"2023-10-16T14:33:36.627304+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:36.627304+0000","last_undegraded":"2023-10-16T14:33:36.627304+0000","last_fullsized":"2023-10-16T14:33:36.627304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040976+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:08.040976+0000","last_peered":"2023-10-16T14:33:08.040976+0000","last_clean":"2023-10-16T14:33:08.040976+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:08.040976+0000","last_undegraded":"2023-10-16T14:33:08.040976+0000","last_fullsized":"2023-10-16T14:33:08.040976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043860+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:08.043860+0000","last_peered":"2023-10-16T14:33:08.043860+0000","last_clean":"2023-10-16T14:33:08.043860+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:08.043860+0000","last_undegraded":"2023-10-16T14:33:08.043860+0000","last_fullsized":"2023-10-16T14:33:08.043860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033618+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:08.033618+0000","last_peered":"2023-10-16T14:33:08.033618+0000","last_clean":"2023-10-16T14:33:08.033618+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:08.033618+0000","last_undegraded":"2023-10-16T14:33:08.033618+0000","last_fullsized":"2023-10-16T14:33:08.033618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626193+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:36.626193+0000","last_peered":"2023-10-16T14:33:36.626193+0000","last_clean":"2023-10-16T14:33:36.626193+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:36.626193+0000","last_undegraded":"2023-10-16T14:33:36.626193+0000","last_fullsized":"2023-10-16T14:33:36.626193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037942+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:08.037942+0000","last_peered":"2023-10-16T14:33:08.037942+0000","last_clean":"2023-10-16T14:33:08.037942+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:08.037942+0000","last_undegraded":"2023-10-16T14:33:08.037942+0000","last_fullsized":"2023-10-16T14:33:08.037942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:40.913037+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:40.913037+0000","last_peered":"2023-10-16T14:33:40.913037+0000","last_clean":"2023-10-16T14:33:40.913037+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:40.913037+0000","last_undegraded":"2023-10-16T14:33:40.913037+0000","last_fullsized":"2023-10-16T14:33:40.913037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638429+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:36.638429+0000","last_peered":"2023-10-16T14:33:36.638429+0000","last_clean":"2023-10-16T14:33:36.638429+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:36.638429+0000","last_undegraded":"2023-10-16T14:33:36.638429+0000","last_fullsized":"2023-10-16T14:33:36.638429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.289756+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:08.289756+0000","last_peered":"2023-10-16T14:33:08.289756+0000","last_clean":"2023-10-16T14:33:08.289756+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:08.289756+0000","last_undegraded":"2023-10-16T14:33:08.289756+0000","last_fullsized":"2023-10-16T14:33:08.289756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041225+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:08.041225+0000","last_peered":"2023-10-16T14:33:08.041225+0000","last_clean":"2023-10-16T14:33:08.041225+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:08.041225+0000","last_undegraded":"2023-10-16T14:33:08.041225+0000","last_fullsized":"2023-10-16T14:33:08.041225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641382+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:36.641382+0000","last_peered":"2023-10-16T14:33:36.641382+0000","last_clean":"2023-10-16T14:33:36.641382+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:36.641382+0000","last_undegraded":"2023-10-16T14:33:36.641382+0000","last_fullsized":"2023-10-16T14:33:36.641382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040831+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:08.040831+0000","last_peered":"2023-10-16T14:33:08.040831+0000","last_clean":"2023-10-16T14:33:08.040831+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:08.040831+0000","last_undegraded":"2023-10-16T14:33:08.040831+0000","last_fullsized":"2023-10-16T14:33:08.040831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454545+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:33:08.454545+0000","last_peered":"2023-10-16T14:33:08.454545+0000","last_clean":"2023-10-16T14:33:08.454545+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:33:08.454545+0000","last_undegraded":"2023-10-16T14:33:08.454545+0000","last_fullsized":"2023-10-16T14:33:08.454545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.622286+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:36.622286+0000","last_peered":"2023-10-16T14:33:36.622286+0000","last_clean":"2023-10-16T14:33:36.622286+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:36.622286+0000","last_undegraded":"2023-10-16T14:33:36.622286+0000","last_fullsized":"2023-10-16T14:33:36.622286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044222+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:08.044222+0000","last_peered":"2023-10-16T14:33:08.044222+0000","last_clean":"2023-10-16T14:33:08.044222+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:08.044222+0000","last_undegraded":"2023-10-16T14:33:08.044222+0000","last_fullsized":"2023-10-16T14:33:08.044222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038020+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:08.038020+0000","last_peered":"2023-10-16T14:33:08.038020+0000","last_clean":"2023-10-16T14:33:08.038020+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:08.038020+0000","last_undegraded":"2023-10-16T14:33:08.038020+0000","last_fullsized":"2023-10-16T14:33:08.038020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637193+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:36.637193+0000","last_peered":"2023-10-16T14:33:36.637193+0000","last_clean":"2023-10-16T14:33:36.637193+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637193+0000","last_undegraded":"2023-10-16T14:33:36.637193+0000","last_fullsized":"2023-10-16T14:33:36.637193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040273+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:08.040273+0000","last_peered":"2023-10-16T14:33:08.040273+0000","last_clean":"2023-10-16T14:33:08.040273+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:08.040273+0000","last_undegraded":"2023-10-16T14:33:08.040273+0000","last_fullsized":"2023-10-16T14:33:08.040273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:48.668945+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:48.668945+0000","last_peered":"2023-10-16T14:33:48.668945+0000","last_clean":"2023-10-16T14:33:48.668945+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:48.668945+0000","last_undegraded":"2023-10-16T14:33:48.668945+0000","last_fullsized":"2023-10-16T14:33:48.668945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640506+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:36.640506+0000","last_peered":"2023-10-16T14:33:36.640506+0000","last_clean":"2023-10-16T14:33:36.640506+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:36.640506+0000","last_undegraded":"2023-10-16T14:33:36.640506+0000","last_fullsized":"2023-10-16T14:33:36.640506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041196+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:08.041196+0000","last_peered":"2023-10-16T14:33:08.041196+0000","last_clean":"2023-10-16T14:33:08.041196+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:08.041196+0000","last_undegraded":"2023-10-16T14:33:08.041196+0000","last_fullsized":"2023-10-16T14:33:08.041196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033443+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:08.033443+0000","last_peered":"2023-10-16T14:33:08.033443+0000","last_clean":"2023-10-16T14:33:08.033443+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:08.033443+0000","last_undegraded":"2023-10-16T14:33:08.033443+0000","last_fullsized":"2023-10-16T14:33:08.033443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635668+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:36.635668+0000","last_peered":"2023-10-16T14:33:36.635668+0000","last_clean":"2023-10-16T14:33:36.635668+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:36.635668+0000","last_undegraded":"2023-10-16T14:33:36.635668+0000","last_fullsized":"2023-10-16T14:33:36.635668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454661+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:33:08.454661+0000","last_peered":"2023-10-16T14:33:08.454661+0000","last_clean":"2023-10-16T14:33:08.454661+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:33:08.454661+0000","last_undegraded":"2023-10-16T14:33:08.454661+0000","last_fullsized":"2023-10-16T14:33:08.454661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290521+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:08.290521+0000","last_peered":"2023-10-16T14:33:08.290521+0000","last_clean":"2023-10-16T14:33:08.290521+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:08.290521+0000","last_undegraded":"2023-10-16T14:33:08.290521+0000","last_fullsized":"2023-10-16T14:33:08.290521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637090+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:36.637090+0000","last_peered":"2023-10-16T14:33:36.637090+0000","last_clean":"2023-10-16T14:33:36.637090+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:36.637090+0000","last_undegraded":"2023-10-16T14:33:36.637090+0000","last_fullsized":"2023-10-16T14:33:36.637090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052803+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:08.052803+0000","last_peered":"2023-10-16T14:33:08.052803+0000","last_clean":"2023-10-16T14:33:08.052803+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:08.052803+0000","last_undegraded":"2023-10-16T14:33:08.052803+0000","last_fullsized":"2023-10-16T14:33:08.052803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044786+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:08.044786+0000","last_peered":"2023-10-16T14:33:08.044786+0000","last_clean":"2023-10-16T14:33:08.044786+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:08.044786+0000","last_undegraded":"2023-10-16T14:33:08.044786+0000","last_fullsized":"2023-10-16T14:33:08.044786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635462+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:36.635462+0000","last_peered":"2023-10-16T14:33:36.635462+0000","last_clean":"2023-10-16T14:33:36.635462+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:36.635462+0000","last_undegraded":"2023-10-16T14:33:36.635462+0000","last_fullsized":"2023-10-16T14:33:36.635462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044128+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:08.044128+0000","last_peered":"2023-10-16T14:33:08.044128+0000","last_clean":"2023-10-16T14:33:08.044128+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:08.044128+0000","last_undegraded":"2023-10-16T14:33:08.044128+0000","last_fullsized":"2023-10-16T14:33:08.044128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416659+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:08.416659+0000","last_peered":"2023-10-16T14:33:08.416659+0000","last_clean":"2023-10-16T14:33:08.416659+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:08.416659+0000","last_undegraded":"2023-10-16T14:33:08.416659+0000","last_fullsized":"2023-10-16T14:33:08.416659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631478+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:36.631478+0000","last_peered":"2023-10-16T14:33:36.631478+0000","last_clean":"2023-10-16T14:33:36.631478+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:36.631478+0000","last_undegraded":"2023-10-16T14:33:36.631478+0000","last_fullsized":"2023-10-16T14:33:36.631478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.625624+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:36.625624+0000","last_peered":"2023-10-16T14:33:36.625624+0000","last_clean":"2023-10-16T14:33:36.625624+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:36.625624+0000","last_undegraded":"2023-10-16T14:33:36.625624+0000","last_fullsized":"2023-10-16T14:33:36.625624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641472+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:36.641472+0000","last_peered":"2023-10-16T14:33:36.641472+0000","last_clean":"2023-10-16T14:33:36.641472+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:36.641472+0000","last_undegraded":"2023-10-16T14:33:36.641472+0000","last_fullsized":"2023-10-16T14:33:36.641472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636199+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:36.636199+0000","last_peered":"2023-10-16T14:33:36.636199+0000","last_clean":"2023-10-16T14:33:36.636199+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:36.636199+0000","last_undegraded":"2023-10-16T14:33:36.636199+0000","last_fullsized":"2023-10-16T14:33:36.636199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641467+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:36.641467+0000","last_peered":"2023-10-16T14:33:36.641467+0000","last_clean":"2023-10-16T14:33:36.641467+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:36.641467+0000","last_undegraded":"2023-10-16T14:33:36.641467+0000","last_fullsized":"2023-10-16T14:33:36.641467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639422+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:36.639422+0000","last_peered":"2023-10-16T14:33:36.639422+0000","last_clean":"2023-10-16T14:33:36.639422+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:36.639422+0000","last_undegraded":"2023-10-16T14:33:36.639422+0000","last_fullsized":"2023-10-16T14:33:36.639422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621053+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:36.621053+0000","last_peered":"2023-10-16T14:33:36.621053+0000","last_clean":"2023-10-16T14:33:36.621053+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:36.621053+0000","last_undegraded":"2023-10-16T14:33:36.621053+0000","last_fullsized":"2023-10-16T14:33:36.621053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626446+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:36.626446+0000","last_peered":"2023-10-16T14:33:36.626446+0000","last_clean":"2023-10-16T14:33:36.626446+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:36.626446+0000","last_undegraded":"2023-10-16T14:33:36.626446+0000","last_fullsized":"2023-10-16T14:33:36.626446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642770+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:36.642770+0000","last_peered":"2023-10-16T14:33:36.642770+0000","last_clean":"2023-10-16T14:33:36.642770+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:36.642770+0000","last_undegraded":"2023-10-16T14:33:36.642770+0000","last_fullsized":"2023-10-16T14:33:36.642770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636872+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:36.636872+0000","last_peered":"2023-10-16T14:33:36.636872+0000","last_clean":"2023-10-16T14:33:36.636872+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:36.636872+0000","last_undegraded":"2023-10-16T14:33:36.636872+0000","last_fullsized":"2023-10-16T14:33:36.636872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637998+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:36.637998+0000","last_peered":"2023-10-16T14:33:36.637998+0000","last_clean":"2023-10-16T14:33:36.637998+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:36.637998+0000","last_undegraded":"2023-10-16T14:33:36.637998+0000","last_fullsized":"2023-10-16T14:33:36.637998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642106+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:33:36.642106+0000","last_peered":"2023-10-16T14:33:36.642106+0000","last_clean":"2023-10-16T14:33:36.642106+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:33:36.642106+0000","last_undegraded":"2023-10-16T14:33:36.642106+0000","last_fullsized":"2023-10-16T14:33:36.642106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639941+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:36.639941+0000","last_peered":"2023-10-16T14:33:36.639941+0000","last_clean":"2023-10-16T14:33:36.639941+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:36.639941+0000","last_undegraded":"2023-10-16T14:33:36.639941+0000","last_fullsized":"2023-10-16T14:33:36.639941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631870+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:36.631870+0000","last_peered":"2023-10-16T14:33:36.631870+0000","last_clean":"2023-10-16T14:33:36.631870+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:36.631870+0000","last_undegraded":"2023-10-16T14:33:36.631870+0000","last_fullsized":"2023-10-16T14:33:36.631870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82100,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":778,"num_read_kb":475,"num_write":178,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449133,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97899999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.56999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.56399999999999995,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.436},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.81200000000000006,"5min":0.871,"15min":0.871},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.62,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.72299999999999998},"min":{"1min":0.59199999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.65100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.70899999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0189999999999999,"5min":1.256,"15min":1.256},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.70799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79700000000000004,"15min":0.78400000000000003},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.75900000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.996,"5min":1.214,"15min":1.214},"last":0.78200000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678064,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1724416,"data_stored":1163249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.997,"15min":0.997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498}]},{"osd":2,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82299999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52000000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.64000000000000001,"15min":0.65300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.83399999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77200000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93200000000000005,"5min":1.01,"15min":1.01},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73799999999999999,"15min":0.75},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":1.016,"15min":1.016},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.92400000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.49099999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.83099999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95699999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.752},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.55900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1120000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85599999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96499999999999997,"5min":1.038,"15min":1.038},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77200000000000002},"min":{"1min":0.59399999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.048,"15min":1.048},"last":0.78900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841590,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.51500000000000001,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59399999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.58899999999999997},"min":{"1min":0.34899999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61099999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69699999999999995,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91100000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.72699999999999998,"15min":0.79900000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81599999999999995,"5min":2.319,"15min":2.319},"last":0.80800000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.749,"15min":0.82899999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78500000000000003,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":1.849,"15min":1.849},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.68200000000000005,"15min":0.76200000000000001},"min":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.73899999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.753,"15min":0.82099999999999995},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88900000000000001,"5min":1.879,"15min":1.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.753,"15min":0.81599999999999995},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84299999999999997,"15min":0.93000000000000005},"min":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83299999999999996,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79500000000000004,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.877,"15min":0.97199999999999998},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1060000000000001,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.82599999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.84999999999999998,"15min":0.92700000000000005},"min":{"1min":0.57699999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.89200000000000002,"5min":1.944,"15min":1.944},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.83599999999999997,"15min":0.91100000000000003},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.86299999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.095,"5min":1.986,"15min":1.986},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.139,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.79200000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83099999999999996,"15min":0.82999999999999996},"min":{"1min":0.73499999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.016,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.93400000000000005,"15min":0.95699999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0429999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.67500000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103227,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.317,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67300000000000004,"5min":0.87,"15min":0.87},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.505},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59999999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67100000000000004},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67400000000000004,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.875,"5min":1.165,"15min":1.165},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.42499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81999999999999995,"5min":1.087,"15min":1.087},"last":0.39300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.47899999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":1.02,"15min":1.02},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.623}]},{"osd":7,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91400000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.78000000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.76700000000000002},"min":{"1min":0.55400000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64200000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.746,"15min":0.69899999999999995},"min":{"1min":0.66300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.006,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.075,"15min":1.075},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81599999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1113909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.625,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80700000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":2.589,"15min":2.589},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.432,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88700000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.63400000000000001},"min":{"1min":0.50700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.878,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84499999999999997,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.82199999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73299999999999998,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.96199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.879},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.86799999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.83699999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.60399999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.60499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.44800000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.497},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85699999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.748}]},{"osd":1,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81799999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83899999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.46000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75,"15min":0.73199999999999998},"min":{"1min":0.59199999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90800000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.64200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0880000000000001,"5min":1.232,"15min":1.232},"last":0.81000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.036,"5min":1.107,"15min":1.107},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93200000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0700000000000001,"5min":1.133,"15min":1.133},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0029999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":1.004,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.1220000000000001,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.90600000000000003},"min":{"1min":0.65900000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.105,"5min":1.157,"15min":1.157},"last":0.83999999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.85999999999999999},"min":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1060000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.4119999999999999,"15min":1.4990000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0680000000000001,"5min":11.394,"15min":11.394},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":566934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.34200000000000003},"max":{"1min":0.83299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.625,"15min":0.55100000000000005},"min":{"1min":0.55400000000000005,"5min":0.376,"15min":0.376},"max":{"1min":0.96599999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.60899999999999999,"15min":0.55100000000000005},"min":{"1min":0.499,"5min":0.309,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:34:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.502},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.53700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.42999999999999999},"min":{"1min":0.40500000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45700000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66700000000000004,"15min":0.625},"min":{"1min":0.57899999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.032,"5min":1.181,"15min":1.181},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.67200000000000004,"15min":0.59899999999999998},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":2.528,"15min":2.528},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.72699999999999998,"15min":0.68799999999999994},"min":{"1min":0.626,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.624},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.76600000000000001,"15min":0.71099999999999997},"min":{"1min":0.59699999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85999999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.56899999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0089999999999999,"5min":1.012,"15min":1.012},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.72099999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0129999999999999,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.72399999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81999999999999995,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77500000000000002,"15min":0.77200000000000002},"min":{"1min":0.47199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97599999999999998,"5min":1.054,"15min":1.054},"last":0.84199999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.89800000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.89700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.44700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"min":{"1min":0.55000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.141},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.60299999999999998},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95999999999999996,"5min":1.202,"15min":1.202},"last":0.88100000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.877,"15min":0.877},"last":0.435},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.754},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.629,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73599999999999999,"15min":0.66500000000000004},"min":{"1min":0.59699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.101,"15min":1.101},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.51900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.624,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92000000000000004,"5min":1.298,"15min":1.298},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.46600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.03,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85599999999999998,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0700000000000001,"5min":1.274,"15min":1.274},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.59899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.216},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79900000000000004,"15min":0.86499999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89600000000000002,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0509999999999999,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":0.95799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.70299999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.75900000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1379999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1}]},{"osd":11,"last update":"Mon Oct 16 14:33:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.95999999999999996},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.115,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":7.2480000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":581570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.372,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78900000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.52700000000000002},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.32700000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":1.022,"15min":1.022},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.50900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.874},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.36399999999999999},"max":{"1min":0.70399999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.49199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.45700000000000002},"min":{"1min":0.33600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.38,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1180000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84699999999999998,"15min":0.83899999999999997},"min":{"1min":0.71499999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.93000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.59999999999999998},"min":{"1min":0.61599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.64000000000000001,"15min":0.57799999999999996},"min":{"1min":0.56899999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70599999999999996,"15min":0.68300000000000005},"min":{"1min":0.63200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.747},"min":{"1min":0.74199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.97899999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.625,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.79700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.075,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.95299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93000000000000005,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.75900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.433},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48999999999999999,"15min":0.45800000000000002},"min":{"1min":0.34200000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.498}]},{"osd":1,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80300000000000005,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.60399999999999998},"min":{"1min":0.59199999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92500000000000004,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.83999999999999997,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.38400000000000001},"max":{"1min":0.91900000000000004,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:33:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":0.79700000000000004,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.443},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.72699999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.407,"5min":0.83499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":10.314,"5min":10.314,"15min":10.314},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.627},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91700000000000004,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73299999999999998,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.88500000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.80400000000000005,"15min":0.76400000000000001},"min":{"1min":0.61199999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.624},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.56399999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.45200000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.751},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.68400000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.91900000000000004,"15min":0.90500000000000003},"min":{"1min":0.71899999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.53600000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.871,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":0.99299999999999999,"15min":0.90300000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":10.361000000000001,"5min":10.361000000000001,"15min":10.361000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41599999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.66200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.433,"15min":0.40500000000000003},"min":{"1min":0.34399999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.71799999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.752,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81899999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"min":{"1min":0.57099999999999995,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98499999999999999,"5min":1.002,"15min":1.002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.56200000000000006},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.91400000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64100000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.49099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73599999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96399999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.81999999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97799999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.69599999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.83799999999999997,"15min":0.83299999999999996},"min":{"1min":0.66200000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.68899999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74199999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.79400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88600000000000001,"5min":1.425,"15min":1.425},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.40000000000000002},"min":{"1min":0.38500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.58099999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.436,"15min":0.40999999999999998},"min":{"1min":0.32200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95899999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82899999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64800000000000002,"15min":0.58399999999999996},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.629,"15min":0.56599999999999995},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.49299999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.52200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:33:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.196},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67300000000000004,"15min":0.64700000000000002},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.64100000000000001},"min":{"1min":0.56499999999999995,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.54600000000000004,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71599999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.66500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.629,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81299999999999994}]},{"osd":11,"last update":"Mon Oct 16 14:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.034,"5min":1.103,"15min":1.103},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.66300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":14489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:34:17.930 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-10-16T14:34:17.931 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T14:34:17.932 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:17 smithi187 ceph-mon[135297]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-16T14:34:17.932 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:17 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1510386399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:17 smithi186 ceph-mon[134892]: pgmap v356: 129 pgs: 129 active+clean; 657 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-10-16T14:34:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:17 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1510386399' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 -- 172.21.15.178:0/3082377416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc100ea0 msgr2=0x7f6dcc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/3082377416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc1012c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009990 tx=0x7f6dbc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 -- 172.21.15.178:0/3082377416 shutdown_connections 2023-10-16T14:34:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/3082377416 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc102a40 0x7f6dcc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/3082377416 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc1012c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/3082377416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 0x7f6dcc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 -- 172.21.15.178:0/3082377416 >> 172.21.15.178:0/3082377416 conn(0x7f6dcc0fc420 msgr2=0x7f6dcc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:18.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 -- 172.21.15.178:0/3082377416 shutdown_connections 2023-10-16T14:34:18.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.155+0000 7f6dd21ff700 1 -- 172.21.15.178:0/3082377416 wait complete. 2023-10-16T14:34:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 Processor -- start 2023-10-16T14:34:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 -- start start 2023-10-16T14:34:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 0x7f6dcc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 0x7f6dcc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc073f80 con 0x7f6dcc100ea0 2023-10-16T14:34:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0740f0 con 0x7f6dcc102a40 2023-10-16T14:34:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.156+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1a0b60 con 0x7f6dcc102080 2023-10-16T14:34:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41018/0 (socket says 172.21.15.178:41018) 2023-10-16T14:34:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 -- 172.21.15.178:0/593903792 learned_addr learned my addr 172.21.15.178:0/593903792 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcaffd700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 0x7f6dcc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcbfff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 0x7f6dcc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 -- 172.21.15.178:0/593903792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 msgr2=0x7f6dcc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 0x7f6dcc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 -- 172.21.15.178:0/593903792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 msgr2=0x7f6dcc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 0x7f6dcc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 -- 172.21.15.178:0/593903792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0095f0 con 0x7f6dcc100ea0 2023-10-16T14:34:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcbfff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 0x7f6dcc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:18.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.157+0000 7f6dcb7fe700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.158+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc01d070 con 0x7f6dcc100ea0 2023-10-16T14:34:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.158+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc1a0de0 con 0x7f6dcc100ea0 2023-10-16T14:34:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.158+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dbc022470 con 0x7f6dcc100ea0 2023-10-16T14:34:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.158+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc017650 con 0x7f6dcc100ea0 2023-10-16T14:34:18.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.158+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1a11b0 con 0x7f6dcc100ea0 2023-10-16T14:34:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.159+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc107db0 con 0x7f6dcc100ea0 2023-10-16T14:34:18.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.161+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6dbc0225e0 con 0x7f6dcc100ea0 2023-10-16T14:34:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.162+0000 7f6dc8ff9700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 0x7f6db40a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.162+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6dbc0ce420 con 0x7f6dcc100ea0 2023-10-16T14:34:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.162+0000 7f6dcaffd700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 0x7f6db40a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.163+0000 7f6dcaffd700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 0x7f6db40a39e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc100bc0 tx=0x7f6dc0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:18.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.166+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6dbc05d100 con 0x7f6dcc100ea0 2023-10-16T14:34:18.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.363+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 --> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6dcc02d2f0 con 0x7f6db40a1520 2023-10-16T14:34:18.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.376+0000 7f6dc8ff9700 1 -- 172.21.15.178:0/593903792 <== mgr.14148 v2:172.21.15.178:6800/706630784 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427827 (secure 0 0 0) 0x7f6dcc02d2f0 con 0x7f6db40a1520 2023-10-16T14:34:18.385 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 msgr2=0x7f6db40a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 0x7f6db40a39e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc100bc0 tx=0x7f6dc0009260 comp rx=0 tx=0).stop 2023-10-16T14:34:18.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 msgr2=0x7f6dcc074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc000c00 tx=0x7f6dbc005720 comp rx=0 tx=0).stop 2023-10-16T14:34:18.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 shutdown_connections 2023-10-16T14:34:18.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6db40a1520 0x7f6db40a39e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6dcc102a40 0x7f6dcc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6dcc100ea0 0x7f6dcc074c60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 --2- 172.21.15.178:0/593903792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6dcc102080 0x7f6dcc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 >> 172.21.15.178:0/593903792 conn(0x7f6dcc0fc420 msgr2=0x7f6dcc0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:18.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.381+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 shutdown_connections 2023-10-16T14:34:18.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.382+0000 7f6dd21ff700 1 -- 172.21.15.178:0/593903792 wait complete. 2023-10-16T14:34:18.392 INFO:teuthology.orchestra.run.smithi178.stderr:dumped all 2023-10-16T14:34:18.405 INFO:teuthology.orchestra.run.smithi178.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2023-10-16T14:34:16.630146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":672714,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":904,"num_read_kb":584,"num_write":361,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1181441,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":793,"ondisk_log_size":793,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338336,"kb_used_data":14144,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603488,"statfs":{"total":1151940427776,"available":1151593971712,"internally_reserved":0,"allocated":14483456,"data_stored":8068243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19132,"internal_metadata":331855172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2484,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":77,"num_read_kb":39,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637229+0000","last_change":"2023-10-16T14:31:37.045482+0000","last_active":"2023-10-16T14:33:36.637229+0000","last_peered":"2023-10-16T14:33:36.637229+0000","last_clean":"2023-10-16T14:33:36.637229+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637229+0000","last_undegraded":"2023-10-16T14:33:36.637229+0000","last_fullsized":"2023-10-16T14:33:36.637229+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:41.972902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641049+0000","last_change":"2023-10-16T14:31:37.043394+0000","last_active":"2023-10-16T14:33:36.641049+0000","last_peered":"2023-10-16T14:33:36.641049+0000","last_clean":"2023-10-16T14:33:36.641049+0000","last_became_active":"2023-10-16T14:31:37.040383+0000","last_became_peered":"2023-10-16T14:31:37.040383+0000","last_unstale":"2023-10-16T14:33:36.641049+0000","last_undegraded":"2023-10-16T14:33:36.641049+0000","last_fullsized":"2023-10-16T14:33:36.641049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:19:37.221591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636867+0000","last_change":"2023-10-16T14:31:37.040022+0000","last_active":"2023-10-16T14:33:36.636867+0000","last_peered":"2023-10-16T14:33:36.636867+0000","last_clean":"2023-10-16T14:33:36.636867+0000","last_became_active":"2023-10-16T14:31:37.039382+0000","last_became_peered":"2023-10-16T14:31:37.039382+0000","last_unstale":"2023-10-16T14:33:36.636867+0000","last_undegraded":"2023-10-16T14:33:36.636867+0000","last_fullsized":"2023-10-16T14:33:36.636867+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:58:21.228077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638334+0000","last_change":"2023-10-16T14:31:37.043096+0000","last_active":"2023-10-16T14:33:36.638334+0000","last_peered":"2023-10-16T14:33:36.638334+0000","last_clean":"2023-10-16T14:33:36.638334+0000","last_became_active":"2023-10-16T14:31:37.042781+0000","last_became_peered":"2023-10-16T14:31:37.042781+0000","last_unstale":"2023-10-16T14:33:36.638334+0000","last_undegraded":"2023-10-16T14:33:36.638334+0000","last_fullsized":"2023-10-16T14:33:36.638334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:31:25.963868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636004+0000","last_change":"2023-10-16T14:31:37.041970+0000","last_active":"2023-10-16T14:33:36.636004+0000","last_peered":"2023-10-16T14:33:36.636004+0000","last_clean":"2023-10-16T14:33:36.636004+0000","last_became_active":"2023-10-16T14:31:37.041021+0000","last_became_peered":"2023-10-16T14:31:37.041021+0000","last_unstale":"2023-10-16T14:33:36.636004+0000","last_undegraded":"2023-10-16T14:33:36.636004+0000","last_fullsized":"2023-10-16T14:33:36.636004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:43.812555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642338+0000","last_change":"2023-10-16T14:31:37.038127+0000","last_active":"2023-10-16T14:33:36.642338+0000","last_peered":"2023-10-16T14:33:36.642338+0000","last_clean":"2023-10-16T14:33:36.642338+0000","last_became_active":"2023-10-16T14:31:37.037805+0000","last_became_peered":"2023-10-16T14:31:37.037805+0000","last_unstale":"2023-10-16T14:33:36.642338+0000","last_undegraded":"2023-10-16T14:33:36.642338+0000","last_fullsized":"2023-10-16T14:33:36.642338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:16:22.777535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627444+0000","last_change":"2023-10-16T14:31:37.037849+0000","last_active":"2023-10-16T14:33:36.627444+0000","last_peered":"2023-10-16T14:33:36.627444+0000","last_clean":"2023-10-16T14:33:36.627444+0000","last_became_active":"2023-10-16T14:31:37.037510+0000","last_became_peered":"2023-10-16T14:31:37.037510+0000","last_unstale":"2023-10-16T14:33:36.627444+0000","last_undegraded":"2023-10-16T14:33:36.627444+0000","last_fullsized":"2023-10-16T14:33:36.627444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:21:19.828571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639455+0000","last_change":"2023-10-16T14:31:37.033127+0000","last_active":"2023-10-16T14:33:36.639455+0000","last_peered":"2023-10-16T14:33:36.639455+0000","last_clean":"2023-10-16T14:33:36.639455+0000","last_became_active":"2023-10-16T14:31:37.028935+0000","last_became_peered":"2023-10-16T14:31:37.028935+0000","last_unstale":"2023-10-16T14:33:36.639455+0000","last_undegraded":"2023-10-16T14:33:36.639455+0000","last_fullsized":"2023-10-16T14:33:36.639455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:48:04.959653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626801+0000","last_change":"2023-10-16T14:31:37.039269+0000","last_active":"2023-10-16T14:33:36.626801+0000","last_peered":"2023-10-16T14:33:36.626801+0000","last_clean":"2023-10-16T14:33:36.626801+0000","last_became_active":"2023-10-16T14:31:37.038966+0000","last_became_peered":"2023-10-16T14:31:37.038966+0000","last_unstale":"2023-10-16T14:33:36.626801+0000","last_undegraded":"2023-10-16T14:33:36.626801+0000","last_fullsized":"2023-10-16T14:33:36.626801+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:23:08.628107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642427+0000","last_change":"2023-10-16T14:31:37.035128+0000","last_active":"2023-10-16T14:33:36.642427+0000","last_peered":"2023-10-16T14:33:36.642427+0000","last_clean":"2023-10-16T14:33:36.642427+0000","last_became_active":"2023-10-16T14:31:37.033870+0000","last_became_peered":"2023-10-16T14:31:37.033870+0000","last_unstale":"2023-10-16T14:33:36.642427+0000","last_undegraded":"2023-10-16T14:33:36.642427+0000","last_fullsized":"2023-10-16T14:33:36.642427+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:00.586091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637246+0000","last_change":"2023-10-16T14:31:37.039675+0000","last_active":"2023-10-16T14:33:36.637246+0000","last_peered":"2023-10-16T14:33:36.637246+0000","last_clean":"2023-10-16T14:33:36.637246+0000","last_became_active":"2023-10-16T14:31:37.039351+0000","last_became_peered":"2023-10-16T14:31:37.039351+0000","last_unstale":"2023-10-16T14:33:36.637246+0000","last_undegraded":"2023-10-16T14:33:36.637246+0000","last_fullsized":"2023-10-16T14:33:36.637246+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:35.424503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639803+0000","last_change":"2023-10-16T14:31:37.042906+0000","last_active":"2023-10-16T14:33:36.639803+0000","last_peered":"2023-10-16T14:33:36.639803+0000","last_clean":"2023-10-16T14:33:36.639803+0000","last_became_active":"2023-10-16T14:31:37.042494+0000","last_became_peered":"2023-10-16T14:31:37.042494+0000","last_unstale":"2023-10-16T14:33:36.639803+0000","last_undegraded":"2023-10-16T14:33:36.639803+0000","last_fullsized":"2023-10-16T14:33:36.639803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:39.714142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641688+0000","last_change":"2023-10-16T14:31:37.031183+0000","last_active":"2023-10-16T14:33:36.641688+0000","last_peered":"2023-10-16T14:33:36.641688+0000","last_clean":"2023-10-16T14:33:36.641688+0000","last_became_active":"2023-10-16T14:31:37.030777+0000","last_became_peered":"2023-10-16T14:31:37.030777+0000","last_unstale":"2023-10-16T14:33:36.641688+0000","last_undegraded":"2023-10-16T14:33:36.641688+0000","last_fullsized":"2023-10-16T14:33:36.641688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:40.494830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641600+0000","last_change":"2023-10-16T14:31:37.041584+0000","last_active":"2023-10-16T14:33:36.641600+0000","last_peered":"2023-10-16T14:33:36.641600+0000","last_clean":"2023-10-16T14:33:36.641600+0000","last_became_active":"2023-10-16T14:31:37.041246+0000","last_became_peered":"2023-10-16T14:31:37.041246+0000","last_unstale":"2023-10-16T14:33:36.641600+0000","last_undegraded":"2023-10-16T14:33:36.641600+0000","last_fullsized":"2023-10-16T14:33:36.641600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:26:14.281941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641196+0000","last_change":"2023-10-16T14:31:37.042180+0000","last_active":"2023-10-16T14:33:36.641196+0000","last_peered":"2023-10-16T14:33:36.641196+0000","last_clean":"2023-10-16T14:33:36.641196+0000","last_became_active":"2023-10-16T14:31:37.041055+0000","last_became_peered":"2023-10-16T14:31:37.041055+0000","last_unstale":"2023-10-16T14:33:36.641196+0000","last_undegraded":"2023-10-16T14:33:36.641196+0000","last_fullsized":"2023-10-16T14:33:36.641196+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:03:20.239650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.040392+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.040078+0000","last_became_peered":"2023-10-16T14:31:37.040078+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:43:11.345814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637421+0000","last_change":"2023-10-16T14:31:37.045220+0000","last_active":"2023-10-16T14:33:36.637421+0000","last_peered":"2023-10-16T14:33:36.637421+0000","last_clean":"2023-10-16T14:33:36.637421+0000","last_became_active":"2023-10-16T14:31:37.044761+0000","last_became_peered":"2023-10-16T14:31:37.044761+0000","last_unstale":"2023-10-16T14:33:36.637421+0000","last_undegraded":"2023-10-16T14:33:36.637421+0000","last_fullsized":"2023-10-16T14:33:36.637421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:17:02.593946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637900+0000","last_change":"2023-10-16T14:31:37.046899+0000","last_active":"2023-10-16T14:33:36.637900+0000","last_peered":"2023-10-16T14:33:36.637900+0000","last_clean":"2023-10-16T14:33:36.637900+0000","last_became_active":"2023-10-16T14:31:37.043889+0000","last_became_peered":"2023-10-16T14:31:37.043889+0000","last_unstale":"2023-10-16T14:33:36.637900+0000","last_undegraded":"2023-10-16T14:33:36.637900+0000","last_fullsized":"2023-10-16T14:33:36.637900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:24:53.823173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621257+0000","last_change":"2023-10-16T14:31:37.038120+0000","last_active":"2023-10-16T14:33:36.621257+0000","last_peered":"2023-10-16T14:33:36.621257+0000","last_clean":"2023-10-16T14:33:36.621257+0000","last_became_active":"2023-10-16T14:31:37.037817+0000","last_became_peered":"2023-10-16T14:31:37.037817+0000","last_unstale":"2023-10-16T14:33:36.621257+0000","last_undegraded":"2023-10-16T14:33:36.621257+0000","last_fullsized":"2023-10-16T14:33:36.621257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:27:54.832794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290036+0000","last_change":"2023-10-16T14:31:41.090681+0000","last_active":"2023-10-16T14:33:08.290036+0000","last_peered":"2023-10-16T14:33:08.290036+0000","last_clean":"2023-10-16T14:33:08.290036+0000","last_became_active":"2023-10-16T14:31:41.090220+0000","last_became_peered":"2023-10-16T14:31:41.090220+0000","last_unstale":"2023-10-16T14:33:08.290036+0000","last_undegraded":"2023-10-16T14:33:08.290036+0000","last_fullsized":"2023-10-16T14:33:08.290036+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:51:11.497132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'9","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044284+0000","last_change":"2023-10-16T14:32:07.693357+0000","last_active":"2023-10-16T14:33:08.044284+0000","last_peered":"2023-10-16T14:33:08.044284+0000","last_clean":"2023-10-16T14:33:08.044284+0000","last_became_active":"2023-10-16T14:32:07.687814+0000","last_became_peered":"2023-10-16T14:32:07.687814+0000","last_unstale":"2023-10-16T14:33:08.044284+0000","last_undegraded":"2023-10-16T14:33:08.044284+0000","last_fullsized":"2023-10-16T14:33:08.044284+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:54:08.391183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638964+0000","last_change":"2023-10-16T14:31:37.034271+0000","last_active":"2023-10-16T14:33:36.638964+0000","last_peered":"2023-10-16T14:33:36.638964+0000","last_clean":"2023-10-16T14:33:36.638964+0000","last_became_active":"2023-10-16T14:31:37.033971+0000","last_became_peered":"2023-10-16T14:31:37.033971+0000","last_unstale":"2023-10-16T14:33:36.638964+0000","last_undegraded":"2023-10-16T14:33:36.638964+0000","last_fullsized":"2023-10-16T14:33:36.638964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:04.864183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.039726+0000","last_change":"2023-10-16T14:31:41.095430+0000","last_active":"2023-10-16T14:33:08.039726+0000","last_peered":"2023-10-16T14:33:08.039726+0000","last_clean":"2023-10-16T14:33:08.039726+0000","last_became_active":"2023-10-16T14:31:41.095128+0000","last_became_peered":"2023-10-16T14:31:41.095128+0000","last_unstale":"2023-10-16T14:33:08.039726+0000","last_undegraded":"2023-10-16T14:33:08.039726+0000","last_fullsized":"2023-10-16T14:33:08.039726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:40:51.779377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038574+0000","last_change":"2023-10-16T14:31:36.059928+0000","last_active":"2023-10-16T14:33:08.038574+0000","last_peered":"2023-10-16T14:33:08.038574+0000","last_clean":"2023-10-16T14:33:08.038574+0000","last_became_active":"2023-10-16T14:31:36.059629+0000","last_became_peered":"2023-10-16T14:31:36.059629+0000","last_unstale":"2023-10-16T14:33:08.038574+0000","last_undegraded":"2023-10-16T14:33:08.038574+0000","last_fullsized":"2023-10-16T14:33:08.038574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:47:00.498231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636879+0000","last_change":"2023-10-16T14:31:37.047435+0000","last_active":"2023-10-16T14:33:36.636879+0000","last_peered":"2023-10-16T14:33:36.636879+0000","last_clean":"2023-10-16T14:33:36.636879+0000","last_became_active":"2023-10-16T14:31:37.046941+0000","last_became_peered":"2023-10-16T14:31:37.046941+0000","last_unstale":"2023-10-16T14:33:36.636879+0000","last_undegraded":"2023-10-16T14:33:36.636879+0000","last_fullsized":"2023-10-16T14:33:36.636879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:31:28.016495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040318+0000","last_change":"2023-10-16T14:31:41.090776+0000","last_active":"2023-10-16T14:33:08.040318+0000","last_peered":"2023-10-16T14:33:08.040318+0000","last_clean":"2023-10-16T14:33:08.040318+0000","last_became_active":"2023-10-16T14:31:41.090415+0000","last_became_peered":"2023-10-16T14:31:41.090415+0000","last_unstale":"2023-10-16T14:33:08.040318+0000","last_undegraded":"2023-10-16T14:33:08.040318+0000","last_fullsized":"2023-10-16T14:33:08.040318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:43.864249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033698+0000","last_change":"2023-10-16T14:31:36.047494+0000","last_active":"2023-10-16T14:33:08.033698+0000","last_peered":"2023-10-16T14:33:08.033698+0000","last_clean":"2023-10-16T14:33:08.033698+0000","last_became_active":"2023-10-16T14:31:36.047179+0000","last_became_peered":"2023-10-16T14:31:36.047179+0000","last_unstale":"2023-10-16T14:33:08.033698+0000","last_undegraded":"2023-10-16T14:33:08.033698+0000","last_fullsized":"2023-10-16T14:33:08.033698+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:17:54.336085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638148+0000","last_change":"2023-10-16T14:31:37.040083+0000","last_active":"2023-10-16T14:33:36.638148+0000","last_peered":"2023-10-16T14:33:36.638148+0000","last_clean":"2023-10-16T14:33:36.638148+0000","last_became_active":"2023-10-16T14:31:37.039757+0000","last_became_peered":"2023-10-16T14:31:37.039757+0000","last_unstale":"2023-10-16T14:33:36.638148+0000","last_undegraded":"2023-10-16T14:33:36.638148+0000","last_fullsized":"2023-10-16T14:33:36.638148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:07:35.466325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.418372+0000","last_change":"2023-10-16T14:33:08.417921+0000","last_active":"2023-10-16T14:33:08.418372+0000","last_peered":"2023-10-16T14:33:08.418372+0000","last_clean":"2023-10-16T14:33:08.418372+0000","last_became_active":"2023-10-16T14:33:08.417342+0000","last_became_peered":"2023-10-16T14:33:08.417342+0000","last_unstale":"2023-10-16T14:33:08.418372+0000","last_undegraded":"2023-10-16T14:33:08.418372+0000","last_fullsized":"2023-10-16T14:33:08.418372+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:48:54.187220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-16T14:33:07.043274+0000","last_change":"2023-10-16T14:32:07.691725+0000","last_active":"2023-10-16T14:33:07.043274+0000","last_peered":"2023-10-16T14:33:07.043274+0000","last_clean":"2023-10-16T14:33:07.043274+0000","last_became_active":"2023-10-16T14:32:07.687917+0000","last_became_peered":"2023-10-16T14:32:07.687917+0000","last_unstale":"2023-10-16T14:33:07.043274+0000","last_undegraded":"2023-10-16T14:33:07.043274+0000","last_fullsized":"2023-10-16T14:33:07.043274+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:59:49.914507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635989+0000","last_change":"2023-10-16T14:31:37.039847+0000","last_active":"2023-10-16T14:33:36.635989+0000","last_peered":"2023-10-16T14:33:36.635989+0000","last_clean":"2023-10-16T14:33:36.635989+0000","last_became_active":"2023-10-16T14:31:37.039300+0000","last_became_peered":"2023-10-16T14:31:37.039300+0000","last_unstale":"2023-10-16T14:33:36.635989+0000","last_undegraded":"2023-10-16T14:33:36.635989+0000","last_fullsized":"2023-10-16T14:33:36.635989+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:59.581598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290088+0000","last_change":"2023-10-16T14:31:41.080745+0000","last_active":"2023-10-16T14:33:08.290088+0000","last_peered":"2023-10-16T14:33:08.290088+0000","last_clean":"2023-10-16T14:33:08.290088+0000","last_became_active":"2023-10-16T14:31:41.080411+0000","last_became_peered":"2023-10-16T14:31:41.080411+0000","last_unstale":"2023-10-16T14:33:08.290088+0000","last_undegraded":"2023-10-16T14:33:08.290088+0000","last_fullsized":"2023-10-16T14:33:08.290088+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:13.322878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'12","reported_seq":166,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:13.904131+0000","last_change":"2023-10-16T14:31:36.049087+0000","last_active":"2023-10-16T14:34:13.904131+0000","last_peered":"2023-10-16T14:34:13.904131+0000","last_clean":"2023-10-16T14:34:13.904131+0000","last_became_active":"2023-10-16T14:31:36.048609+0000","last_became_peered":"2023-10-16T14:31:36.048609+0000","last_unstale":"2023-10-16T14:34:13.904131+0000","last_undegraded":"2023-10-16T14:34:13.904131+0000","last_fullsized":"2023-10-16T14:34:13.904131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:06:45.368790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":17,"num_write":9,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.618324+0000","last_change":"2023-10-16T14:31:37.040168+0000","last_active":"2023-10-16T14:33:36.618324+0000","last_peered":"2023-10-16T14:33:36.618324+0000","last_clean":"2023-10-16T14:33:36.618324+0000","last_became_active":"2023-10-16T14:31:37.039866+0000","last_became_peered":"2023-10-16T14:31:37.039866+0000","last_unstale":"2023-10-16T14:33:36.618324+0000","last_undegraded":"2023-10-16T14:33:36.618324+0000","last_fullsized":"2023-10-16T14:33:36.618324+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:47:30.848506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290297+0000","last_change":"2023-10-16T14:31:41.083910+0000","last_active":"2023-10-16T14:33:08.290297+0000","last_peered":"2023-10-16T14:33:08.290297+0000","last_clean":"2023-10-16T14:33:08.290297+0000","last_became_active":"2023-10-16T14:31:41.083557+0000","last_became_peered":"2023-10-16T14:31:41.083557+0000","last_unstale":"2023-10-16T14:33:08.290297+0000","last_undegraded":"2023-10-16T14:33:08.290297+0000","last_fullsized":"2023-10-16T14:33:08.290297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:43.676228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037660+0000","last_change":"2023-10-16T14:32:07.688550+0000","last_active":"2023-10-16T14:33:08.037660+0000","last_peered":"2023-10-16T14:33:08.037660+0000","last_clean":"2023-10-16T14:33:08.037660+0000","last_became_active":"2023-10-16T14:32:07.687948+0000","last_became_peered":"2023-10-16T14:32:07.687948+0000","last_unstale":"2023-10-16T14:33:08.037660+0000","last_undegraded":"2023-10-16T14:33:08.037660+0000","last_fullsized":"2023-10-16T14:33:08.037660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:45.514049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641111+0000","last_change":"2023-10-16T14:31:37.033839+0000","last_active":"2023-10-16T14:33:36.641111+0000","last_peered":"2023-10-16T14:33:36.641111+0000","last_clean":"2023-10-16T14:33:36.641111+0000","last_became_active":"2023-10-16T14:31:37.033441+0000","last_became_peered":"2023-10-16T14:31:37.033441+0000","last_unstale":"2023-10-16T14:33:36.641111+0000","last_undegraded":"2023-10-16T14:33:36.641111+0000","last_fullsized":"2023-10-16T14:33:36.641111+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:13:24.140677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044215+0000","last_change":"2023-10-16T14:31:41.089990+0000","last_active":"2023-10-16T14:33:08.044215+0000","last_peered":"2023-10-16T14:33:08.044215+0000","last_clean":"2023-10-16T14:33:08.044215+0000","last_became_active":"2023-10-16T14:31:41.089554+0000","last_became_peered":"2023-10-16T14:31:41.089554+0000","last_unstale":"2023-10-16T14:33:08.044215+0000","last_undegraded":"2023-10-16T14:33:08.044215+0000","last_fullsized":"2023-10-16T14:33:08.044215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:49.550272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416188+0000","last_change":"2023-10-16T14:31:36.051744+0000","last_active":"2023-10-16T14:33:08.416188+0000","last_peered":"2023-10-16T14:33:08.416188+0000","last_clean":"2023-10-16T14:33:08.416188+0000","last_became_active":"2023-10-16T14:31:36.051344+0000","last_became_peered":"2023-10-16T14:31:36.051344+0000","last_unstale":"2023-10-16T14:33:08.416188+0000","last_undegraded":"2023-10-16T14:33:08.416188+0000","last_fullsized":"2023-10-16T14:33:08.416188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:09:16.958436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638397+0000","last_change":"2023-10-16T14:31:37.043299+0000","last_active":"2023-10-16T14:33:36.638397+0000","last_peered":"2023-10-16T14:33:36.638397+0000","last_clean":"2023-10-16T14:33:36.638397+0000","last_became_active":"2023-10-16T14:31:37.041104+0000","last_became_peered":"2023-10-16T14:31:37.041104+0000","last_unstale":"2023-10-16T14:33:36.638397+0000","last_undegraded":"2023-10-16T14:33:36.638397+0000","last_fullsized":"2023-10-16T14:33:36.638397+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:20:20.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052948+0000","last_change":"2023-10-16T14:31:41.098398+0000","last_active":"2023-10-16T14:33:08.052948+0000","last_peered":"2023-10-16T14:33:08.052948+0000","last_clean":"2023-10-16T14:33:08.052948+0000","last_became_active":"2023-10-16T14:31:41.097955+0000","last_became_peered":"2023-10-16T14:31:41.097955+0000","last_unstale":"2023-10-16T14:33:08.052948+0000","last_undegraded":"2023-10-16T14:33:08.052948+0000","last_fullsized":"2023-10-16T14:33:08.052948+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:40:22.910899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044478+0000","last_change":"2023-10-16T14:31:36.044410+0000","last_active":"2023-10-16T14:33:08.044478+0000","last_peered":"2023-10-16T14:33:08.044478+0000","last_clean":"2023-10-16T14:33:08.044478+0000","last_became_active":"2023-10-16T14:31:36.044093+0000","last_became_peered":"2023-10-16T14:31:36.044093+0000","last_unstale":"2023-10-16T14:33:08.044478+0000","last_undegraded":"2023-10-16T14:33:08.044478+0000","last_fullsized":"2023-10-16T14:33:08.044478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:13:24.116250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640406+0000","last_change":"2023-10-16T14:31:37.041875+0000","last_active":"2023-10-16T14:33:36.640406+0000","last_peered":"2023-10-16T14:33:36.640406+0000","last_clean":"2023-10-16T14:33:36.640406+0000","last_became_active":"2023-10-16T14:31:37.040364+0000","last_became_peered":"2023-10-16T14:31:37.040364+0000","last_unstale":"2023-10-16T14:33:36.640406+0000","last_undegraded":"2023-10-16T14:33:36.640406+0000","last_fullsized":"2023-10-16T14:33:36.640406+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:38:31.939431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038135+0000","last_change":"2023-10-16T14:31:41.089101+0000","last_active":"2023-10-16T14:33:08.038135+0000","last_peered":"2023-10-16T14:33:08.038135+0000","last_clean":"2023-10-16T14:33:08.038135+0000","last_became_active":"2023-10-16T14:31:41.088734+0000","last_became_peered":"2023-10-16T14:31:41.088734+0000","last_unstale":"2023-10-16T14:33:08.038135+0000","last_undegraded":"2023-10-16T14:33:08.038135+0000","last_fullsized":"2023-10-16T14:33:08.038135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:08.135935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'5","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.542711+0000","last_change":"2023-10-16T14:31:36.048260+0000","last_active":"2023-10-16T14:34:11.542711+0000","last_peered":"2023-10-16T14:34:11.542711+0000","last_clean":"2023-10-16T14:34:11.542711+0000","last_became_active":"2023-10-16T14:31:36.047954+0000","last_became_peered":"2023-10-16T14:31:36.047954+0000","last_unstale":"2023-10-16T14:34:11.542711+0000","last_undegraded":"2023-10-16T14:34:11.542711+0000","last_fullsized":"2023-10-16T14:34:11.542711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:00.831721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":123,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638618+0000","last_change":"2023-10-16T14:31:37.045821+0000","last_active":"2023-10-16T14:33:36.638618+0000","last_peered":"2023-10-16T14:33:36.638618+0000","last_clean":"2023-10-16T14:33:36.638618+0000","last_became_active":"2023-10-16T14:31:37.045453+0000","last_became_peered":"2023-10-16T14:31:37.045453+0000","last_unstale":"2023-10-16T14:33:36.638618+0000","last_undegraded":"2023-10-16T14:33:36.638618+0000","last_fullsized":"2023-10-16T14:33:36.638618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:13.267260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043503+0000","last_change":"2023-10-16T14:31:41.095949+0000","last_active":"2023-10-16T14:33:08.043503+0000","last_peered":"2023-10-16T14:33:08.043503+0000","last_clean":"2023-10-16T14:33:08.043503+0000","last_became_active":"2023-10-16T14:31:41.094265+0000","last_became_peered":"2023-10-16T14:31:41.094265+0000","last_unstale":"2023-10-16T14:33:08.043503+0000","last_undegraded":"2023-10-16T14:33:08.043503+0000","last_fullsized":"2023-10-16T14:33:08.043503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:44.726670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'2","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041325+0000","last_change":"2023-10-16T14:32:07.922742+0000","last_active":"2023-10-16T14:33:08.041325+0000","last_peered":"2023-10-16T14:33:08.041325+0000","last_clean":"2023-10-16T14:33:08.041325+0000","last_became_active":"2023-10-16T14:32:07.921422+0000","last_became_peered":"2023-10-16T14:32:07.921422+0000","last_unstale":"2023-10-16T14:33:08.041325+0000","last_undegraded":"2023-10-16T14:33:08.041325+0000","last_fullsized":"2023-10-16T14:33:08.041325+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:18:25.014122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.643248+0000","last_change":"2023-10-16T14:31:37.041673+0000","last_active":"2023-10-16T14:33:36.643248+0000","last_peered":"2023-10-16T14:33:36.643248+0000","last_clean":"2023-10-16T14:33:36.643248+0000","last_became_active":"2023-10-16T14:31:37.041355+0000","last_became_peered":"2023-10-16T14:31:37.041355+0000","last_unstale":"2023-10-16T14:33:36.643248+0000","last_undegraded":"2023-10-16T14:33:36.643248+0000","last_fullsized":"2023-10-16T14:33:36.643248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:57:04.042995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040462+0000","last_change":"2023-10-16T14:31:41.085871+0000","last_active":"2023-10-16T14:33:08.040462+0000","last_peered":"2023-10-16T14:33:08.040462+0000","last_clean":"2023-10-16T14:33:08.040462+0000","last_became_active":"2023-10-16T14:31:41.085472+0000","last_became_peered":"2023-10-16T14:31:41.085472+0000","last_unstale":"2023-10-16T14:33:08.040462+0000","last_undegraded":"2023-10-16T14:33:08.040462+0000","last_fullsized":"2023-10-16T14:33:08.040462+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:50:34.468914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043844+0000","last_change":"2023-10-16T14:31:36.055561+0000","last_active":"2023-10-16T14:33:08.043844+0000","last_peered":"2023-10-16T14:33:08.043844+0000","last_clean":"2023-10-16T14:33:08.043844+0000","last_became_active":"2023-10-16T14:31:36.055263+0000","last_became_peered":"2023-10-16T14:31:36.055263+0000","last_unstale":"2023-10-16T14:33:08.043844+0000","last_undegraded":"2023-10-16T14:33:08.043844+0000","last_fullsized":"2023-10-16T14:33:08.043844+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:51:03.502179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641090+0000","last_change":"2023-10-16T14:31:37.044234+0000","last_active":"2023-10-16T14:33:36.641090+0000","last_peered":"2023-10-16T14:33:36.641090+0000","last_clean":"2023-10-16T14:33:36.641090+0000","last_became_active":"2023-10-16T14:31:37.043973+0000","last_became_peered":"2023-10-16T14:31:37.043973+0000","last_unstale":"2023-10-16T14:33:36.641090+0000","last_undegraded":"2023-10-16T14:33:36.641090+0000","last_fullsized":"2023-10-16T14:33:36.641090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:03:36.958008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.045902+0000","last_change":"2023-10-16T14:33:08.045902+0000","last_active":"2023-10-16T14:33:08.045902+0000","last_peered":"2023-10-16T14:33:08.045902+0000","last_clean":"2023-10-16T14:33:08.045902+0000","last_became_active":"2023-10-16T14:33:08.045481+0000","last_became_peered":"2023-10-16T14:33:08.045481+0000","last_unstale":"2023-10-16T14:33:08.045902+0000","last_undegraded":"2023-10-16T14:33:08.045902+0000","last_fullsized":"2023-10-16T14:33:08.045902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:22:18.091442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":314,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:11.899607+0000","last_change":"2023-10-16T14:31:36.047160+0000","last_active":"2023-10-16T14:34:11.899607+0000","last_peered":"2023-10-16T14:34:11.899607+0000","last_clean":"2023-10-16T14:34:11.899607+0000","last_became_active":"2023-10-16T14:31:36.046804+0000","last_became_peered":"2023-10-16T14:31:36.046804+0000","last_unstale":"2023-10-16T14:34:11.899607+0000","last_undegraded":"2023-10-16T14:34:11.899607+0000","last_fullsized":"2023-10-16T14:34:11.899607+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:48:04.289976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65477,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":255,"num_read_kb":192,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.629796+0000","last_change":"2023-10-16T14:31:37.041259+0000","last_active":"2023-10-16T14:33:36.629796+0000","last_peered":"2023-10-16T14:33:36.629796+0000","last_clean":"2023-10-16T14:33:36.629796+0000","last_became_active":"2023-10-16T14:31:37.040951+0000","last_became_peered":"2023-10-16T14:31:37.040951+0000","last_unstale":"2023-10-16T14:33:36.629796+0000","last_undegraded":"2023-10-16T14:33:36.629796+0000","last_fullsized":"2023-10-16T14:33:36.629796+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:55:05.710357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044920+0000","last_change":"2023-10-16T14:31:41.095314+0000","last_active":"2023-10-16T14:33:08.044920+0000","last_peered":"2023-10-16T14:33:08.044920+0000","last_clean":"2023-10-16T14:33:08.044920+0000","last_became_active":"2023-10-16T14:31:41.094857+0000","last_became_peered":"2023-10-16T14:31:41.094857+0000","last_unstale":"2023-10-16T14:33:08.044920+0000","last_undegraded":"2023-10-16T14:33:08.044920+0000","last_fullsized":"2023-10-16T14:33:08.044920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:23:24.817700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290160+0000","last_change":"2023-10-16T14:31:36.044619+0000","last_active":"2023-10-16T14:33:08.290160+0000","last_peered":"2023-10-16T14:33:08.290160+0000","last_clean":"2023-10-16T14:33:08.290160+0000","last_became_active":"2023-10-16T14:31:36.043477+0000","last_became_peered":"2023-10-16T14:31:36.043477+0000","last_unstale":"2023-10-16T14:33:08.290160+0000","last_undegraded":"2023-10-16T14:33:08.290160+0000","last_fullsized":"2023-10-16T14:33:08.290160+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:03.285166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.632140+0000","last_change":"2023-10-16T14:31:37.034793+0000","last_active":"2023-10-16T14:33:36.632140+0000","last_peered":"2023-10-16T14:33:36.632140+0000","last_clean":"2023-10-16T14:33:36.632140+0000","last_became_active":"2023-10-16T14:31:37.034445+0000","last_became_peered":"2023-10-16T14:31:37.034445+0000","last_unstale":"2023-10-16T14:33:36.632140+0000","last_undegraded":"2023-10-16T14:33:36.632140+0000","last_fullsized":"2023-10-16T14:33:36.632140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:06.250864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044018+0000","last_change":"2023-10-16T14:31:41.095858+0000","last_active":"2023-10-16T14:33:08.044018+0000","last_peered":"2023-10-16T14:33:08.044018+0000","last_clean":"2023-10-16T14:33:08.044018+0000","last_became_active":"2023-10-16T14:31:41.095432+0000","last_became_peered":"2023-10-16T14:31:41.095432+0000","last_unstale":"2023-10-16T14:33:08.044018+0000","last_undegraded":"2023-10-16T14:33:08.044018+0000","last_fullsized":"2023-10-16T14:33:08.044018+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:40:21.373478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044199+0000","last_change":"2023-10-16T14:31:36.049697+0000","last_active":"2023-10-16T14:33:08.044199+0000","last_peered":"2023-10-16T14:33:08.044199+0000","last_clean":"2023-10-16T14:33:08.044199+0000","last_became_active":"2023-10-16T14:31:36.048716+0000","last_became_peered":"2023-10-16T14:31:36.048716+0000","last_unstale":"2023-10-16T14:33:08.044199+0000","last_undegraded":"2023-10-16T14:33:08.044199+0000","last_fullsized":"2023-10-16T14:33:08.044199+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:12:33.182135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642241+0000","last_change":"2023-10-16T14:31:37.034005+0000","last_active":"2023-10-16T14:33:36.642241+0000","last_peered":"2023-10-16T14:33:36.642241+0000","last_clean":"2023-10-16T14:33:36.642241+0000","last_became_active":"2023-10-16T14:31:37.033714+0000","last_became_peered":"2023-10-16T14:31:37.033714+0000","last_unstale":"2023-10-16T14:33:36.642241+0000","last_undegraded":"2023-10-16T14:33:36.642241+0000","last_fullsized":"2023-10-16T14:33:36.642241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:36:40.087010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043967+0000","last_change":"2023-10-16T14:31:41.096071+0000","last_active":"2023-10-16T14:33:08.043967+0000","last_peered":"2023-10-16T14:33:08.043967+0000","last_clean":"2023-10-16T14:33:08.043967+0000","last_became_active":"2023-10-16T14:31:41.093966+0000","last_became_peered":"2023-10-16T14:31:41.093966+0000","last_unstale":"2023-10-16T14:33:08.043967+0000","last_undegraded":"2023-10-16T14:33:08.043967+0000","last_fullsized":"2023-10-16T14:33:08.043967+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:21:42.678729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'3","reported_seq":152,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:12.900730+0000","last_change":"2023-10-16T14:31:36.045101+0000","last_active":"2023-10-16T14:34:12.900730+0000","last_peered":"2023-10-16T14:34:12.900730+0000","last_clean":"2023-10-16T14:34:12.900730+0000","last_became_active":"2023-10-16T14:31:36.044753+0000","last_became_peered":"2023-10-16T14:31:36.044753+0000","last_unstale":"2023-10-16T14:34:12.900730+0000","last_undegraded":"2023-10-16T14:34:12.900730+0000","last_fullsized":"2023-10-16T14:34:12.900730+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:24:54.205074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637695+0000","last_change":"2023-10-16T14:31:37.045333+0000","last_active":"2023-10-16T14:33:36.637695+0000","last_peered":"2023-10-16T14:33:36.637695+0000","last_clean":"2023-10-16T14:33:36.637695+0000","last_became_active":"2023-10-16T14:31:37.045044+0000","last_became_peered":"2023-10-16T14:31:37.045044+0000","last_unstale":"2023-10-16T14:33:36.637695+0000","last_undegraded":"2023-10-16T14:33:36.637695+0000","last_fullsized":"2023-10-16T14:33:36.637695+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:53:38.979995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416657+0000","last_change":"2023-10-16T14:31:41.089198+0000","last_active":"2023-10-16T14:33:08.416657+0000","last_peered":"2023-10-16T14:33:08.416657+0000","last_clean":"2023-10-16T14:33:08.416657+0000","last_became_active":"2023-10-16T14:31:41.088793+0000","last_became_peered":"2023-10-16T14:31:41.088793+0000","last_unstale":"2023-10-16T14:33:08.416657+0000","last_undegraded":"2023-10-16T14:33:08.416657+0000","last_fullsized":"2023-10-16T14:33:08.416657+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:05:59.531186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'15","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033313+0000","last_change":"2023-10-16T14:31:36.051051+0000","last_active":"2023-10-16T14:33:08.033313+0000","last_peered":"2023-10-16T14:33:08.033313+0000","last_clean":"2023-10-16T14:33:08.033313+0000","last_became_active":"2023-10-16T14:31:36.050746+0000","last_became_peered":"2023-10-16T14:31:36.050746+0000","last_unstale":"2023-10-16T14:33:08.033313+0000","last_undegraded":"2023-10-16T14:33:08.033313+0000","last_fullsized":"2023-10-16T14:33:08.033313+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:47.122405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640793+0000","last_change":"2023-10-16T14:31:37.038123+0000","last_active":"2023-10-16T14:33:36.640793+0000","last_peered":"2023-10-16T14:33:36.640793+0000","last_clean":"2023-10-16T14:33:36.640793+0000","last_became_active":"2023-10-16T14:31:37.037822+0000","last_became_peered":"2023-10-16T14:31:37.037822+0000","last_unstale":"2023-10-16T14:33:36.640793+0000","last_undegraded":"2023-10-16T14:33:36.640793+0000","last_fullsized":"2023-10-16T14:33:36.640793+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:35:44.028208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043832+0000","last_change":"2023-10-16T14:31:41.088944+0000","last_active":"2023-10-16T14:33:08.043832+0000","last_peered":"2023-10-16T14:33:08.043832+0000","last_clean":"2023-10-16T14:33:08.043832+0000","last_became_active":"2023-10-16T14:31:41.088566+0000","last_became_peered":"2023-10-16T14:31:41.088566+0000","last_unstale":"2023-10-16T14:33:08.043832+0000","last_undegraded":"2023-10-16T14:33:08.043832+0000","last_fullsized":"2023-10-16T14:33:08.043832+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:01:29.814710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033787+0000","last_change":"2023-10-16T14:31:36.045992+0000","last_active":"2023-10-16T14:33:08.033787+0000","last_peered":"2023-10-16T14:33:08.033787+0000","last_clean":"2023-10-16T14:33:08.033787+0000","last_became_active":"2023-10-16T14:31:36.045704+0000","last_became_peered":"2023-10-16T14:31:36.045704+0000","last_unstale":"2023-10-16T14:33:08.033787+0000","last_undegraded":"2023-10-16T14:33:08.033787+0000","last_fullsized":"2023-10-16T14:33:08.033787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:16:31.532630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640908+0000","last_change":"2023-10-16T14:31:37.034968+0000","last_active":"2023-10-16T14:33:36.640908+0000","last_peered":"2023-10-16T14:33:36.640908+0000","last_clean":"2023-10-16T14:33:36.640908+0000","last_became_active":"2023-10-16T14:31:37.034633+0000","last_became_peered":"2023-10-16T14:31:37.034633+0000","last_unstale":"2023-10-16T14:33:36.640908+0000","last_undegraded":"2023-10-16T14:33:36.640908+0000","last_fullsized":"2023-10-16T14:33:36.640908+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:31:36.101558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033930+0000","last_change":"2023-10-16T14:31:41.139541+0000","last_active":"2023-10-16T14:33:08.033930+0000","last_peered":"2023-10-16T14:33:08.033930+0000","last_clean":"2023-10-16T14:33:08.033930+0000","last_became_active":"2023-10-16T14:31:41.139075+0000","last_became_peered":"2023-10-16T14:31:41.139075+0000","last_unstale":"2023-10-16T14:33:08.033930+0000","last_undegraded":"2023-10-16T14:33:08.033930+0000","last_fullsized":"2023-10-16T14:33:08.033930+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:04:51.304740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043926+0000","last_change":"2023-10-16T14:31:36.056393+0000","last_active":"2023-10-16T14:33:08.043926+0000","last_peered":"2023-10-16T14:33:08.043926+0000","last_clean":"2023-10-16T14:33:08.043926+0000","last_became_active":"2023-10-16T14:31:36.056017+0000","last_became_peered":"2023-10-16T14:31:36.056017+0000","last_unstale":"2023-10-16T14:33:08.043926+0000","last_undegraded":"2023-10-16T14:33:08.043926+0000","last_fullsized":"2023-10-16T14:33:08.043926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:39:42.620327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638858+0000","last_change":"2023-10-16T14:31:37.035280+0000","last_active":"2023-10-16T14:33:36.638858+0000","last_peered":"2023-10-16T14:33:36.638858+0000","last_clean":"2023-10-16T14:33:36.638858+0000","last_became_active":"2023-10-16T14:31:37.034871+0000","last_became_peered":"2023-10-16T14:31:37.034871+0000","last_unstale":"2023-10-16T14:33:36.638858+0000","last_undegraded":"2023-10-16T14:33:36.638858+0000","last_fullsized":"2023-10-16T14:33:36.638858+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:07.815897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416348+0000","last_change":"2023-10-16T14:31:41.090742+0000","last_active":"2023-10-16T14:33:08.416348+0000","last_peered":"2023-10-16T14:33:08.416348+0000","last_clean":"2023-10-16T14:33:08.416348+0000","last_became_active":"2023-10-16T14:31:41.088033+0000","last_became_peered":"2023-10-16T14:31:41.088033+0000","last_unstale":"2023-10-16T14:33:08.416348+0000","last_undegraded":"2023-10-16T14:33:08.416348+0000","last_fullsized":"2023-10-16T14:33:08.416348+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:21:26.491006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041272+0000","last_change":"2023-10-16T14:31:36.049705+0000","last_active":"2023-10-16T14:33:08.041272+0000","last_peered":"2023-10-16T14:33:08.041272+0000","last_clean":"2023-10-16T14:33:08.041272+0000","last_became_active":"2023-10-16T14:31:36.049375+0000","last_became_peered":"2023-10-16T14:31:36.049375+0000","last_unstale":"2023-10-16T14:33:08.041272+0000","last_undegraded":"2023-10-16T14:33:08.041272+0000","last_fullsized":"2023-10-16T14:33:08.041272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:30:43.473151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637932+0000","last_change":"2023-10-16T14:32:07.681861+0000","last_active":"2023-10-16T14:33:36.637932+0000","last_peered":"2023-10-16T14:33:36.637932+0000","last_clean":"2023-10-16T14:33:36.637932+0000","last_became_active":"2023-10-16T14:32:07.681589+0000","last_became_peered":"2023-10-16T14:32:07.681589+0000","last_unstale":"2023-10-16T14:33:36.637932+0000","last_undegraded":"2023-10-16T14:33:36.637932+0000","last_fullsized":"2023-10-16T14:33:36.637932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:33:56.338853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416540+0000","last_change":"2023-10-16T14:31:41.139586+0000","last_active":"2023-10-16T14:33:08.416540+0000","last_peered":"2023-10-16T14:33:08.416540+0000","last_clean":"2023-10-16T14:33:08.416540+0000","last_became_active":"2023-10-16T14:31:41.139020+0000","last_became_peered":"2023-10-16T14:31:41.139020+0000","last_unstale":"2023-10-16T14:33:08.416540+0000","last_undegraded":"2023-10-16T14:33:08.416540+0000","last_fullsized":"2023-10-16T14:33:08.416540+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:37:17.523726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627499+0000","last_change":"2023-10-16T14:31:37.034448+0000","last_active":"2023-10-16T14:33:36.627499+0000","last_peered":"2023-10-16T14:33:36.627499+0000","last_clean":"2023-10-16T14:33:36.627499+0000","last_became_active":"2023-10-16T14:31:37.033869+0000","last_became_peered":"2023-10-16T14:31:37.033869+0000","last_unstale":"2023-10-16T14:33:36.627499+0000","last_undegraded":"2023-10-16T14:33:36.627499+0000","last_fullsized":"2023-10-16T14:33:36.627499+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:40:36.440578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'4","reported_seq":153,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:34:14.903045+0000","last_change":"2023-10-16T14:31:36.056035+0000","last_active":"2023-10-16T14:34:14.903045+0000","last_peered":"2023-10-16T14:34:14.903045+0000","last_clean":"2023-10-16T14:34:14.903045+0000","last_became_active":"2023-10-16T14:31:36.055696+0000","last_became_peered":"2023-10-16T14:31:36.055696+0000","last_unstale":"2023-10-16T14:34:14.903045+0000","last_undegraded":"2023-10-16T14:34:14.903045+0000","last_fullsized":"2023-10-16T14:34:14.903045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:07:32.543937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":127,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.050766+0000","last_change":"2023-10-16T14:33:08.050385+0000","last_active":"2023-10-16T14:33:08.050766+0000","last_peered":"2023-10-16T14:33:08.050766+0000","last_clean":"2023-10-16T14:33:08.050766+0000","last_became_active":"2023-10-16T14:33:08.045491+0000","last_became_peered":"2023-10-16T14:33:08.045491+0000","last_unstale":"2023-10-16T14:33:08.050766+0000","last_undegraded":"2023-10-16T14:33:08.050766+0000","last_fullsized":"2023-10-16T14:33:08.050766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:19:54.262369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641082+0000","last_change":"2023-10-16T14:31:37.038443+0000","last_active":"2023-10-16T14:33:36.641082+0000","last_peered":"2023-10-16T14:33:36.641082+0000","last_clean":"2023-10-16T14:33:36.641082+0000","last_became_active":"2023-10-16T14:31:37.037846+0000","last_became_peered":"2023-10-16T14:31:37.037846+0000","last_unstale":"2023-10-16T14:33:36.641082+0000","last_undegraded":"2023-10-16T14:33:36.641082+0000","last_fullsized":"2023-10-16T14:33:36.641082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:58:54.232337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'3","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040458+0000","last_change":"2023-10-16T14:31:36.044546+0000","last_active":"2023-10-16T14:33:08.040458+0000","last_peered":"2023-10-16T14:33:08.040458+0000","last_clean":"2023-10-16T14:33:08.040458+0000","last_became_active":"2023-10-16T14:31:36.044236+0000","last_became_peered":"2023-10-16T14:31:36.044236+0000","last_unstale":"2023-10-16T14:33:08.040458+0000","last_undegraded":"2023-10-16T14:33:08.040458+0000","last_fullsized":"2023-10-16T14:33:08.040458+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:06:54.353686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":317,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040909+0000","last_change":"2023-10-16T14:30:30.640766+0000","last_active":"2023-10-16T14:33:08.040909+0000","last_peered":"2023-10-16T14:33:08.040909+0000","last_clean":"2023-10-16T14:33:08.040909+0000","last_became_active":"2023-10-16T14:30:30.640250+0000","last_became_peered":"2023-10-16T14:30:30.640250+0000","last_unstale":"2023-10-16T14:33:08.040909+0000","last_undegraded":"2023-10-16T14:33:08.040909+0000","last_fullsized":"2023-10-16T14:33:08.040909+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:27:25.157800+0000","last_clean_scrub_stamp":"2023-10-16T14:27:25.157800+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:27:37.887563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044956+0000","last_change":"2023-10-16T14:32:07.688480+0000","last_active":"2023-10-16T14:33:08.044956+0000","last_peered":"2023-10-16T14:33:08.044956+0000","last_clean":"2023-10-16T14:33:08.044956+0000","last_became_active":"2023-10-16T14:32:07.687942+0000","last_became_peered":"2023-10-16T14:32:07.687942+0000","last_unstale":"2023-10-16T14:33:08.044956+0000","last_undegraded":"2023-10-16T14:33:08.044956+0000","last_fullsized":"2023-10-16T14:33:08.044956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:47:04.633747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.627304+0000","last_change":"2023-10-16T14:32:07.685575+0000","last_active":"2023-10-16T14:33:36.627304+0000","last_peered":"2023-10-16T14:33:36.627304+0000","last_clean":"2023-10-16T14:33:36.627304+0000","last_became_active":"2023-10-16T14:32:07.685112+0000","last_became_peered":"2023-10-16T14:32:07.685112+0000","last_unstale":"2023-10-16T14:33:36.627304+0000","last_undegraded":"2023-10-16T14:33:36.627304+0000","last_fullsized":"2023-10-16T14:33:36.627304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:30:56.355993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'3","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040976+0000","last_change":"2023-10-16T14:32:07.686280+0000","last_active":"2023-10-16T14:33:08.040976+0000","last_peered":"2023-10-16T14:33:08.040976+0000","last_clean":"2023-10-16T14:33:08.040976+0000","last_became_active":"2023-10-16T14:32:07.685264+0000","last_became_peered":"2023-10-16T14:32:07.685264+0000","last_unstale":"2023-10-16T14:33:08.040976+0000","last_undegraded":"2023-10-16T14:33:08.040976+0000","last_fullsized":"2023-10-16T14:33:08.040976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:20:50.232457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,8],"acting":[7,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.043860+0000","last_change":"2023-10-16T14:31:41.095792+0000","last_active":"2023-10-16T14:33:08.043860+0000","last_peered":"2023-10-16T14:33:08.043860+0000","last_clean":"2023-10-16T14:33:08.043860+0000","last_became_active":"2023-10-16T14:31:41.095385+0000","last_became_peered":"2023-10-16T14:31:41.095385+0000","last_unstale":"2023-10-16T14:33:08.043860+0000","last_undegraded":"2023-10-16T14:33:08.043860+0000","last_fullsized":"2023-10-16T14:33:08.043860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:40:07.635804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033618+0000","last_change":"2023-10-16T14:31:36.055534+0000","last_active":"2023-10-16T14:33:08.033618+0000","last_peered":"2023-10-16T14:33:08.033618+0000","last_clean":"2023-10-16T14:33:08.033618+0000","last_became_active":"2023-10-16T14:31:36.055204+0000","last_became_peered":"2023-10-16T14:31:36.055204+0000","last_unstale":"2023-10-16T14:33:08.033618+0000","last_undegraded":"2023-10-16T14:33:08.033618+0000","last_fullsized":"2023-10-16T14:33:08.033618+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:17:37.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626193+0000","last_change":"2023-10-16T14:31:37.046447+0000","last_active":"2023-10-16T14:33:36.626193+0000","last_peered":"2023-10-16T14:33:36.626193+0000","last_clean":"2023-10-16T14:33:36.626193+0000","last_became_active":"2023-10-16T14:31:37.045987+0000","last_became_peered":"2023-10-16T14:31:37.045987+0000","last_unstale":"2023-10-16T14:33:36.626193+0000","last_undegraded":"2023-10-16T14:33:36.626193+0000","last_fullsized":"2023-10-16T14:33:36.626193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:34:26.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.037942+0000","last_change":"2023-10-16T14:31:41.139678+0000","last_active":"2023-10-16T14:33:08.037942+0000","last_peered":"2023-10-16T14:33:08.037942+0000","last_clean":"2023-10-16T14:33:08.037942+0000","last_became_active":"2023-10-16T14:31:41.139222+0000","last_became_peered":"2023-10-16T14:31:41.139222+0000","last_unstale":"2023-10-16T14:33:08.037942+0000","last_undegraded":"2023-10-16T14:33:08.037942+0000","last_fullsized":"2023-10-16T14:33:08.037942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:35:14.448312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'7","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:40.913037+0000","last_change":"2023-10-16T14:31:36.054377+0000","last_active":"2023-10-16T14:33:40.913037+0000","last_peered":"2023-10-16T14:33:40.913037+0000","last_clean":"2023-10-16T14:33:40.913037+0000","last_became_active":"2023-10-16T14:31:36.053782+0000","last_became_peered":"2023-10-16T14:31:36.053782+0000","last_unstale":"2023-10-16T14:33:40.913037+0000","last_undegraded":"2023-10-16T14:33:40.913037+0000","last_fullsized":"2023-10-16T14:33:40.913037+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:29:13.590396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.638429+0000","last_change":"2023-10-16T14:31:37.036613+0000","last_active":"2023-10-16T14:33:36.638429+0000","last_peered":"2023-10-16T14:33:36.638429+0000","last_clean":"2023-10-16T14:33:36.638429+0000","last_became_active":"2023-10-16T14:31:37.036315+0000","last_became_peered":"2023-10-16T14:31:37.036315+0000","last_unstale":"2023-10-16T14:33:36.638429+0000","last_undegraded":"2023-10-16T14:33:36.638429+0000","last_fullsized":"2023-10-16T14:33:36.638429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:06:05.696238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.289756+0000","last_change":"2023-10-16T14:31:41.080180+0000","last_active":"2023-10-16T14:33:08.289756+0000","last_peered":"2023-10-16T14:33:08.289756+0000","last_clean":"2023-10-16T14:33:08.289756+0000","last_became_active":"2023-10-16T14:31:41.079732+0000","last_became_peered":"2023-10-16T14:31:41.079732+0000","last_unstale":"2023-10-16T14:33:08.289756+0000","last_undegraded":"2023-10-16T14:33:08.289756+0000","last_fullsized":"2023-10-16T14:33:08.289756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:39:52.099348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041225+0000","last_change":"2023-10-16T14:31:36.050154+0000","last_active":"2023-10-16T14:33:08.041225+0000","last_peered":"2023-10-16T14:33:08.041225+0000","last_clean":"2023-10-16T14:33:08.041225+0000","last_became_active":"2023-10-16T14:31:36.049808+0000","last_became_peered":"2023-10-16T14:31:36.049808+0000","last_unstale":"2023-10-16T14:33:08.041225+0000","last_undegraded":"2023-10-16T14:33:08.041225+0000","last_fullsized":"2023-10-16T14:33:08.041225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:46:33.058908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641382+0000","last_change":"2023-10-16T14:31:37.049708+0000","last_active":"2023-10-16T14:33:36.641382+0000","last_peered":"2023-10-16T14:33:36.641382+0000","last_clean":"2023-10-16T14:33:36.641382+0000","last_became_active":"2023-10-16T14:31:37.049388+0000","last_became_peered":"2023-10-16T14:31:37.049388+0000","last_unstale":"2023-10-16T14:33:36.641382+0000","last_undegraded":"2023-10-16T14:33:36.641382+0000","last_fullsized":"2023-10-16T14:33:36.641382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:27:53.264892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040831+0000","last_change":"2023-10-16T14:31:41.140379+0000","last_active":"2023-10-16T14:33:08.040831+0000","last_peered":"2023-10-16T14:33:08.040831+0000","last_clean":"2023-10-16T14:33:08.040831+0000","last_became_active":"2023-10-16T14:31:41.139893+0000","last_became_peered":"2023-10-16T14:31:41.139893+0000","last_unstale":"2023-10-16T14:33:08.040831+0000","last_undegraded":"2023-10-16T14:33:08.040831+0000","last_fullsized":"2023-10-16T14:33:08.040831+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:11:14.496106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454545+0000","last_change":"2023-10-16T14:31:36.054446+0000","last_active":"2023-10-16T14:33:08.454545+0000","last_peered":"2023-10-16T14:33:08.454545+0000","last_clean":"2023-10-16T14:33:08.454545+0000","last_became_active":"2023-10-16T14:31:36.054173+0000","last_became_peered":"2023-10-16T14:31:36.054173+0000","last_unstale":"2023-10-16T14:33:08.454545+0000","last_undegraded":"2023-10-16T14:33:08.454545+0000","last_fullsized":"2023-10-16T14:33:08.454545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:31:10.525458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.622286+0000","last_change":"2023-10-16T14:31:37.038990+0000","last_active":"2023-10-16T14:33:36.622286+0000","last_peered":"2023-10-16T14:33:36.622286+0000","last_clean":"2023-10-16T14:33:36.622286+0000","last_became_active":"2023-10-16T14:31:37.038703+0000","last_became_peered":"2023-10-16T14:31:37.038703+0000","last_unstale":"2023-10-16T14:33:36.622286+0000","last_undegraded":"2023-10-16T14:33:36.622286+0000","last_fullsized":"2023-10-16T14:33:36.622286+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:50:09.328138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044222+0000","last_change":"2023-10-16T14:31:41.092679+0000","last_active":"2023-10-16T14:33:08.044222+0000","last_peered":"2023-10-16T14:33:08.044222+0000","last_clean":"2023-10-16T14:33:08.044222+0000","last_became_active":"2023-10-16T14:31:41.090556+0000","last_became_peered":"2023-10-16T14:31:41.090556+0000","last_unstale":"2023-10-16T14:33:08.044222+0000","last_undegraded":"2023-10-16T14:33:08.044222+0000","last_fullsized":"2023-10-16T14:33:08.044222+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:46:43.059727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.038020+0000","last_change":"2023-10-16T14:31:36.053848+0000","last_active":"2023-10-16T14:33:08.038020+0000","last_peered":"2023-10-16T14:33:08.038020+0000","last_clean":"2023-10-16T14:33:08.038020+0000","last_became_active":"2023-10-16T14:31:36.049282+0000","last_became_peered":"2023-10-16T14:31:36.049282+0000","last_unstale":"2023-10-16T14:33:08.038020+0000","last_undegraded":"2023-10-16T14:33:08.038020+0000","last_fullsized":"2023-10-16T14:33:08.038020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:43:04.648689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637193+0000","last_change":"2023-10-16T14:31:37.046137+0000","last_active":"2023-10-16T14:33:36.637193+0000","last_peered":"2023-10-16T14:33:36.637193+0000","last_clean":"2023-10-16T14:33:36.637193+0000","last_became_active":"2023-10-16T14:31:37.045157+0000","last_became_peered":"2023-10-16T14:31:37.045157+0000","last_unstale":"2023-10-16T14:33:36.637193+0000","last_undegraded":"2023-10-16T14:33:36.637193+0000","last_fullsized":"2023-10-16T14:33:36.637193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:20:21.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.040273+0000","last_change":"2023-10-16T14:31:41.085056+0000","last_active":"2023-10-16T14:33:08.040273+0000","last_peered":"2023-10-16T14:33:08.040273+0000","last_clean":"2023-10-16T14:33:08.040273+0000","last_became_active":"2023-10-16T14:31:41.084548+0000","last_became_peered":"2023-10-16T14:31:41.084548+0000","last_unstale":"2023-10-16T14:33:08.040273+0000","last_undegraded":"2023-10-16T14:33:08.040273+0000","last_fullsized":"2023-10-16T14:33:08.040273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T22:09:51.747969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:48.668945+0000","last_change":"2023-10-16T14:31:36.048825+0000","last_active":"2023-10-16T14:33:48.668945+0000","last_peered":"2023-10-16T14:33:48.668945+0000","last_clean":"2023-10-16T14:33:48.668945+0000","last_became_active":"2023-10-16T14:31:36.048512+0000","last_became_peered":"2023-10-16T14:31:36.048512+0000","last_unstale":"2023-10-16T14:33:48.668945+0000","last_undegraded":"2023-10-16T14:33:48.668945+0000","last_fullsized":"2023-10-16T14:33:48.668945+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:45:04.696734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.640506+0000","last_change":"2023-10-16T14:31:37.042185+0000","last_active":"2023-10-16T14:33:36.640506+0000","last_peered":"2023-10-16T14:33:36.640506+0000","last_clean":"2023-10-16T14:33:36.640506+0000","last_became_active":"2023-10-16T14:31:37.041789+0000","last_became_peered":"2023-10-16T14:31:37.041789+0000","last_unstale":"2023-10-16T14:33:36.640506+0000","last_undegraded":"2023-10-16T14:33:36.640506+0000","last_fullsized":"2023-10-16T14:33:36.640506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:50:22.586473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.041196+0000","last_change":"2023-10-16T14:31:41.139556+0000","last_active":"2023-10-16T14:33:08.041196+0000","last_peered":"2023-10-16T14:33:08.041196+0000","last_clean":"2023-10-16T14:33:08.041196+0000","last_became_active":"2023-10-16T14:31:41.138990+0000","last_became_peered":"2023-10-16T14:31:41.138990+0000","last_unstale":"2023-10-16T14:33:08.041196+0000","last_undegraded":"2023-10-16T14:33:08.041196+0000","last_fullsized":"2023-10-16T14:33:08.041196+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:11:56.909384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.033443+0000","last_change":"2023-10-16T14:32:07.926237+0000","last_active":"2023-10-16T14:33:08.033443+0000","last_peered":"2023-10-16T14:33:08.033443+0000","last_clean":"2023-10-16T14:33:08.033443+0000","last_became_active":"2023-10-16T14:32:07.921517+0000","last_became_peered":"2023-10-16T14:32:07.921517+0000","last_unstale":"2023-10-16T14:33:08.033443+0000","last_undegraded":"2023-10-16T14:33:08.033443+0000","last_fullsized":"2023-10-16T14:33:08.033443+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T02:08:10.065187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635668+0000","last_change":"2023-10-16T14:31:37.040286+0000","last_active":"2023-10-16T14:33:36.635668+0000","last_peered":"2023-10-16T14:33:36.635668+0000","last_clean":"2023-10-16T14:33:36.635668+0000","last_became_active":"2023-10-16T14:31:37.039975+0000","last_became_peered":"2023-10-16T14:31:37.039975+0000","last_unstale":"2023-10-16T14:33:36.635668+0000","last_undegraded":"2023-10-16T14:33:36.635668+0000","last_fullsized":"2023-10-16T14:33:36.635668+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:11:49.555456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.454661+0000","last_change":"2023-10-16T14:31:41.097779+0000","last_active":"2023-10-16T14:33:08.454661+0000","last_peered":"2023-10-16T14:33:08.454661+0000","last_clean":"2023-10-16T14:33:08.454661+0000","last_became_active":"2023-10-16T14:31:41.097304+0000","last_became_peered":"2023-10-16T14:31:41.097304+0000","last_unstale":"2023-10-16T14:33:08.454661+0000","last_undegraded":"2023-10-16T14:33:08.454661+0000","last_fullsized":"2023-10-16T14:33:08.454661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.879512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"89'3","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.290521+0000","last_change":"2023-10-16T14:31:36.051369+0000","last_active":"2023-10-16T14:33:08.290521+0000","last_peered":"2023-10-16T14:33:08.290521+0000","last_clean":"2023-10-16T14:33:08.290521+0000","last_became_active":"2023-10-16T14:31:36.051064+0000","last_became_peered":"2023-10-16T14:31:36.051064+0000","last_unstale":"2023-10-16T14:33:08.290521+0000","last_undegraded":"2023-10-16T14:33:08.290521+0000","last_fullsized":"2023-10-16T14:33:08.290521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T01:03:59.741974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637090+0000","last_change":"2023-10-16T14:31:37.036344+0000","last_active":"2023-10-16T14:33:36.637090+0000","last_peered":"2023-10-16T14:33:36.637090+0000","last_clean":"2023-10-16T14:33:36.637090+0000","last_became_active":"2023-10-16T14:31:37.036081+0000","last_became_peered":"2023-10-16T14:31:37.036081+0000","last_unstale":"2023-10-16T14:33:36.637090+0000","last_undegraded":"2023-10-16T14:33:36.637090+0000","last_fullsized":"2023-10-16T14:33:36.637090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:53:03.150549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.052803+0000","last_change":"2023-10-16T14:31:41.086498+0000","last_active":"2023-10-16T14:33:08.052803+0000","last_peered":"2023-10-16T14:33:08.052803+0000","last_clean":"2023-10-16T14:33:08.052803+0000","last_became_active":"2023-10-16T14:31:41.086068+0000","last_became_peered":"2023-10-16T14:31:41.086068+0000","last_unstale":"2023-10-16T14:33:08.052803+0000","last_undegraded":"2023-10-16T14:33:08.052803+0000","last_fullsized":"2023-10-16T14:33:08.052803+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T23:00:57.697283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044786+0000","last_change":"2023-10-16T14:31:36.048901+0000","last_active":"2023-10-16T14:33:08.044786+0000","last_peered":"2023-10-16T14:33:08.044786+0000","last_clean":"2023-10-16T14:33:08.044786+0000","last_became_active":"2023-10-16T14:31:36.048622+0000","last_became_peered":"2023-10-16T14:31:36.048622+0000","last_unstale":"2023-10-16T14:33:08.044786+0000","last_undegraded":"2023-10-16T14:33:08.044786+0000","last_fullsized":"2023-10-16T14:33:08.044786+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:44:49.831890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.635462+0000","last_change":"2023-10-16T14:31:37.044724+0000","last_active":"2023-10-16T14:33:36.635462+0000","last_peered":"2023-10-16T14:33:36.635462+0000","last_clean":"2023-10-16T14:33:36.635462+0000","last_became_active":"2023-10-16T14:31:37.044409+0000","last_became_peered":"2023-10-16T14:31:37.044409+0000","last_unstale":"2023-10-16T14:33:36.635462+0000","last_undegraded":"2023-10-16T14:33:36.635462+0000","last_fullsized":"2023-10-16T14:33:36.635462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:08:04.570245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.044128+0000","last_change":"2023-10-16T14:31:41.084048+0000","last_active":"2023-10-16T14:33:08.044128+0000","last_peered":"2023-10-16T14:33:08.044128+0000","last_clean":"2023-10-16T14:33:08.044128+0000","last_became_active":"2023-10-16T14:31:41.083465+0000","last_became_peered":"2023-10-16T14:31:41.083465+0000","last_unstale":"2023-10-16T14:33:08.044128+0000","last_undegraded":"2023-10-16T14:33:08.044128+0000","last_fullsized":"2023-10-16T14:33:08.044128+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:40.061406+0000","last_clean_scrub_stamp":"2023-10-16T14:31:40.061406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:18:37.632144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:08.416659+0000","last_change":"2023-10-16T14:31:36.057437+0000","last_active":"2023-10-16T14:33:08.416659+0000","last_peered":"2023-10-16T14:33:08.416659+0000","last_clean":"2023-10-16T14:33:08.416659+0000","last_became_active":"2023-10-16T14:31:36.056390+0000","last_became_peered":"2023-10-16T14:31:36.056390+0000","last_unstale":"2023-10-16T14:33:08.416659+0000","last_undegraded":"2023-10-16T14:33:08.416659+0000","last_fullsized":"2023-10-16T14:33:08.416659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:35.022987+0000","last_clean_scrub_stamp":"2023-10-16T14:31:35.022987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:39.730673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631478+0000","last_change":"2023-10-16T14:31:37.036178+0000","last_active":"2023-10-16T14:33:36.631478+0000","last_peered":"2023-10-16T14:33:36.631478+0000","last_clean":"2023-10-16T14:33:36.631478+0000","last_became_active":"2023-10-16T14:31:37.035866+0000","last_became_peered":"2023-10-16T14:31:37.035866+0000","last_unstale":"2023-10-16T14:33:36.631478+0000","last_undegraded":"2023-10-16T14:33:36.631478+0000","last_fullsized":"2023-10-16T14:33:36.631478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:57:59.689810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.625624+0000","last_change":"2023-10-16T14:31:37.037391+0000","last_active":"2023-10-16T14:33:36.625624+0000","last_peered":"2023-10-16T14:33:36.625624+0000","last_clean":"2023-10-16T14:33:36.625624+0000","last_became_active":"2023-10-16T14:31:37.037050+0000","last_became_peered":"2023-10-16T14:31:37.037050+0000","last_unstale":"2023-10-16T14:33:36.625624+0000","last_undegraded":"2023-10-16T14:33:36.625624+0000","last_fullsized":"2023-10-16T14:33:36.625624+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T21:51:26.079853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":41,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641472+0000","last_change":"2023-10-16T14:32:07.920726+0000","last_active":"2023-10-16T14:33:36.641472+0000","last_peered":"2023-10-16T14:33:36.641472+0000","last_clean":"2023-10-16T14:33:36.641472+0000","last_became_active":"2023-10-16T14:32:07.920339+0000","last_became_peered":"2023-10-16T14:32:07.920339+0000","last_unstale":"2023-10-16T14:33:36.641472+0000","last_undegraded":"2023-10-16T14:33:36.641472+0000","last_fullsized":"2023-10-16T14:33:36.641472+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:47:27.237610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636199+0000","last_change":"2023-10-16T14:31:37.039491+0000","last_active":"2023-10-16T14:33:36.636199+0000","last_peered":"2023-10-16T14:33:36.636199+0000","last_clean":"2023-10-16T14:33:36.636199+0000","last_became_active":"2023-10-16T14:31:37.039177+0000","last_became_peered":"2023-10-16T14:31:37.039177+0000","last_unstale":"2023-10-16T14:33:36.636199+0000","last_undegraded":"2023-10-16T14:33:36.636199+0000","last_fullsized":"2023-10-16T14:33:36.636199+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:44:20.777901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.641467+0000","last_change":"2023-10-16T14:31:37.049881+0000","last_active":"2023-10-16T14:33:36.641467+0000","last_peered":"2023-10-16T14:33:36.641467+0000","last_clean":"2023-10-16T14:33:36.641467+0000","last_became_active":"2023-10-16T14:31:37.049559+0000","last_became_peered":"2023-10-16T14:31:37.049559+0000","last_unstale":"2023-10-16T14:33:36.641467+0000","last_undegraded":"2023-10-16T14:33:36.641467+0000","last_fullsized":"2023-10-16T14:33:36.641467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:49:54.053454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639422+0000","last_change":"2023-10-16T14:31:37.040328+0000","last_active":"2023-10-16T14:33:36.639422+0000","last_peered":"2023-10-16T14:33:36.639422+0000","last_clean":"2023-10-16T14:33:36.639422+0000","last_became_active":"2023-10-16T14:31:37.040018+0000","last_became_peered":"2023-10-16T14:31:37.040018+0000","last_unstale":"2023-10-16T14:33:36.639422+0000","last_undegraded":"2023-10-16T14:33:36.639422+0000","last_fullsized":"2023-10-16T14:33:36.639422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:16:50.629361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.621053+0000","last_change":"2023-10-16T14:31:37.036453+0000","last_active":"2023-10-16T14:33:36.621053+0000","last_peered":"2023-10-16T14:33:36.621053+0000","last_clean":"2023-10-16T14:33:36.621053+0000","last_became_active":"2023-10-16T14:31:37.036194+0000","last_became_peered":"2023-10-16T14:31:37.036194+0000","last_unstale":"2023-10-16T14:33:36.621053+0000","last_undegraded":"2023-10-16T14:33:36.621053+0000","last_fullsized":"2023-10-16T14:33:36.621053+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:18.925374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.626446+0000","last_change":"2023-10-16T14:31:37.035541+0000","last_active":"2023-10-16T14:33:36.626446+0000","last_peered":"2023-10-16T14:33:36.626446+0000","last_clean":"2023-10-16T14:33:36.626446+0000","last_became_active":"2023-10-16T14:31:37.035236+0000","last_became_peered":"2023-10-16T14:31:37.035236+0000","last_unstale":"2023-10-16T14:33:36.626446+0000","last_undegraded":"2023-10-16T14:33:36.626446+0000","last_fullsized":"2023-10-16T14:33:36.626446+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:37:45.215573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642770+0000","last_change":"2023-10-16T14:31:37.036676+0000","last_active":"2023-10-16T14:33:36.642770+0000","last_peered":"2023-10-16T14:33:36.642770+0000","last_clean":"2023-10-16T14:33:36.642770+0000","last_became_active":"2023-10-16T14:31:37.035848+0000","last_became_peered":"2023-10-16T14:31:37.035848+0000","last_unstale":"2023-10-16T14:33:36.642770+0000","last_undegraded":"2023-10-16T14:33:36.642770+0000","last_fullsized":"2023-10-16T14:33:36.642770+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:43.098735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.636872+0000","last_change":"2023-10-16T14:31:37.039087+0000","last_active":"2023-10-16T14:33:36.636872+0000","last_peered":"2023-10-16T14:33:36.636872+0000","last_clean":"2023-10-16T14:33:36.636872+0000","last_became_active":"2023-10-16T14:31:37.038529+0000","last_became_peered":"2023-10-16T14:31:37.038529+0000","last_unstale":"2023-10-16T14:33:36.636872+0000","last_undegraded":"2023-10-16T14:33:36.636872+0000","last_fullsized":"2023-10-16T14:33:36.636872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:21:17.112069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.637998+0000","last_change":"2023-10-16T14:31:37.034647+0000","last_active":"2023-10-16T14:33:36.637998+0000","last_peered":"2023-10-16T14:33:36.637998+0000","last_clean":"2023-10-16T14:33:36.637998+0000","last_became_active":"2023-10-16T14:31:37.034314+0000","last_became_peered":"2023-10-16T14:31:37.034314+0000","last_unstale":"2023-10-16T14:33:36.637998+0000","last_undegraded":"2023-10-16T14:33:36.637998+0000","last_fullsized":"2023-10-16T14:33:36.637998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:28:57.823333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.642106+0000","last_change":"2023-10-16T14:31:37.045824+0000","last_active":"2023-10-16T14:33:36.642106+0000","last_peered":"2023-10-16T14:33:36.642106+0000","last_clean":"2023-10-16T14:33:36.642106+0000","last_became_active":"2023-10-16T14:31:37.045535+0000","last_became_peered":"2023-10-16T14:31:37.045535+0000","last_unstale":"2023-10-16T14:33:36.642106+0000","last_undegraded":"2023-10-16T14:33:36.642106+0000","last_fullsized":"2023-10-16T14:33:36.642106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:55:47.535028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.639941+0000","last_change":"2023-10-16T14:31:37.038992+0000","last_active":"2023-10-16T14:33:36.639941+0000","last_peered":"2023-10-16T14:33:36.639941+0000","last_clean":"2023-10-16T14:33:36.639941+0000","last_became_active":"2023-10-16T14:31:37.038633+0000","last_became_peered":"2023-10-16T14:31:37.038633+0000","last_unstale":"2023-10-16T14:33:36.639941+0000","last_undegraded":"2023-10-16T14:33:36.639941+0000","last_fullsized":"2023-10-16T14:33:36.639941+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:53:08.742227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-10-16T14:33:36.631870+0000","last_change":"2023-10-16T14:31:37.035735+0000","last_active":"2023-10-16T14:33:36.631870+0000","last_peered":"2023-10-16T14:33:36.631870+0000","last_clean":"2023-10-16T14:33:36.631870+0000","last_became_active":"2023-10-16T14:31:37.035392+0000","last_became_peered":"2023-10-16T14:31:37.035392+0000","last_unstale":"2023-10-16T14:33:36.631870+0000","last_undegraded":"2023-10-16T14:33:36.631870+0000","last_fullsized":"2023-10-16T14:33:36.631870+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T14:31:36.027789+0000","last_clean_scrub_stamp":"2023-10-16T14:31:36.027789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-18T00:23:21.815285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2969600,"data_stored":2951840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82100,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":778,"num_read_kb":475,"num_write":178,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":246300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449133,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57499999999999996,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.97899999999999998,"5min":4.1749999999999998,"15min":4.1749999999999998},"last":0.374},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.436,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.64400000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.56999999999999995,"5min":0.433,"15min":0.433},"max":{"1min":0.77200000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.56399999999999995,"5min":0.377,"15min":0.377},"max":{"1min":0.84899999999999998,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.72099999999999997}]},{"osd":2,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.49399999999999999,"15min":0.48499999999999999},"min":{"1min":0.38200000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.753,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.436},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.81200000000000006,"5min":0.871,"15min":0.871},"last":0.38900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.57499999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.56799999999999995}]},{"osd":5,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57899999999999996,"15min":0.56499999999999995},"min":{"1min":0.34799999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79400000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.61099999999999999,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80900000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47899999999999998}]},{"osd":6,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69899999999999995},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.495},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.62,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.61099999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.71999999999999997,"15min":0.72299999999999998},"min":{"1min":0.59199999999999997,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.872,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.65100000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92200000000000004,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67300000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72599999999999998,"15min":0.72099999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.78900000000000003},"min":{"1min":0.70899999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0189999999999999,"5min":1.256,"15min":1.256},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54500000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.70799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79700000000000004,"15min":0.78400000000000003},"min":{"1min":0.73099999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.96099999999999997,"5min":1.046,"15min":1.046},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.75900000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.996,"5min":1.214,"15min":1.214},"last":0.78200000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678064,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28700,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716452,"statfs":{"total":95995035648,"available":95965646848,"internally_reserved":0,"allocated":1724416,"data_stored":1163249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.38800000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.997,"15min":0.997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.63900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.443},"min":{"1min":0.29999999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66400000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.45900000000000002},"min":{"1min":0.32100000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64500000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.498}]},{"osd":2,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60399999999999998,"15min":0.59799999999999998},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.82299999999999995,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52000000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.47299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.64000000000000001,"15min":0.65300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.90400000000000003,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.56599999999999995}]},{"osd":4,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.35399999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.83399999999999996,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77200000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67800000000000005,"15min":0.67400000000000004},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93200000000000005,"5min":1.01,"15min":1.01},"last":0.61599999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73799999999999999,"15min":0.75},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89300000000000002,"5min":1.016,"15min":1.016},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.52300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.92400000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60199999999999998}]},{"osd":7,"last update":"Mon Oct 16 14:31:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.56999999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.49099999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.83099999999999996}]},{"osd":9,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95699999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.752},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.55900000000000005,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1120000000000001,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85599999999999998}]},{"osd":11,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78700000000000003,"15min":0.78600000000000003},"min":{"1min":0.65600000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96499999999999997,"5min":1.038,"15min":1.038},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78100000000000003,"15min":0.77200000000000002},"min":{"1min":0.59399999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.99399999999999999,"5min":1.048,"15min":1.048},"last":0.78900000000000003}]}]},{"osd":9,"up_from":66,"seq":283467841590,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":506644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.51500000000000001,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.59399999999999997,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.53300000000000003,"15min":0.58899999999999997},"min":{"1min":0.34899999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.61099999999999999,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69699999999999995,"15min":0.76000000000000001},"min":{"1min":0.41999999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.91100000000000003,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.72699999999999998,"15min":0.79900000000000004},"min":{"1min":0.438,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81599999999999995,"5min":2.319,"15min":2.319},"last":0.80800000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.749,"15min":0.82899999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.78500000000000003,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72499999999999998,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.86099999999999999,"5min":1.849,"15min":1.849},"last":0.60999999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.60299999999999998,"15min":0.64000000000000001},"min":{"1min":0.34899999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.66600000000000004,"5min":2.7090000000000001,"15min":2.7090000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.68200000000000005,"15min":0.76200000000000001},"min":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.76000000000000001,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.73899999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.753,"15min":0.82099999999999995},"min":{"1min":0.46400000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.88900000000000001,"5min":1.879,"15min":1.879},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.753,"15min":0.81599999999999995},"min":{"1min":0.505,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76900000000000002,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.875,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.84299999999999997,"15min":0.93000000000000005},"min":{"1min":0.58899999999999997,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.83299999999999996,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Oct 16 14:31:11 2023","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.79500000000000004,"15min":0.88800000000000001},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89300000000000002,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.877,"15min":0.97199999999999998},"min":{"1min":0.52500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1060000000000001,"5min":3.2559999999999998,"15min":3.2559999999999998},"last":0.82599999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:34:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.84999999999999998,"15min":0.92700000000000005},"min":{"1min":0.57699999999999996,"5min":0.496,"15min":0.496},"max":{"1min":0.89200000000000002,"5min":1.944,"15min":1.944},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.83599999999999997,"15min":0.91100000000000003},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.86299999999999999,"5min":2.6379999999999999,"15min":2.6379999999999999},"last":0.63400000000000001}]},{"osd":10,"last update":"Mon Oct 16 14:33:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85899999999999999,"15min":0.875},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.095,"5min":1.986,"15min":1.986},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85699999999999998,"15min":0.86499999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.139,"5min":2.5840000000000001,"15min":2.5840000000000001},"last":0.79200000000000004}]},{"osd":11,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83099999999999996,"15min":0.82999999999999996},"min":{"1min":0.73499999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.016,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.93400000000000005,"15min":0.95699999999999996},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.0429999999999999,"5min":2.6779999999999999,"15min":2.6779999999999999},"last":0.67500000000000004}]}]},{"osd":8,"up_from":58,"seq":249108103227,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80900000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55700000000000005,"15min":0.56899999999999995},"min":{"1min":0.40000000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.71199999999999997,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.50800000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:32:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.495,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.51900000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.50700000000000001}]},{"osd":2,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48199999999999998,"15min":0.48999999999999999},"min":{"1min":0.317,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67300000000000004,"5min":0.87,"15min":0.87},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.497,"15min":0.505},"min":{"1min":0.34200000000000003,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59999999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47299999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60899999999999999,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.89300000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67100000000000004},"min":{"1min":0.46500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001}]},{"osd":4,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67400000000000004,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.312,"15min":0.312},"max":{"1min":0.875,"5min":1.165,"15min":1.165},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.42499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93999999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79300000000000004,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.35099999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81999999999999995,"5min":1.087,"15min":1.087},"last":0.39300000000000002}]},{"osd":6,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73899999999999999,"15min":0.76900000000000002},"min":{"1min":0.47899999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.89900000000000002,"5min":1.02,"15min":1.02},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70499999999999996,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.623}]},{"osd":7,"last update":"Mon Oct 16 14:33:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.50800000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.91400000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.751,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96399999999999997,"5min":1.044,"15min":1.044},"last":0.78000000000000003}]},{"osd":9,"last update":"Mon Oct 16 14:33:11 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.76600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63100000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.76700000000000002},"min":{"1min":0.55400000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.64200000000000002}]},{"osd":10,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.746,"15min":0.69899999999999995},"min":{"1min":0.66300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.006,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98699999999999999,"5min":1.075,"15min":1.075},"last":0.70299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.95499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.96999999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.81599999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1638400,"data_stored":1113909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57299999999999995,"15min":0.56599999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.44600000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.44500000000000001,"15min":0.438},"min":{"1min":0.33200000000000002,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.439,"15min":0.435},"min":{"1min":0.33100000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.625,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.65500000000000003}]},{"osd":2,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.55300000000000005},"min":{"1min":0.43099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.45500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80700000000000005,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72799999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.61599999999999999,"15min":0.58999999999999997},"min":{"1min":0.48499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.77200000000000002,"5min":2.589,"15min":2.589},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.748,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.432,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.77800000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.77200000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51800000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.48799999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.88700000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65600000000000003,"15min":0.63400000000000001},"min":{"1min":0.50700000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.878,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84499999999999997,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.70899999999999996,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.82199999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.73299999999999998,"15min":0.69299999999999995},"min":{"1min":0.46999999999999997,"5min":0.375,"15min":0.375},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":2.9319999999999999,"5min":2.9319999999999999,"15min":2.9319999999999999},"last":0.96199999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:09 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.83599999999999997},"min":{"1min":0.58399999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.879},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.59199999999999997,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":3.246,"5min":3.246,"15min":3.246},"last":0.86799999999999999}]},{"osd":10,"last update":"Mon Oct 16 14:33:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.90000000000000002,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.75600000000000001,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92600000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.83699999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.82499999999999996,"15min":0.82299999999999995},"min":{"1min":0.60399999999999998,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.60499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":1.03,"15min":1.03},"last":0.94899999999999995}]}]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":501489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.44800000000000001},"min":{"1min":0.32200000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51000000000000001,"15min":0.497},"min":{"1min":0.39800000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.85699999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.748}]},{"osd":1,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.81799999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76600000000000001,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68200000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.62},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.41899999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86199999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59599999999999997}]},{"osd":3,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98799999999999999,"5min":1.002,"15min":1.002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.875,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.83899999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.46000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Mon Oct 16 14:34:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.72399999999999998,"15min":0.72699999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.84899999999999998,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75,"15min":0.73199999999999998},"min":{"1min":0.59199999999999997,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.90800000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.48599999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77200000000000002,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.0269999999999999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.77900000000000003,"15min":0.76000000000000001},"min":{"1min":0.64200000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0880000000000001,"5min":1.232,"15min":1.232},"last":0.81000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.036,"5min":1.107,"15min":1.107},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.52300000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.93200000000000005,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.66800000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0700000000000001,"5min":1.133,"15min":1.133},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0029999999999999,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":1.004,"15min":0.92900000000000005},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.1220000000000001,"5min":11.375999999999999,"15min":11.375999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.90600000000000003},"min":{"1min":0.65900000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.105,"5min":1.157,"15min":1.157},"last":0.83999999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.85999999999999999},"min":{"1min":0.71099999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.1060000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.88900000000000001,"5min":1.4119999999999999,"15min":1.4990000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.0680000000000001,"5min":11.394,"15min":11.394},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":566934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.35199999999999998,"15min":0.34200000000000003},"max":{"1min":0.83299999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84499999999999997,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.625,"15min":0.55100000000000005},"min":{"1min":0.55400000000000005,"5min":0.376,"15min":0.376},"max":{"1min":0.96599999999999997,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.60899999999999999,"15min":0.55100000000000005},"min":{"1min":0.499,"5min":0.309,"15min":0.309},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53300000000000003}]},{"osd":3,"last update":"Mon Oct 16 14:34:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.56000000000000005},"min":{"1min":0.48499999999999999,"5min":0.38,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.502},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64100000000000001,"15min":0.57299999999999995},"min":{"1min":0.53700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.97599999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Mon Oct 16 14:33:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.42999999999999999},"min":{"1min":0.40500000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.435},"min":{"1min":0.30399999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.45700000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.66700000000000004,"15min":0.625},"min":{"1min":0.57899999999999996,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.032,"5min":1.181,"15min":1.181},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.67200000000000004,"15min":0.59899999999999998},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71399999999999997,"15min":0.67600000000000005},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88300000000000001,"5min":2.528,"15min":2.528},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68700000000000006,"15min":0.63900000000000001},"min":{"1min":0.49399999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94199999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76600000000000001}]},{"osd":7,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.51900000000000002},"min":{"1min":0.34300000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.58999999999999997,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.72699999999999998,"15min":0.68799999999999994},"min":{"1min":0.626,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.624},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.76600000000000001,"15min":0.71099999999999997},"min":{"1min":0.59699999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.85999999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:34:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78200000000000003,"15min":0.77000000000000002},"min":{"1min":0.56899999999999995,"5min":0.437,"15min":0.437},"max":{"1min":1.0089999999999999,"5min":1.012,"15min":1.012},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.72099999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0129999999999999,"5min":6.9989999999999997,"15min":6.9989999999999997},"last":0.72399999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.81999999999999995,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77500000000000002,"15min":0.77200000000000002},"min":{"1min":0.47199999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.97599999999999998,"5min":1.054,"15min":1.054},"last":0.84199999999999997}]},{"osd":11,"last update":"Mon Oct 16 14:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84099999999999997,"15min":0.83299999999999996},"min":{"1min":0.64000000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.89800000000000002,"15min":0.89200000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.89700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":502604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.66300000000000003,"15min":0.61499999999999999},"min":{"1min":0.44700000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.95499999999999996,"5min":1.0389999999999999,"15min":2.7349999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69799999999999995,"15min":0.59799999999999998},"min":{"1min":0.55000000000000004,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.069,"5min":1.069,"15min":1.141},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.56299999999999994},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68700000000000006,"15min":0.60299999999999998},"min":{"1min":0.44900000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.95999999999999996,"5min":1.202,"15min":1.202},"last":0.88100000000000001}]},{"osd":3,"last update":"Mon Oct 16 14:33:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80300000000000005,"5min":0.877,"15min":0.877},"last":0.435},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65500000000000003,"15min":0.64900000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Oct 16 14:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.622,"15min":0.60199999999999998},"min":{"1min":0.41599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.754},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.629,"15min":0.58999999999999997},"min":{"1min":0.46899999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.61099999999999999}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.73599999999999999,"15min":0.66500000000000004},"min":{"1min":0.59699999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0529999999999999,"5min":1.101,"15min":1.101},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.51900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51000000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.624,"15min":0.59299999999999997},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92000000000000004,"5min":1.298,"15min":1.298},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.46600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.03,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":0.95899999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79000000000000004}]},{"osd":8,"last update":"Mon Oct 16 14:33:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85599999999999998,"15min":0.84599999999999997},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.0700000000000001,"5min":1.274,"15min":1.274},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76400000000000001,"15min":0.755},"min":{"1min":0.59899999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0640000000000001,"5min":1.216,"15min":1.216},"last":0.63700000000000001}]},{"osd":9,"last update":"Mon Oct 16 14:34:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.79900000000000004,"15min":0.86499999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89600000000000002,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.0509999999999999,"5min":3.9860000000000002,"15min":3.9860000000000002},"last":0.95799999999999996}]},{"osd":10,"last update":"Mon Oct 16 14:33:31 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.70299999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.95799999999999996,"15min":0.96199999999999997},"min":{"1min":0.75900000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1379999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1}]},{"osd":11,"last update":"Mon Oct 16 14:33:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.95599999999999996,"15min":0.95999999999999996},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.115,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2050000000000001,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.70199999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":7.2480000000000002,"5min":7.2480000000000002,"15min":7.2480000000000002},"last":0.70699999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":581570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.504},"min":{"1min":0.372,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.73299999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":1.002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52600000000000002,"15min":0.498},"min":{"1min":0.377,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.78900000000000003}]},{"osd":1,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.63500000000000001,"15min":0.52700000000000002},"min":{"1min":0.44600000000000001,"5min":0.36199999999999999,"15min":0.32700000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.57899999999999996,"15min":0.53600000000000003},"min":{"1min":0.41699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.81499999999999995,"5min":1.022,"15min":1.022},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65300000000000002,"15min":0.58499999999999996},"min":{"1min":0.50900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.89300000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.874},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60899999999999999,"15min":0.56699999999999995},"min":{"1min":0.47099999999999997,"5min":0.376,"15min":0.36399999999999999},"max":{"1min":0.70399999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.85999999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.62,"15min":0.62},"min":{"1min":0.49199999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.45700000000000002},"min":{"1min":0.33600000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.501,"15min":0.45400000000000001},"min":{"1min":0.38,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57699999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.64600000000000002,"15min":0.622},"min":{"1min":0.53700000000000003,"5min":0.436,"15min":0.436},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1180000000000001,"5min":5.2359999999999998,"15min":5.2359999999999998},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.84699999999999998,"15min":0.83899999999999997},"min":{"1min":0.71499999999999997,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.93000000000000005}]},{"osd":8,"last update":"Mon Oct 16 14:33:46 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.59999999999999998},"min":{"1min":0.61599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.64000000000000001,"15min":0.57799999999999996},"min":{"1min":0.56899999999999995,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.68999999999999995}]},{"osd":9,"last update":"Mon Oct 16 14:33:14 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.70599999999999996,"15min":0.68300000000000005},"min":{"1min":0.63200000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.78700000000000003,"15min":0.747},"min":{"1min":0.74199999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.97899999999999998}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.625,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.83199999999999996,"15min":0.79400000000000004},"min":{"1min":0.79700000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.075,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.95299999999999996}]},{"osd":11,"last update":"Mon Oct 16 14:33:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.93000000000000005,"15min":0.92700000000000005},"min":{"1min":0.77000000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.75900000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":501741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48399999999999999,"15min":0.433},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82599999999999996,"5min":2.5249999999999999,"15min":2.5249999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48999999999999999,"15min":0.45800000000000002},"min":{"1min":0.34200000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70699999999999996,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":0.498}]},{"osd":1,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.66000000000000003,"15min":0.60799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80300000000000005,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.69099999999999995,"15min":0.60399999999999998},"min":{"1min":0.59199999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92500000000000004,"5min":3.0670000000000002,"15min":3.0670000000000002},"last":0.42399999999999999}]},{"osd":2,"last update":"Mon Oct 16 14:33:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66500000000000004,"15min":0.65200000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42099999999999999},"max":{"1min":0.83999999999999997,"5min":3.2519999999999998,"15min":3.2519999999999998},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.41899999999999998,"15min":0.38400000000000001},"max":{"1min":0.91900000000000004,"5min":2.8069999999999999,"15min":2.8069999999999999},"last":0.66700000000000004}]},{"osd":4,"last update":"Mon Oct 16 14:33:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":0.79700000000000004,"15min":0.68700000000000006},"min":{"1min":0.51400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":10.292999999999999,"5min":10.292999999999999,"15min":10.292999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.755,"15min":0.69799999999999995},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.443},"max":{"1min":3.1869999999999998,"5min":3.1869999999999998,"15min":3.1869999999999998},"last":0.52000000000000002}]},{"osd":5,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.72699999999999998,"15min":0.66300000000000003},"min":{"1min":0.52700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":2.7189999999999999,"5min":2.7189999999999999,"15min":2.7189999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.407,"5min":0.83499999999999996,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":10.314,"5min":10.314,"15min":10.314},"last":0.70499999999999996}]},{"osd":6,"last update":"Mon Oct 16 14:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.627},"min":{"1min":0.443,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.91700000000000004,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73299999999999998,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.88500000000000001,"5min":3.2200000000000002,"15min":3.2200000000000002},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.80400000000000005,"15min":0.76400000000000001},"min":{"1min":0.61199999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.0329999999999999,"5min":3.0329999999999999,"15min":3.0329999999999999},"last":0.624},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.56399999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":3.2080000000000002,"5min":3.2080000000000002,"15min":3.2080000000000002},"last":0.45200000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:30:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.746},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66500000000000004}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.82999999999999996,"15min":0.82399999999999995},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.497},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.751},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.88500000000000001,"15min":0.88300000000000001},"min":{"1min":0.63,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":3.2759999999999998,"5min":3.2759999999999998,"15min":3.2759999999999998},"last":0.68400000000000005}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.91900000000000004,"15min":0.90500000000000003},"min":{"1min":0.71899999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.0979999999999999,"5min":3.0979999999999999,"15min":3.0979999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.54800000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":3.1589999999999998,"5min":3.1589999999999998,"15min":3.1589999999999998},"last":0.53600000000000003}]},{"osd":11,"last update":"Mon Oct 16 14:33:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.871,"15min":0.83899999999999997},"min":{"1min":0.65700000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.5329999999999999,"5min":0.99299999999999999,"15min":0.90300000000000002},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":10.361000000000001,"5min":10.361000000000001,"15min":10.361000000000001},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":506197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:31:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.51500000000000001,"15min":0.49199999999999999},"min":{"1min":0.39000000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.44900000000000001}]},{"osd":1,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41599999999999998,"15min":0.38900000000000001},"min":{"1min":0.308,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.66200000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.433,"15min":0.40500000000000003},"min":{"1min":0.34399999999999997,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.71799999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.55600000000000005}]},{"osd":2,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59199999999999997,"15min":0.56999999999999995},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78900000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60199999999999998}]},{"osd":3,"last update":"Mon Oct 16 14:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.61399999999999999,"15min":0.56999999999999995},"min":{"1min":0.56000000000000005,"5min":0.309,"15min":0.309},"max":{"1min":0.752,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.45900000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81899999999999995,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.85299999999999998}]},{"osd":5,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.64500000000000002,"15min":0.59699999999999998},"min":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77700000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67700000000000005}]},{"osd":6,"last update":"Mon Oct 16 14:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.67300000000000004,"15min":0.64200000000000002},"min":{"1min":0.57099999999999995,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.98499999999999999,"5min":1.002,"15min":1.002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.42999999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90200000000000002,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Oct 16 14:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59399999999999997,"15min":0.56200000000000006},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83499999999999996,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.91400000000000003,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64100000000000001}]},{"osd":8,"last update":"Mon Oct 16 14:33:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.67800000000000005,"15min":0.65800000000000003},"min":{"1min":0.49099999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.89600000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73599999999999999}]},{"osd":9,"last update":"Mon Oct 16 14:33:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.96399999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.81999999999999995}]},{"osd":10,"last update":"Mon Oct 16 14:33:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.97799999999999998,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.77600000000000002},"min":{"1min":0.48799999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.623}]},{"osd":11,"last update":"Mon Oct 16 14:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82099999999999995,"15min":0.81599999999999995},"min":{"1min":0.69599999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.83799999999999997,"15min":0.83299999999999996},"min":{"1min":0.66200000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.68899999999999995}]}]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1658880,"data_stored":1110683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54100000000000004,"15min":0.54500000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.74199999999999999,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.372,"5min":0.314,"15min":0.314},"max":{"1min":0.79400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58699999999999997,"15min":0.54500000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80900000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63200000000000001,"15min":0.56499999999999995},"min":{"1min":0.374,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.88600000000000001,"5min":1.425,"15min":1.425},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.434,"15min":0.40000000000000002},"min":{"1min":0.38500000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.58099999999999996,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.436,"15min":0.40999999999999998},"min":{"1min":0.32200000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999}]},{"osd":3,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57699999999999996,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95899999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61199999999999999,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82899999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Mon Oct 16 14:33:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.64800000000000002,"15min":0.58399999999999996},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.93400000000000005,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.629,"15min":0.56599999999999995},"min":{"1min":0.41199999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90700000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.51600000000000001}]},{"osd":6,"last update":"Mon Oct 16 14:34:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.66200000000000003},"min":{"1min":0.49299999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.67200000000000004},"min":{"1min":0.52200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.92600000000000005}]},{"osd":7,"last update":"Mon Oct 16 14:33:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72599999999999998,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98899999999999999,"5min":1.196,"15min":1.196},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.67300000000000004,"15min":0.64700000000000002},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.69299999999999995}]},{"osd":8,"last update":"Mon Oct 16 14:33:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.67200000000000004,"15min":0.64100000000000001},"min":{"1min":0.56499999999999995,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.62,"15min":0.58099999999999996},"min":{"1min":0.54600000000000004,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.71599999999999997}]},{"osd":9,"last update":"Mon Oct 16 14:33:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.71599999999999997,"15min":0.68799999999999994},"min":{"1min":0.66500000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.872},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71499999999999997,"15min":0.69499999999999995},"min":{"1min":0.629,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.91100000000000003}]},{"osd":10,"last update":"Mon Oct 16 14:33:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73499999999999999,"15min":0.69899999999999995},"min":{"1min":0.59599999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.75,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81299999999999994}]},{"osd":11,"last update":"Mon Oct 16 14:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.82199999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.034,"5min":1.103,"15min":1.103},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81899999999999995,"15min":0.82099999999999995},"min":{"1min":0.66300000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.85399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":14489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":14223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T14:34:18.414 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:18.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 -- 172.21.15.178:0/4131569032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 msgr2=0x7f58bc1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 --2- 172.21.15.178:0/4131569032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc1012d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58b0009990 tx=0x7f58b0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 -- 172.21.15.178:0/4131569032 shutdown_connections 2023-10-16T14:34:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 --2- 172.21.15.178:0/4131569032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102a50 0x7f58bc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 --2- 172.21.15.178:0/4131569032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc1012d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 --2- 172.21.15.178:0/4131569032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102090 0x7f58bc102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 -- 172.21.15.178:0/4131569032 >> 172.21.15.178:0/4131569032 conn(0x7f58bc0fc430 msgr2=0x7f58bc0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 -- 172.21.15.178:0/4131569032 shutdown_connections 2023-10-16T14:34:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.641+0000 7f58c2dff700 1 -- 172.21.15.178:0/4131569032 wait complete. 2023-10-16T14:34:18.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 Processor -- start 2023-10-16T14:34:18.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 -- start start 2023-10-16T14:34:18.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102090 0x7f58bc0732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 0x7f58bc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc073f80 con 0x7f58bc100eb0 2023-10-16T14:34:18.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0740f0 con 0x7f58bc102a50 2023-10-16T14:34:18.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.642+0000 7f58c2dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc1a0bb0 con 0x7f58bc102090 2023-10-16T14:34:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41036/0 (socket says 172.21.15.178:41036) 2023-10-16T14:34:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 -- 172.21.15.178:0/609290516 learned_addr learned my addr 172.21.15.178:0/609290516 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58bbfff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102090 0x7f58bc0732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c139c700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 0x7f58bc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 -- 172.21.15.178:0/609290516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102090 msgr2=0x7f58bc0732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102090 0x7f58bc0732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 -- 172.21.15.178:0/609290516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 msgr2=0x7f58bc0737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 0x7f58bc0737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 -- 172.21.15.178:0/609290516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00095f0 con 0x7f58bc100eb0 2023-10-16T14:34:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c139c700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 0x7f58bc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c0b9b700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58b0004710 tx=0x7f58b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b001d070 con 0x7f58bc100eb0 2023-10-16T14:34:18.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58b0021410 con 0x7f58bc100eb0 2023-10-16T14:34:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc1a0e30 con 0x7f58bc100eb0 2023-10-16T14:34:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.643+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0017650 con 0x7f58bc100eb0 2023-10-16T14:34:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.644+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc1a11f0 con 0x7f58bc100eb0 2023-10-16T14:34:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.645+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc107db0 con 0x7f58bc100eb0 2023-10-16T14:34:18.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.646+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f58b0021580 con 0x7f58bc100eb0 2023-10-16T14:34:18.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.649+0000 7f58b9ffb700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 0x7f58ac0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:18.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.649+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f58b00ce410 con 0x7f58bc100eb0 2023-10-16T14:34:18.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.649+0000 7f58bbfff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 0x7f58ac0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:18.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.649+0000 7f58bbfff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 0x7f58ac0a3a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58bc100bd0 tx=0x7f58a8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:18.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.653+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f58b0097790 con 0x7f58bc100eb0 2023-10-16T14:34:18.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:18 smithi178 ceph-mon[98495]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:18.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:18 smithi178 ceph-mon[98495]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:18.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.858+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58bc10c770 con 0x7f58bc100eb0 2023-10-16T14:34:18.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.860+0000 7f58b9ffb700 1 -- 172.21.15.178:0/609290516 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7f58b009adb0 con 0x7f58bc100eb0 2023-10-16T14:34:18.865 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:18.865 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:18.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.862+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 msgr2=0x7f58ac0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.862+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 0x7f58ac0a3a60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f58bc100bd0 tx=0x7f58a8009260 comp rx=0 tx=0).stop 2023-10-16T14:34:18.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.862+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 msgr2=0x7f58bc074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:18.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.862+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58b0004710 tx=0x7f58b0009580 comp rx=0 tx=0).stop 2023-10-16T14:34:18.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 shutdown_connections 2023-10-16T14:34:18.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f58ac0a15a0 0x7f58ac0a3a60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f58bc102a50 0x7f58bc0737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58bc100eb0 0x7f58bc074c60 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 --2- 172.21.15.178:0/609290516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58bc102090 0x7f58bc0732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:18.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 >> 172.21.15.178:0/609290516 conn(0x7f58bc0fc430 msgr2=0x7f58bc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:18.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 shutdown_connections 2023-10-16T14:34:18.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:18.863+0000 7f58c2dff700 1 -- 172.21.15.178:0/609290516 wait complete. 2023-10-16T14:34:18.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:34:18.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:34:18.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:18 smithi187 ceph-mon[135297]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:18.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:18 smithi187 ceph-mon[135297]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:19.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:18 smithi186 ceph-mon[134892]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:19.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:18 smithi186 ceph-mon[134892]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:19.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 -- 172.21.15.178:0/1947339163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838104260 msgr2=0x7f0838106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 --2- 172.21.15.178:0/1947339163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838104260 0x7f0838106650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f082c0098f0 tx=0x7f082c009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:19.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 -- 172.21.15.178:0/1947339163 shutdown_connections 2023-10-16T14:34:19.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 --2- 172.21.15.178:0/1947339163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838106b90 0x7f0838109000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 --2- 172.21.15.178:0/1947339163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838101930 0x7f0838103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 --2- 172.21.15.178:0/1947339163 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838104260 0x7f0838106650 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 -- 172.21.15.178:0/1947339163 >> 172.21.15.178:0/1947339163 conn(0x7f08380fb320 msgr2=0x7f08380fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:19.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 -- 172.21.15.178:0/1947339163 shutdown_connections 2023-10-16T14:34:19.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.085+0000 7f083d1ff700 1 -- 172.21.15.178:0/1947339163 wait complete. 2023-10-16T14:34:19.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.086+0000 7f083d1ff700 1 Processor -- start 2023-10-16T14:34:19.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.086+0000 7f083d1ff700 1 -- start start 2023-10-16T14:34:19.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.086+0000 7f083d1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838101930 0x7f0838196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.086+0000 7f083d1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.086+0000 7f083d1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838106b90 0x7f083819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083d1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819e780 con 0x7f0838104260 2023-10-16T14:34:19.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083d1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819e8f0 con 0x7f0838106b90 2023-10-16T14:34:19.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083d1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819eab0 con 0x7f0838101930 2023-10-16T14:34:19.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083759e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838106b90 0x7f083819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57070/0 (socket says 172.21.15.178:57070) 2023-10-16T14:34:19.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f0836d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838101930 0x7f0838196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 -- 172.21.15.178:0/269655466 learned_addr learned my addr 172.21.15.178:0/269655466 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:19.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 -- 172.21.15.178:0/269655466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838101930 msgr2=0x7f0838196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838101930 0x7f0838196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 -- 172.21.15.178:0/269655466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838106b90 msgr2=0x7f083819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838106b90 0x7f083819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 -- 172.21.15.178:0/269655466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c009520 con 0x7f0838104260 2023-10-16T14:34:19.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.087+0000 7f083659c700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f082c009fd0 tx=0x7f082c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.088+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c01c070 con 0x7f0838104260 2023-10-16T14:34:19.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.088+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082c00f7d0 con 0x7f0838104260 2023-10-16T14:34:19.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.088+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083819ed30 con 0x7f0838104260 2023-10-16T14:34:19.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.088+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c004ab0 con 0x7f0838104260 2023-10-16T14:34:19.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.088+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083819f150 con 0x7f0838104260 2023-10-16T14:34:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.089+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08380fcea0 con 0x7f0838104260 2023-10-16T14:34:19.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.091+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f082c020070 con 0x7f0838104260 2023-10-16T14:34:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.092+0000 7f081ffff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 0x7f08200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.092+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f082c0cdc80 con 0x7f0838104260 2023-10-16T14:34:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.092+0000 7f0836d9d700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 0x7f08200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.093+0000 7f0836d9d700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 0x7f08200a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0838198110 tx=0x7f0828009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.096+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f082c05d100 con 0x7f0838104260 2023-10-16T14:34:19.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.296+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f083802d1b0 con 0x7f0838104260 2023-10-16T14:34:19.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.296+0000 7f081ffff700 1 -- 172.21.15.178:0/269655466 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f082c096fa0 con 0x7f0838104260 2023-10-16T14:34:19.300 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:34:19.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.298+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 msgr2=0x7f08200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 0x7f08200a3940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0838198110 tx=0x7f0828009310 comp rx=0 tx=0).stop 2023-10-16T14:34:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 msgr2=0x7f0838197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f082c009fd0 tx=0x7f082c00b790 comp rx=0 tx=0).stop 2023-10-16T14:34:19.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 shutdown_connections 2023-10-16T14:34:19.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f08200a1480 0x7f08200a3940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0838106b90 0x7f083819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0838101930 0x7f0838196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 --2- 172.21.15.178:0/269655466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0838104260 0x7f0838197180 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 >> 172.21.15.178:0/269655466 conn(0x7f08380fb320 msgr2=0x7f08381025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:19.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 shutdown_connections 2023-10-16T14:34:19.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.299+0000 7f083d1ff700 1 -- 172.21.15.178:0/269655466 wait complete. 2023-10-16T14:34:19.322 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:34:19.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/600832088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88100ea0 msgr2=0x7f6d881012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/600832088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88100ea0 0x7f6d881012c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6d780098f0 tx=0x7f6d78009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:19.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/600832088 shutdown_connections 2023-10-16T14:34:19.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/600832088 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 0x7f6d88109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/600832088 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88100ea0 0x7f6d881012c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/600832088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88102080 0x7f6d88102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/600832088 >> 172.21.15.178:0/600832088 conn(0x7f6d880fc420 msgr2=0x7f6d880fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:19.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/600832088 shutdown_connections 2023-10-16T14:34:19.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.520+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/600832088 wait complete. 2023-10-16T14:34:19.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 Processor -- start 2023-10-16T14:34:19.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 -- start start 2023-10-16T14:34:19.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 0x7f6d881971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 0x7f6d8819e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819e7e0 con 0x7f6d88100ea0 2023-10-16T14:34:19.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819e950 con 0x7f6d88102a40 2023-10-16T14:34:19.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.522+0000 7f6d8d5ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8819eb10 con 0x7f6d88102080 2023-10-16T14:34:19.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57084/0 (socket says 172.21.15.178:57084) 2023-10-16T14:34:19.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 -- 172.21.15.178:0/3994061993 learned_addr learned my addr 172.21.15.178:0/3994061993 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:19.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d877fe700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 0x7f6d8819e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d867fc700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 0x7f6d881971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 -- 172.21.15.178:0/3994061993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 msgr2=0x7f6d881971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 0x7f6d881971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 -- 172.21.15.178:0/3994061993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 msgr2=0x7f6d8819e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 0x7f6d8819e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 -- 172.21.15.178:0/3994061993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0095f0 con 0x7f6d88100ea0 2023-10-16T14:34:19.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d867fc700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 0x7f6d881971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:19.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.523+0000 7f6d86ffd700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d7800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.524+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7801c070 con 0x7f6d88100ea0 2023-10-16T14:34:19.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.524+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d78009520 con 0x7f6d88100ea0 2023-10-16T14:34:19.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.524+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7800f7d0 con 0x7f6d88100ea0 2023-10-16T14:34:19.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.524+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d78003ae0 con 0x7f6d88100ea0 2023-10-16T14:34:19.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.524+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8819f080 con 0x7f6d88100ea0 2023-10-16T14:34:19.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.525+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8810c710 con 0x7f6d88100ea0 2023-10-16T14:34:19.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.526+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f6d78020030 con 0x7f6d88100ea0 2023-10-16T14:34:19.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.528+0000 7f6d6ffff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 0x7f6d700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.528+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6d780ce150 con 0x7f6d88100ea0 2023-10-16T14:34:19.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.528+0000 7f6d867fc700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 0x7f6d700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.529+0000 7f6d867fc700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 0x7f6d700a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d88198170 tx=0x7f6d7c009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.532+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f6d78097470 con 0x7f6d88100ea0 2023-10-16T14:34:19.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.740+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6d8819f300 con 0x7f6d88100ea0 2023-10-16T14:34:19.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.740+0000 7f6d6ffff700 1 -- 172.21.15.178:0/3994061993 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6d7809aa90 con 0x7f6d88100ea0 2023-10-16T14:34:19.744 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:19.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 msgr2=0x7f6d700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 0x7f6d700a3940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6d88198170 tx=0x7f6d7c009310 comp rx=0 tx=0).stop 2023-10-16T14:34:19.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 msgr2=0x7f6d88196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6d78000c00 tx=0x7f6d7800b790 comp rx=0 tx=0).stop 2023-10-16T14:34:19.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 shutdown_connections 2023-10-16T14:34:19.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f6d700a1480 0x7f6d700a3940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f6d88102a40 0x7f6d8819e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d88100ea0 0x7f6d88196c70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 --2- 172.21.15.178:0/3994061993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d88102080 0x7f6d881971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 >> 172.21.15.178:0/3994061993 conn(0x7f6d880fc420 msgr2=0x7f6d880fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:19.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 shutdown_connections 2023-10-16T14:34:19.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.743+0000 7f6d8d5ff700 1 -- 172.21.15.178:0/3994061993 wait complete. 2023-10-16T14:34:19.765 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:34:19.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:19 smithi187 ceph-mon[135297]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:19.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:19 smithi187 ceph-mon[135297]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:19.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:19 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/609290516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:19.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:19 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/269655466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:19.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 -- 172.21.15.178:0/3606421406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102060 msgr2=0x7f88541024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 --2- 172.21.15.178:0/3606421406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102060 0x7f88541024e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8848009990 tx=0x7f8848009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 -- 172.21.15.178:0/3606421406 shutdown_connections 2023-10-16T14:34:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 --2- 172.21.15.178:0/3606421406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102a20 0x7f8854109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 --2- 172.21.15.178:0/3606421406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f88541012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 --2- 172.21.15.178:0/3606421406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102060 0x7f88541024e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 -- 172.21.15.178:0/3606421406 >> 172.21.15.178:0/3606421406 conn(0x7f88540fc400 msgr2=0x7f88540fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:19.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 -- 172.21.15.178:0/3606421406 shutdown_connections 2023-10-16T14:34:19.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.965+0000 7f88599ff700 1 -- 172.21.15.178:0/3606421406 wait complete. 2023-10-16T14:34:19.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 Processor -- start 2023-10-16T14:34:19.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 -- start start 2023-10-16T14:34:19.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f8854196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419e760 con 0x7f8854102a20 2023-10-16T14:34:19.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419e8d0 con 0x7f8854100e80 2023-10-16T14:34:19.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.966+0000 7f88599ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885419ea90 con 0x7f8854102060 2023-10-16T14:34:19.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:40556/0 (socket says 172.21.15.178:40556) 2023-10-16T14:34:19.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 -- 172.21.15.178:0/348874787 learned_addr learned my addr 172.21.15.178:0/348874787 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:19.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88537fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57104/0 (socket says 172.21.15.178:57104) 2023-10-16T14:34:19.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f8852ffd700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f8854196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 -- 172.21.15.178:0/348874787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 msgr2=0x7f8854196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f8854196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 -- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 msgr2=0x7f885419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:19.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:19.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 -- 172.21.15.178:0/348874787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480095f0 con 0x7f8854102060 2023-10-16T14:34:19.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f8852ffd700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f8854196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:19.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88537fe700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:19.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.967+0000 7f88527fc700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8848005720 tx=0x7f8848009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.968+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884801d070 con 0x7f8854102060 2023-10-16T14:34:19.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.968+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885419ed10 con 0x7f8854102060 2023-10-16T14:34:19.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.968+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8848021410 con 0x7f8854102060 2023-10-16T14:34:19.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.968+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848017650 con 0x7f8854102060 2023-10-16T14:34:19.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.968+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885419f190 con 0x7f8854102060 2023-10-16T14:34:19.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.969+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8854107e80 con 0x7f8854102060 2023-10-16T14:34:19.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.971+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8848021580 con 0x7f8854102060 2023-10-16T14:34:19.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.972+0000 7f88589fd700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 0x7f883c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:19.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.972+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f88480ce6a0 con 0x7f8854102060 2023-10-16T14:34:19.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.972+0000 7f8852ffd700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 0x7f883c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:19.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.973+0000 7f8852ffd700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 0x7f883c0a39c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88541980f0 tx=0x7f8844009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:19.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:19.974+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f88480979c0 con 0x7f8854102060 2023-10-16T14:34:20.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:19 smithi178 ceph-mon[98495]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:19 smithi178 ceph-mon[98495]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:19 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/609290516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:19 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/269655466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:19 smithi186 ceph-mon[134892]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-10-16T14:34:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:19 smithi186 ceph-mon[134892]: pgmap v357: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:19 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/609290516' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:19 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/269655466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:34:20.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.179+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f885410c8f0 con 0x7f8854102060 2023-10-16T14:34:20.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.180+0000 7f88589fd700 1 -- 172.21.15.178:0/348874787 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8848026020 con 0x7f8854102060 2023-10-16T14:34:20.184 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:34:20.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 msgr2=0x7f883c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 0x7f883c0a39c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f88541980f0 tx=0x7f8844009260 comp rx=0 tx=0).stop 2023-10-16T14:34:20.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 msgr2=0x7f8854197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8848005720 tx=0x7f8848009580 comp rx=0 tx=0).stop 2023-10-16T14:34:20.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 shutdown_connections 2023-10-16T14:34:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f883c0a1500 0x7f883c0a39c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8854102a20 0x7f885419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8854100e80 0x7f8854196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 --2- 172.21.15.178:0/348874787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8854102060 0x7f8854197160 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 >> 172.21.15.178:0/348874787 conn(0x7f88540fc400 msgr2=0x7f88540fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:20.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.182+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 shutdown_connections 2023-10-16T14:34:20.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.183+0000 7f88599ff700 1 -- 172.21.15.178:0/348874787 wait complete. 2023-10-16T14:34:20.206 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:34:20.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 -- 172.21.15.178:0/36105326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102060 msgr2=0x7f4a7c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 --2- 172.21.15.178:0/36105326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102060 0x7f4a7c1024e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009990 tx=0x7f4a6c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 -- 172.21.15.178:0/36105326 shutdown_connections 2023-10-16T14:34:20.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 --2- 172.21.15.178:0/36105326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c102a20 0x7f4a7c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 --2- 172.21.15.178:0/36105326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c100e80 0x7f4a7c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 --2- 172.21.15.178:0/36105326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102060 0x7f4a7c1024e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 -- 172.21.15.178:0/36105326 >> 172.21.15.178:0/36105326 conn(0x7f4a7c0fc400 msgr2=0x7f4a7c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:20.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 -- 172.21.15.178:0/36105326 shutdown_connections 2023-10-16T14:34:20.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.403+0000 7f4a83bff700 1 -- 172.21.15.178:0/36105326 wait complete. 2023-10-16T14:34:20.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 Processor -- start 2023-10-16T14:34:20.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 -- start start 2023-10-16T14:34:20.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c100e80 0x7f4a7c198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102a20 0x7f4a7c1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c1a09b0 con 0x7f4a7c102060 2023-10-16T14:34:20.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c1a0af0 con 0x7f4a7c100e80 2023-10-16T14:34:20.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a83bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7c1a0c80 con 0x7f4a7c102a20 2023-10-16T14:34:20.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a8119a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.404+0000 7f4a8119a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57132/0 (socket says 172.21.15.178:57132) 2023-10-16T14:34:20.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 -- 172.21.15.178:0/3714247759 learned_addr learned my addr 172.21.15.178:0/3714247759 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:20.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8199b700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c100e80 0x7f4a7c198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8219c700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102a20 0x7f4a7c1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 -- 172.21.15.178:0/3714247759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102a20 msgr2=0x7f4a7c1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102a20 0x7f4a7c1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 -- 172.21.15.178:0/3714247759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c100e80 msgr2=0x7f4a7c198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c100e80 0x7f4a7c198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 -- 172.21.15.178:0/3714247759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78009520 con 0x7f4a7c102060 2023-10-16T14:34:20.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a8119a700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0098e0 tx=0x7f4a6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:20.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c01d070 con 0x7f4a7c102060 2023-10-16T14:34:20.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6c0095f0 con 0x7f4a7c102060 2023-10-16T14:34:20.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a6c0054f0 con 0x7f4a7c102060 2023-10-16T14:34:20.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.405+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c004be0 con 0x7f4a7c102060 2023-10-16T14:34:20.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.406+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7c1a1130 con 0x7f4a7c102060 2023-10-16T14:34:20.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.406+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a7c193530 con 0x7f4a7c102060 2023-10-16T14:34:20.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.408+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4a6c00fbc0 con 0x7f4a7c102060 2023-10-16T14:34:20.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.410+0000 7f4a72ffd700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 0x7f4a680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.410+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4a6c0ce300 con 0x7f4a7c102060 2023-10-16T14:34:20.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.410+0000 7f4a8199b700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 0x7f4a680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.410+0000 7f4a8199b700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 0x7f4a680a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c101bd0 tx=0x7f4a78009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:20.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.414+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f4a6c05b100 con 0x7f4a7c102060 2023-10-16T14:34:20.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.614+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4a7c02d020 con 0x7f4a7c102060 2023-10-16T14:34:20.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.614+0000 7f4a72ffd700 1 -- 172.21.15.178:0/3714247759 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4a6c0a1070 con 0x7f4a7c102060 2023-10-16T14:34:20.618 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:34:20.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 msgr2=0x7f4a680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 0x7f4a680a3940 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c101bd0 tx=0x7f4a78009260 comp rx=0 tx=0).stop 2023-10-16T14:34:20.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 msgr2=0x7f4a7c199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0098e0 tx=0x7f4a6c004730 comp rx=0 tx=0).stop 2023-10-16T14:34:20.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 shutdown_connections 2023-10-16T14:34:20.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4a680a1480 0x7f4a680a3940 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a7c102a20 0x7f4a7c1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4a7c100e80 0x7f4a7c198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 --2- 172.21.15.178:0/3714247759 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a7c102060 0x7f4a7c199370 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 >> 172.21.15.178:0/3714247759 conn(0x7f4a7c0fc400 msgr2=0x7f4a7c107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:20.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 shutdown_connections 2023-10-16T14:34:20.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.617+0000 7f4a83bff700 1 -- 172.21.15.178:0/3714247759 wait complete. 2023-10-16T14:34:20.642 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:34:20.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 -- 172.21.15.178:0/2804939320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728100eb0 msgr2=0x7f97281012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 --2- 172.21.15.178:0/2804939320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728100eb0 0x7f97281012d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9718009990 tx=0x7f9718009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 -- 172.21.15.178:0/2804939320 shutdown_connections 2023-10-16T14:34:20.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 --2- 172.21.15.178:0/2804939320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102a50 0x7f9728109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 --2- 172.21.15.178:0/2804939320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728100eb0 0x7f97281012d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 --2- 172.21.15.178:0/2804939320 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728102090 0x7f9728102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 -- 172.21.15.178:0/2804939320 >> 172.21.15.178:0/2804939320 conn(0x7f97280fc430 msgr2=0x7f97280fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:20.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 -- 172.21.15.178:0/2804939320 shutdown_connections 2023-10-16T14:34:20.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.836+0000 7f972c9ff700 1 -- 172.21.15.178:0/2804939320 wait complete. 2023-10-16T14:34:20.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.837+0000 7f972c9ff700 1 Processor -- start 2023-10-16T14:34:20.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.837+0000 7f972c9ff700 1 -- start start 2023-10-16T14:34:20.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.837+0000 7f972c9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728100eb0 0x7f9728198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.837+0000 7f972c9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102090 0x7f9728199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f972c9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f972c9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281a0980 con 0x7f9728102a50 2023-10-16T14:34:20.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f972c9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281a0ac0 con 0x7f9728100eb0 2023-10-16T14:34:20.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f972c9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97281a0c50 con 0x7f9728102090 2023-10-16T14:34:20.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57146/0 (socket says 172.21.15.178:57146) 2023-10-16T14:34:20.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 -- 172.21.15.178:0/1041549714 learned_addr learned my addr 172.21.15.178:0/1041549714 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:20.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f971e59c700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728100eb0 0x7f9728198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f972659c700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102090 0x7f9728199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 -- 172.21.15.178:0/1041549714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102090 msgr2=0x7f9728199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102090 0x7f9728199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 -- 172.21.15.178:0/1041549714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728100eb0 msgr2=0x7f9728198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:20.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728100eb0 0x7f9728198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:20.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 -- 172.21.15.178:0/1041549714 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97180095f0 con 0x7f9728102a50 2023-10-16T14:34:20.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.838+0000 7f9726d9d700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f972000ef30 tx=0x7f972000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:20.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.839+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720018070 con 0x7f9728102a50 2023-10-16T14:34:20.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.839+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f972000f040 con 0x7f9728102a50 2023-10-16T14:34:20.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.839+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97281a0ed0 con 0x7f9728102a50 2023-10-16T14:34:20.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.839+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000cd00 con 0x7f9728102a50 2023-10-16T14:34:20.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.839+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97281a1300 con 0x7f9728102a50 2023-10-16T14:34:20.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.840+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972810c770 con 0x7f9728102a50 2023-10-16T14:34:20.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.842+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9720003a00 con 0x7f9728102a50 2023-10-16T14:34:20.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.843+0000 7f971ffff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 0x7f97140a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:20.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.843+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f97200ccaf0 con 0x7f9728102a50 2023-10-16T14:34:20.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.844+0000 7f971e59c700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 0x7f97140a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:20.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.844+0000 7f971e59c700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 0x7f97140a39e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f971800b2e0 tx=0x7f9718005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:20.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:20.847+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f9720095e70 con 0x7f9728102a50 2023-10-16T14:34:20.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3994061993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:20.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/348874787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:20.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:20 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3714247759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3994061993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/348874787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:20 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3714247759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3994061993' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/348874787' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:20 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3714247759' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:34:21.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.097+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97281a14f0 con 0x7f9728102a50 2023-10-16T14:34:21.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.098+0000 7f971ffff700 1 -- 172.21.15.178:0/1041549714 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11341 (secure 0 0 0) 0x7f9720099490 con 0x7f9728102a50 2023-10-16T14:34:21.102 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:21.102 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:32:15.910283+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:34:21.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.100+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 msgr2=0x7f97140a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.100+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 0x7f97140a39e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f971800b2e0 tx=0x7f9718005fb0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.100+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 msgr2=0x7f97281a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.100+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f972000ef30 tx=0x7f972000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 shutdown_connections 2023-10-16T14:34:21.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97140a1520 0x7f97140a39e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9728102a50 0x7f97281a03b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9728100eb0 0x7f9728198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 --2- 172.21.15.178:0/1041549714 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9728102090 0x7f9728199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 >> 172.21.15.178:0/1041549714 conn(0x7f97280fc430 msgr2=0x7f97281079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 shutdown_connections 2023-10-16T14:34:21.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.101+0000 7f972c9ff700 1 -- 172.21.15.178:0/1041549714 wait complete. 2023-10-16T14:34:21.107 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 26 2023-10-16T14:34:21.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:34:21.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.320+0000 7fa762bff700 1 -- 172.21.15.178:0/718543619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102090 msgr2=0x7fa75c102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.320+0000 7fa762bff700 1 --2- 172.21.15.178:0/718543619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102090 0x7fa75c102510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa748009990 tx=0x7fa748009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 -- 172.21.15.178:0/718543619 shutdown_connections 2023-10-16T14:34:21.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 --2- 172.21.15.178:0/718543619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c102a50 0x7fa75c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 --2- 172.21.15.178:0/718543619 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c100eb0 0x7fa75c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 --2- 172.21.15.178:0/718543619 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102090 0x7fa75c102510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 -- 172.21.15.178:0/718543619 >> 172.21.15.178:0/718543619 conn(0x7fa75c0fc430 msgr2=0x7fa75c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 -- 172.21.15.178:0/718543619 shutdown_connections 2023-10-16T14:34:21.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.321+0000 7fa762bff700 1 -- 172.21.15.178:0/718543619 wait complete. 2023-10-16T14:34:21.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 Processor -- start 2023-10-16T14:34:21.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 -- start start 2023-10-16T14:34:21.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c100eb0 0x7fa75c196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c102090 0x7fa75c197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c19e6d0 con 0x7fa75c102a50 2023-10-16T14:34:21.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c19e810 con 0x7fa75c102090 2023-10-16T14:34:21.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa762bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c19e9a0 con 0x7fa75c100eb0 2023-10-16T14:34:21.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa76119c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa76119c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57160/0 (socket says 172.21.15.178:57160) 2023-10-16T14:34:21.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa76119c700 1 -- 172.21.15.178:0/4273653262 learned_addr learned my addr 172.21.15.178:0/4273653262 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa75bfff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c102090 0x7fa75c197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.322+0000 7fa76099b700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c100eb0 0x7fa75c196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 -- 172.21.15.178:0/4273653262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c100eb0 msgr2=0x7fa75c196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c100eb0 0x7fa75c196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 -- 172.21.15.178:0/4273653262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c102090 msgr2=0x7fa75c197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c102090 0x7fa75c197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 -- 172.21.15.178:0/4273653262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480095f0 con 0x7fa75c102a50 2023-10-16T14:34:21.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa76119c700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa754007ee0 tx=0x7fa75400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754017070 con 0x7fa75c102a50 2023-10-16T14:34:21.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75400f040 con 0x7fa75c102a50 2023-10-16T14:34:21.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75c19ec80 con 0x7fa75c102a50 2023-10-16T14:34:21.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754005580 con 0x7fa75c102a50 2023-10-16T14:34:21.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.323+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c19f190 con 0x7fa75c102a50 2023-10-16T14:34:21.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.324+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c108420 con 0x7fa75c102a50 2023-10-16T14:34:21.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.326+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa75400db20 con 0x7fa75c102a50 2023-10-16T14:34:21.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.327+0000 7fa759ffb700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 0x7fa74c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.328+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa7540cc730 con 0x7fa75c102a50 2023-10-16T14:34:21.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.328+0000 7fa76099b700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 0x7fa74c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.328+0000 7fa76099b700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 0x7fa74c0a3a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa75c101c00 tx=0x7fa75000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.331+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa754095a20 con 0x7fa75c102a50 2023-10-16T14:34:21.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.533+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa75c10c770 con 0x7fa75c102a50 2023-10-16T14:34:21.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.535+0000 7fa759ffb700 1 -- 172.21.15.178:0/4273653262 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fa754099040 con 0x7fa75c102a50 2023-10-16T14:34:21.540 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:21.541 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:34:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 msgr2=0x7fa74c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 0x7fa74c0a3a60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa75c101c00 tx=0x7fa75000b410 comp rx=0 tx=0).stop 2023-10-16T14:34:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 msgr2=0x7fa75c19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa754007ee0 tx=0x7fa75400d3b0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 shutdown_connections 2023-10-16T14:34:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa74c0a15a0 0x7fa74c0a3a60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa75c102a50 0x7fa75c19e100 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa75c100eb0 0x7fa75c196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 --2- 172.21.15.178:0/4273653262 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa75c102090 0x7fa75c197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 >> 172.21.15.178:0/4273653262 conn(0x7fa75c0fc430 msgr2=0x7fa75c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 shutdown_connections 2023-10-16T14:34:21.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.537+0000 7fa762bff700 1 -- 172.21.15.178:0/4273653262 wait complete. 2023-10-16T14:34:21.562 DEBUG:tasks.fwd_scrub:Yielding 2023-10-16T14:34:21.562 INFO:teuthology.run_tasks:Running task workunit... 2023-10-16T14:34:21.628 INFO:tasks.workunit:Pulling workunits from ref 33be745ec4531df08cd1bb70b4c9c1f89be58078 2023-10-16T14:34:21.629 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-16T14:34:21.629 INFO:tasks.workunit:timeout=3h 2023-10-16T14:34:21.629 INFO:tasks.workunit:cleanup=True 2023-10-16T14:34:21.630 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:21.631 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-10-16T14:34:21.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:34:21.631 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:34:21.684 INFO:teuthology.orchestra.run.smithi178.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:21.685 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-10-16T14:34:21.685 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-10-16T14:34:21.685 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T14:34:21.685 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:cephfs_t:s0 2023-10-16T14:34:21.685 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-10-16 14:32:10.684734131 +0000 2023-10-16T14:34:21.686 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-10-16 14:32:10.684734131 +0000 2023-10-16T14:34:21.686 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-10-16 14:34:11.622023575 +0000 2023-10-16T14:34:21.686 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2023-10-16 14:32:10.684734131 +0000 2023-10-16T14:34:21.686 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-10-16T14:34:21.687 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-10-16T14:34:21.806 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/rishabh-d-dave/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 33be745ec4531df08cd1bb70b4c9c1f89be58078 2023-10-16T14:34:21.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 -- 172.21.15.178:0/2957880537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102070 msgr2=0x7f28e01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 --2- 172.21.15.178:0/2957880537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102070 0x7f28e01024f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0098f0 tx=0x7f28cc009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:21.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 -- 172.21.15.178:0/2957880537 shutdown_connections 2023-10-16T14:34:21.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 --2- 172.21.15.178:0/2957880537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0102a30 0x7f28e0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 --2- 172.21.15.178:0/2957880537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0100e90 0x7f28e01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 --2- 172.21.15.178:0/2957880537 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102070 0x7f28e01024f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 -- 172.21.15.178:0/2957880537 >> 172.21.15.178:0/2957880537 conn(0x7f28e00fc430 msgr2=0x7f28e00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 -- 172.21.15.178:0/2957880537 shutdown_connections 2023-10-16T14:34:21.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.846+0000 7f28e6fff700 1 -- 172.21.15.178:0/2957880537 wait complete. 2023-10-16T14:34:21.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 Processor -- start 2023-10-16T14:34:21.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 -- start start 2023-10-16T14:34:21.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 0x7f28e0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102a30 0x7f28e01a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a09f0 con 0x7f28e0102070 2023-10-16T14:34:21.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a0b30 con 0x7f28e0102a30 2023-10-16T14:34:21.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.848+0000 7f28e6fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e01a0cc0 con 0x7f28e0100e90 2023-10-16T14:34:21.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 0x7f28e0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57178/0 (socket says 172.21.15.178:57178) 2023-10-16T14:34:21.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28dffff700 1 -- 172.21.15.178:0/3938425450 learned_addr learned my addr 172.21.15.178:0/3938425450 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e559c700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102a30 0x7f28e01a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3938425450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102a30 msgr2=0x7f28e01a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102a30 0x7f28e01a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 msgr2=0x7f28e01993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3938425450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc009520 con 0x7f28e0100e90 2023-10-16T14:34:21.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28dffff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:21.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.849+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 0x7f28e0198e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005fd0 tx=0x7f28d400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.850+0000 7f28ddffb700 1 -- 172.21.15.178:0/3938425450 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4004220 con 0x7f28e0100e90 2023-10-16T14:34:21.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.850+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e01a0fa0 con 0x7f28e0100e90 2023-10-16T14:34:21.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.850+0000 7f28ddffb700 1 -- 172.21.15.178:0/3938425450 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d4004380 con 0x7f28e0100e90 2023-10-16T14:34:21.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.850+0000 7f28ddffb700 1 -- 172.21.15.178:0/3938425450 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d4003be0 con 0x7f28e0100e90 2023-10-16T14:34:21.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.850+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a13c0 con 0x7f28e0100e90 2023-10-16T14:34:21.860 INFO:tasks.workunit.client.0.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-16T14:34:21.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.853+0000 7f28ddffb700 1 -- 172.21.15.178:0/3938425450 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f28d4003d40 con 0x7f28e0100e90 2023-10-16T14:34:21.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.854+0000 7f28fa942b80 1 Processor -- start 2023-10-16T14:34:21.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.854+0000 7f28fa942b80 1 -- start start 2023-10-16T14:34:21.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.854+0000 7f28ddffb700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 0x7f28d00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a0219c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28ddffb700 1 -- 172.21.15.178:0/3938425450 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28d4014070 con 0x7f28e0100e90 2023-10-16T14:34:21.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219c6a0 0x564a0219cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0210fcf0 con 0x564a0219b800 2023-10-16T14:34:21.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a020f4ce0 con 0x564a0219aea0 2023-10-16T14:34:21.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28fa942b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a020f2600 con 0x564a0219c6a0 2023-10-16T14:34:21.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e559c700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 0x7f28d00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:36912/0 (socket says 172.21.15.178:36912) 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3881156870 learned_addr learned my addr 172.21.15.178:0/3881156870 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28dffff700 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a0219c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e559c700 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219c6a0 0x564a0219cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3881156870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219c6a0 msgr2=0x564a0219cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219c6a0 0x564a0219cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3881156870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 msgr2=0x564a0219c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a0219c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.855+0000 7f28e4d9b700 1 -- 172.21.15.178:0/3881156870 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a020ab470 con 0x564a0219aea0 2023-10-16T14:34:21.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.856+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28d4096b10 tx=0x7f28d4099a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74544b23c2d2e3fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.856+0000 7f28e559c700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 0x7f28d00a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x564a0219be80 tx=0x7f28d8009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.856+0000 7f28caffd700 1 -- 172.21.15.178:0/3881156870 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d409a0a0 con 0x564a0219aea0 2023-10-16T14:34:21.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.856+0000 7f28caffd700 1 -- 172.21.15.178:0/3881156870 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d409e1f0 con 0x564a0219aea0 2023-10-16T14:34:21.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.856+0000 7f28caffd700 1 -- 172.21.15.178:0/3881156870 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40997f0 con 0x564a0219aea0 2023-10-16T14:34:21.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 -- 172.21.15.178:0/3881156870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 msgr2=0x564a0219b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f28d4096b10 tx=0x7f28d4099a00 comp rx=0 tx=0).stop 2023-10-16T14:34:21.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 -- 172.21.15.178:0/3881156870 shutdown_connections 2023-10-16T14:34:21.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219c6a0 0x564a0219cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219aea0 0x564a0219b2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 --2- 172.21.15.178:0/3881156870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a0219c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 -- 172.21.15.178:0/3881156870 >> 172.21.15.178:0/3881156870 conn(0x564a020c8e50 msgr2=0x564a02196060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 -- 172.21.15.178:0/3881156870 shutdown_connections 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.857+0000 7f28fa942b80 1 -- 172.21.15.178:0/3881156870 wait complete. 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 -- start start 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 Processor -- start 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a021ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a020edef0 con 0x564a0219b800 2023-10-16T14:34:21.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0208a790 con 0x564a0219c6a0 2023-10-16T14:34:21.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0208a430 con 0x564a0219aea0 2023-10-16T14:34:21.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.858+0000 7f28fa942b80 10 client.0 _ll_register_callbacks cb 0x564a01ee7880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:21.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:40648/0 (socket says 172.21.15.178:40648) 2023-10-16T14:34:21.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 -- 172.21.15.178:0/1957940000 learned_addr learned my addr 172.21.15.178:0/1957940000 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e559c700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a021ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 -- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 msgr2=0x564a021ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a021ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 -- 172.21.15.178:0/1957940000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 msgr2=0x564a021a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 -- 172.21.15.178:0/1957940000 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e01a13c0 con 0x564a0219aea0 2023-10-16T14:34:21.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e559c700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a021ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:21.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009e50 tx=0x7f28d4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc392a45a92bdf17 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28bffff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:21.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d409a0a0 con 0x564a0219aea0 2023-10-16T14:34:21.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a020f2f70 con 0x564a0219aea0 2023-10-16T14:34:21.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.859+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28d40a1fa0 con 0x564a0219aea0 2023-10-16T14:34:21.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d40a2160 con 0x564a0219aea0 2023-10-16T14:34:21.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 10 client.25546 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:21.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a01f18e60 con 0x564a0219aea0 2023-10-16T14:34:21.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564a020f2f70 con 0x564a0219aea0 2023-10-16T14:34:21.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28d40991c0 con 0x564a0219aea0 2023-10-16T14:34:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 10 client.25546 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 10 client.25546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 20 client.25546 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a021b0570 0x564a021b2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 4 client.25546 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.860+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a0216ac90 con 0x564a021b0570 2023-10-16T14:34:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.861+0000 7f28dffff700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a021b0570 0x564a021b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.861+0000 7f28dffff700 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a021b0570 0x564a021b2990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f9d541c05acb55 server_cookie=166ffb8a4a16ee34 in_seq=0 out_seq=0 2023-10-16T14:34:21.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.861+0000 7f28bffff700 10 client.25546 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:21.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28bffff700 1 -- 172.21.15.178:0/1957940000 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a0216ac90 con 0x564a021b0570 2023-10-16T14:34:21.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28bffff700 10 client.25546 handle_command_reply: tid=0 2023-10-16T14:34:21.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 client.25546 shutdown 2023-10-16T14:34:21.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 msgr2=0x564a021a4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28d4009e50 tx=0x7f28d4004750 comp rx=0 tx=0).stop 2023-10-16T14:34:21.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 shutdown_connections 2023-10-16T14:34:21.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a021b0570 0x564a021b2990 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a021ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021a4d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 --2- 172.21.15.178:0/1957940000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28bffff700 0 client.25546 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:21.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 >> 172.21.15.178:0/1957940000 conn(0x564a020c8e50 msgr2=0x564a02195c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 shutdown_connections 2023-10-16T14:34:21.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.862+0000 7f28fa942b80 1 -- 172.21.15.178:0/1957940000 wait complete. 2023-10-16T14:34:21.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.863+0000 7f28fa942b80 20 client.25546 trim_cache size 0 max 16384 2023-10-16T14:34:21.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 Processor -- start 2023-10-16T14:34:21.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 -- start start 2023-10-16T14:34:21.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219b800 0x564a021ac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a0219fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a01f6e9a0 con 0x564a0219aea0 2023-10-16T14:34:21.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a02194ea0 con 0x564a0219c6a0 2023-10-16T14:34:21.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28fa942b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a02190c80 con 0x564a0219b800 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57208/0 (socket says 172.21.15.178:57208) 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2109881504 learned_addr learned my addr 172.21.15.178:0/2109881504 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28e559c700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a0219fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.880+0000 7f28dffff700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219b800 0x564a021ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2109881504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219b800 msgr2=0x564a021ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219b800 0x564a021ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2109881504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 msgr2=0x564a0219fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a0219fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2109881504 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a01f01ce0 con 0x564a0219aea0 2023-10-16T14:34:21.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e559c700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a0219fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28d4099fe0 tx=0x7f28d4098650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ac59010d6000ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28cb7fe700 1 -- 172.21.15.178:0/2109881504 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40834e0 con 0x564a0219aea0 2023-10-16T14:34:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28cb7fe700 1 -- 172.21.15.178:0/2109881504 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d40044f0 con 0x564a0219aea0 2023-10-16T14:34:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.881+0000 7f28cb7fe700 1 -- 172.21.15.178:0/2109881504 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40a1fa0 con 0x564a0219aea0 2023-10-16T14:34:21.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 -- 172.21.15.178:0/2109881504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 msgr2=0x564a0219f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28d4099fe0 tx=0x7f28d4098650 comp rx=0 tx=0).stop 2023-10-16T14:34:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 -- 172.21.15.178:0/2109881504 shutdown_connections 2023-10-16T14:34:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a0219fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219aea0 0x564a0219f610 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2109881504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219b800 0x564a021ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 -- 172.21.15.178:0/2109881504 >> 172.21.15.178:0/2109881504 conn(0x564a020c8e50 msgr2=0x564a021a2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 -- 172.21.15.178:0/2109881504 shutdown_connections 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.882+0000 7f28fa942b80 1 -- 172.21.15.178:0/2109881504 wait complete. 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 -- start start 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 Processor -- start 2023-10-16T14:34:21.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a02204390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a021abb80 con 0x564a0219b800 2023-10-16T14:34:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a021abcf0 con 0x564a0219c6a0 2023-10-16T14:34:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a02204950 con 0x564a0219aea0 2023-10-16T14:34:21.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.883+0000 7f28fa942b80 10 client.0 _ll_register_callbacks cb 0x564a01ee7880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:40660/0 (socket says 172.21.15.178:40660) 2023-10-16T14:34:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2339961225 learned_addr learned my addr 172.21.15.178:0/2339961225 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:21.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e559c700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a02204390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 msgr2=0x564a02204390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a02204390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2339961225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 msgr2=0x564a021ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 -- 172.21.15.178:0/2339961225 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d8008d50 con 0x564a0219aea0 2023-10-16T14:34:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28e4d9b700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28d40a1f00 tx=0x7f28d4009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db79051dde9d4649 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28bffff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.884+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d40834e0 con 0x564a0219aea0 2023-10-16T14:34:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28d409e1f0 con 0x564a0219aea0 2023-10-16T14:34:21.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28d409a0a0 con 0x564a0219aea0 2023-10-16T14:34:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28d40a2430 con 0x564a0219aea0 2023-10-16T14:34:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 10 client.25552 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f28e01a13c0 con 0x564a0219aea0 2023-10-16T14:34:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d409e1f0 con 0x564a0219aea0 2023-10-16T14:34:21.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28d409efc0 con 0x564a0219aea0 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 10 client.25552 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 10 client.25552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 20 client.25552 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a02205ef0 0x564a02208310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 4 client.25552 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:34:21.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.885+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564a020e6ed0 con 0x564a02205ef0 2023-10-16T14:34:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.886+0000 7f28dffff700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a02205ef0 0x564a02208310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.886+0000 7f28dffff700 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a02205ef0 0x564a02208310 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfe5ec9020b0f20 server_cookie=4f3d4e3e46c3f172 in_seq=0 out_seq=0 2023-10-16T14:34:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.886+0000 7f28bffff700 10 client.25552 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28bffff700 1 -- 172.21.15.178:0/2339961225 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564a020e6ed0 con 0x564a02205ef0 2023-10-16T14:34:21.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28bffff700 10 client.25552 handle_command_reply: tid=0 2023-10-16T14:34:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28fa942b80 1 client.25552 shutdown 2023-10-16T14:34:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 msgr2=0x564a021aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f28d40a1f00 tx=0x7f28d4009380 comp rx=0 tx=0).stop 2023-10-16T14:34:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.887+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 shutdown_connections 2023-10-16T14:34:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564a02205ef0 0x564a02208310 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564a0219c6a0 0x564a02204390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a0219aea0 0x564a021aae40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 --2- 172.21.15.178:0/2339961225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564a0219b800 0x564a021ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 >> 172.21.15.178:0/2339961225 conn(0x564a020c8e50 msgr2=0x564a021a4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28bffff700 0 client.25552 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 shutdown_connections 2023-10-16T14:34:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 1 -- 172.21.15.178:0/2339961225 wait complete. 2023-10-16T14:34:21.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28fa942b80 20 client.25552 trim_cache size 0 max 16384 2023-10-16T14:34:21.907 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:21.907 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:34:21.907 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "0c655c04-ecf5-493f-b202-aa41c4b7ddd7", 2023-10-16T14:34:21.907 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:34:21.908 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:21.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 msgr2=0x7f28d00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 0x7f28d00a3940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x564a0219be80 tx=0x7f28d8009920 comp rx=0 tx=0).stop 2023-10-16T14:34:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 msgr2=0x7f28e0198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.888+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 0x7f28e0198e70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005fd0 tx=0x7f28d400eca0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 shutdown_connections 2023-10-16T14:34:21.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28d00a1480 0x7f28d00a3940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f28e0102a30 0x7f28e01a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f28e0100e90 0x7f28e0198e70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 --2- 172.21.15.178:0/3938425450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28e0102070 0x7f28e01993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:21.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 >> 172.21.15.178:0/3938425450 conn(0x7f28e00fc430 msgr2=0x7f28e00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:21.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 shutdown_connections 2023-10-16T14:34:21.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:21.889+0000 7f28e6fff700 1 -- 172.21.15.178:0/3938425450 wait complete. 2023-10-16T14:34:21.911 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:21 smithi178 ceph-mon[98495]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:21.911 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:21 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1041549714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:21.911 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:21 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4273653262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:21.912 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:34:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:21 smithi187 ceph-mon[135297]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:21 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1041549714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:21 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4273653262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:21 smithi186 ceph-mon[134892]: pgmap v358: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:21 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1041549714' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:21 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4273653262' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:34:22.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 -- 172.21.15.178:0/3629751631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 msgr2=0x7f5468073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 --2- 172.21.15.178:0/3629751631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468073340 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5464009990 tx=0x7f5464009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 -- 172.21.15.178:0/3629751631 shutdown_connections 2023-10-16T14:34:22.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 --2- 172.21.15.178:0/3629751631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f546810b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 --2- 172.21.15.178:0/3629751631 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468073340 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 --2- 172.21.15.178:0/3629751631 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 -- 172.21.15.178:0/3629751631 >> 172.21.15.178:0/3629751631 conn(0x7f54680fc420 msgr2=0x7f54680fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 -- 172.21.15.178:0/3629751631 shutdown_connections 2023-10-16T14:34:22.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.158+0000 7f54705ff700 1 -- 172.21.15.178:0/3629751631 wait complete. 2023-10-16T14:34:22.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.159+0000 7f54705ff700 1 Processor -- start 2023-10-16T14:34:22.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.159+0000 7f54705ff700 1 -- start start 2023-10-16T14:34:22.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681a0980 con 0x7f5468108db0 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681a0ac0 con 0x7f5468073910 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f54705ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54681a0c50 con 0x7f5468074ef0 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57230/0 (socket says 172.21.15.178:57230) 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 -- 172.21.15.178:0/4184751753 learned_addr learned my addr 172.21.15.178:0/4184751753 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:22.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546db9a700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546e39b700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 -- 172.21.15.178:0/4184751753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 msgr2=0x7f5468199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 -- 172.21.15.178:0/4184751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 msgr2=0x7f5468198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546eb9c700 1 -- 172.21.15.178:0/4184751753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640095f0 con 0x7f5468108db0 2023-10-16T14:34:22.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.160+0000 7f546e39b700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:22.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f546eb9c700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f546000ef30 tx=0x7f546000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f545f7fe700 1 -- 172.21.15.178:0/4184751753 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460018070 con 0x7f5468108db0 2023-10-16T14:34:22.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f545f7fe700 1 -- 172.21.15.178:0/4184751753 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546000f040 con 0x7f5468108db0 2023-10-16T14:34:22.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54681a0ed0 con 0x7f5468108db0 2023-10-16T14:34:22.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f545f7fe700 1 -- 172.21.15.178:0/4184751753 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546000cdb0 con 0x7f5468108db0 2023-10-16T14:34:22.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.161+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54681a1300 con 0x7f5468108db0 2023-10-16T14:34:22.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.164+0000 7f545f7fe700 1 -- 172.21.15.178:0/4184751753 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5460003a00 con 0x7f5468108db0 2023-10-16T14:34:22.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.165+0000 7f548401fb80 1 Processor -- start 2023-10-16T14:34:22.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.165+0000 7f545f7fe700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 0x7f54540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.165+0000 7f548401fb80 1 -- start start 2023-10-16T14:34:22.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f545f7fe700 1 -- 172.21.15.178:0/4184751753 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f54600ccca0 con 0x7f5468108db0 2023-10-16T14:34:22.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c78040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c78aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582bebc70 con 0x55d582c77780 2023-10-16T14:34:22.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582bd0c60 con 0x55d582c78620 2023-10-16T14:34:22.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f548401fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582bce580 con 0x55d582c76e20 2023-10-16T14:34:22.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546db9a700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 0x7f54540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:40694/0 (socket says 172.21.15.178:40694) 2023-10-16T14:34:22.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 -- 172.21.15.178:0/2584788890 learned_addr learned my addr 172.21.15.178:0/2584788890 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:22.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546eb9c700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c78aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546db9a700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c78040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 -- 172.21.15.178:0/2584788890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 msgr2=0x55d582c78aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c78aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 -- 172.21.15.178:0/2584788890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 msgr2=0x55d582c78040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c78040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.166+0000 7f546e39b700 1 -- 172.21.15.178:0/2584788890 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d582b873f0 con 0x55d582c76e20 2023-10-16T14:34:22.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f546db9a700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c78040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:34:22.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f546db9a700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 0x7f54540a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5468074c10 tx=0x7f54580098c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f546e39b700 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f546400b2e0 tx=0x7f5464005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31a2a1b791c45bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f545cbfe700 1 -- 172.21.15.178:0/2584788890 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400bb50 con 0x55d582c76e20 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f545cbfe700 1 -- 172.21.15.178:0/2584788890 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546401e040 con 0x55d582c76e20 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.167+0000 7f545cbfe700 1 -- 172.21.15.178:0/2584788890 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464005490 con 0x55d582c76e20 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- 172.21.15.178:0/2584788890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 msgr2=0x55d582c77240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f546400b2e0 tx=0x7f5464005c00 comp rx=0 tx=0).stop 2023-10-16T14:34:22.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- 172.21.15.178:0/2584788890 shutdown_connections 2023-10-16T14:34:22.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c78aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c77240 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 --2- 172.21.15.178:0/2584788890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c78040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- 172.21.15.178:0/2584788890 >> 172.21.15.178:0/2584788890 conn(0x55d582ba4dd0 msgr2=0x55d582c71fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- 172.21.15.178:0/2584788890 shutdown_connections 2023-10-16T14:34:22.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- 172.21.15.178:0/2584788890 wait complete. 2023-10-16T14:34:22.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.168+0000 7f548401fb80 1 -- start start 2023-10-16T14:34:22.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 Processor -- start 2023-10-16T14:34:22.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 0x55d582c80bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c77780 0x55d582c81100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582bc9e70 con 0x55d582c78620 2023-10-16T14:34:22.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582b66710 con 0x55d582c76e20 2023-10-16T14:34:22.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582b663b0 con 0x55d582c77780 2023-10-16T14:34:22.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f548401fb80 10 client.0 _ll_register_callbacks cb 0x55d5829c3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:22.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f546eb9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f546e39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 0x55d582c80bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f546eb9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57242/0 (socket says 172.21.15.178:57242) 2023-10-16T14:34:22.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.169+0000 7f546eb9c700 1 -- 172.21.15.178:0/1552488427 learned_addr learned my addr 172.21.15.178:0/1552488427 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:22.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546db9a700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c77780 0x55d582c81100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 -- 172.21.15.178:0/1552488427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c77780 msgr2=0x55d582c81100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c77780 0x55d582c81100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 -- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 msgr2=0x55d582c80bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 0x55d582c80bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 -- 172.21.15.178:0/1552488427 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546810be60 con 0x55d582c78620 2023-10-16T14:34:22.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546e39b700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 0x55d582c80bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:22.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f546eb9c700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f546000ef00 tx=0x7f54600995e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa7cb31237d9088 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f5451ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:22.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54600b1070 con 0x55d582c78620 2023-10-16T14:34:22.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d582bceef0 con 0x55d582c78620 2023-10-16T14:34:22.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f546009a380 con 0x55d582c78620 2023-10-16T14:34:22.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5460099c80 con 0x55d582c78620 2023-10-16T14:34:22.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.170+0000 7f548401fb80 10 client.15534 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:22.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d5829f4de0 con 0x55d582c78620 2023-10-16T14:34:22.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d582bceef0 con 0x55d582c78620 2023-10-16T14:34:22.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5460099ee0 con 0x55d582c78620 2023-10-16T14:34:22.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 10 client.15534 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:22.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:22.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 20 client.15534 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:22.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582c8c900 0x55d582c8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 4 client.15534 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:22.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d582c46c10 con 0x55d582c8c900 2023-10-16T14:34:22.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.171+0000 7f546e39b700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582c8c900 0x55d582c8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.172+0000 7f546e39b700 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582c8c900 0x55d582c8ed20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6b0fa09df6016 server_cookie=6b1e9de6a1a84e40 in_seq=0 out_seq=0 2023-10-16T14:34:22.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.172+0000 7f5451ffb700 10 client.15534 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:22.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f5451ffb700 1 -- 172.21.15.178:0/1552488427 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d582c46c10 con 0x55d582c8c900 2023-10-16T14:34:22.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f5451ffb700 10 client.15534 handle_command_reply: tid=0 2023-10-16T14:34:22.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 client.15534 shutdown 2023-10-16T14:34:22.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 msgr2=0x55d582c88140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f546000ef00 tx=0x7f54600995e0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 shutdown_connections 2023-10-16T14:34:22.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582c8c900 0x55d582c8ed20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c78620 0x55d582c88140 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c76e20 0x55d582c80bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 --2- 172.21.15.178:0/1552488427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c77780 0x55d582c81100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 >> 172.21.15.178:0/1552488427 conn(0x55d582ba4dd0 msgr2=0x55d582c71bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f5451ffb700 0 client.15534 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:22.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 shutdown_connections 2023-10-16T14:34:22.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 1 -- 172.21.15.178:0/1552488427 wait complete. 2023-10-16T14:34:22.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.173+0000 7f548401fb80 20 client.15534 trim_cache size 0 max 16384 2023-10-16T14:34:22.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.190+0000 7f548401fb80 1 Processor -- start 2023-10-16T14:34:22.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 -- start start 2023-10-16T14:34:22.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582a4a920 con 0x55d582c77780 2023-10-16T14:34:22.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582c80070 con 0x55d582c78620 2023-10-16T14:34:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f548401fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582c80730 con 0x55d582c76e20 2023-10-16T14:34:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f546db9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f546db9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57256/0 (socket says 172.21.15.178:57256) 2023-10-16T14:34:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f546db9a700 1 -- 172.21.15.178:0/3813365703 learned_addr learned my addr 172.21.15.178:0/3813365703 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:22.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f546e39b700 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.191+0000 7f546eb9c700 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 -- 172.21.15.178:0/3813365703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 msgr2=0x55d582c707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 -- 172.21.15.178:0/3813365703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 msgr2=0x55d582c7f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c7f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 -- 172.21.15.178:0/3813365703 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5829ddc60 con 0x55d582c77780 2023-10-16T14:34:22.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f546db9a700 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f54580205e0 tx=0x7f5458015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f977981a50107f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f545d3ff700 1 -- 172.21.15.178:0/3813365703 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5458004220 con 0x55d582c77780 2023-10-16T14:34:22.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f545d3ff700 1 -- 172.21.15.178:0/3813365703 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5458004380 con 0x55d582c77780 2023-10-16T14:34:22.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.192+0000 7f545d3ff700 1 -- 172.21.15.178:0/3813365703 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54580108a0 con 0x55d582c77780 2023-10-16T14:34:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- 172.21.15.178:0/3813365703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 msgr2=0x55d582c88100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f54580205e0 tx=0x7f5458015040 comp rx=0 tx=0).stop 2023-10-16T14:34:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- 172.21.15.178:0/3813365703 shutdown_connections 2023-10-16T14:34:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c78620 0x55d582c7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c76e20 0x55d582c707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 --2- 172.21.15.178:0/3813365703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c77780 0x55d582c88100 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- 172.21.15.178:0/3813365703 >> 172.21.15.178:0/3813365703 conn(0x55d582ba4dd0 msgr2=0x55d582c936c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- 172.21.15.178:0/3813365703 shutdown_connections 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- 172.21.15.178:0/3813365703 wait complete. 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 -- start start 2023-10-16T14:34:22.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.193+0000 7f548401fb80 1 Processor -- start 2023-10-16T14:34:22.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c77780 0x55d582c841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 0x55d582c87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582c87750 con 0x55d582c76e20 2023-10-16T14:34:22.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582c878c0 con 0x55d582c77780 2023-10-16T14:34:22.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582c87a80 con 0x55d582c78620 2023-10-16T14:34:22.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f548401fb80 10 client.0 _ll_register_callbacks cb 0x7f5460099a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:22.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:57264/0 (socket says 172.21.15.178:57264) 2023-10-16T14:34:22.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 -- 172.21.15.178:0/754079957 learned_addr learned my addr 172.21.15.178:0/754079957 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:22.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546db9a700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c77780 0x55d582c841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546eb9c700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 0x55d582c87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 -- 172.21.15.178:0/754079957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 msgr2=0x55d582c87210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 0x55d582c87210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 -- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c77780 msgr2=0x55d582c841f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:22.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c77780 0x55d582c841f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.194+0000 7f546e39b700 1 -- 172.21.15.178:0/754079957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5458012280 con 0x55d582c76e20 2023-10-16T14:34:22.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f546eb9c700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 0x55d582c87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:22.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f546e39b700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5464009350 tx=0x7f54640051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea2a85d972e2cd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:22.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f5451ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:22.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5464031d40 con 0x55d582c76e20 2023-10-16T14:34:22.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f54640310c0 con 0x55d582c76e20 2023-10-16T14:34:22.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f546401e040 con 0x55d582c76e20 2023-10-16T14:34:22.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5464031ea0 con 0x55d582c76e20 2023-10-16T14:34:22.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f548401fb80 10 client.15540 fetch_fsmap learned FSMap version 26 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.195+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d582a1b250 con 0x55d582c76e20 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f546401e040 con 0x55d582c76e20 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 26) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f546400f460 con 0x55d582c76e20 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 10 client.15540 fetch_fsmap finished waiting for FSMap version 26 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:22.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 20 client.15540 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:22.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582ce1260 0x55d582ce3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:22.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 4 client.15540 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:34:22.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d582bc2e50 con 0x55d582ce1260 2023-10-16T14:34:22.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.196+0000 7f546db9a700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582ce1260 0x55d582ce3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:22.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f546db9a700 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582ce1260 0x55d582ce3680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f44f8cbb541653b server_cookie=f18c13558066211e in_seq=0 out_seq=0 2023-10-16T14:34:22.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f5451ffb700 10 client.15540 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:22.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f5451ffb700 1 -- 172.21.15.178:0/754079957 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d582bc2e50 con 0x55d582ce1260 2023-10-16T14:34:22.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f5451ffb700 10 client.15540 handle_command_reply: tid=0 2023-10-16T14:34:22.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 client.15540 shutdown 2023-10-16T14:34:22.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 msgr2=0x55d582c83cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5464009350 tx=0x7f54640051d0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 shutdown_connections 2023-10-16T14:34:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d582ce1260 0x55d582ce3680 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d582c78620 0x55d582c87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d582c76e20 0x55d582c83cb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f5451ffb700 0 client.15540 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 --2- 172.21.15.178:0/754079957 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d582c77780 0x55d582c841f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 >> 172.21.15.178:0/754079957 conn(0x55d582ba4dd0 msgr2=0x55d582c7f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 shutdown_connections 2023-10-16T14:34:22.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 1 -- 172.21.15.178:0/754079957 wait complete. 2023-10-16T14:34:22.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.197+0000 7f548401fb80 20 client.15540 trim_cache size 0 max 16384 2023-10-16T14:34:22.217 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:22.217 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-16T14:34:22.217 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: "0c655c04-ecf5-493f-b202-aa41c4b7ddd7": { 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "0c655c04-ecf5-493f-b202-aa41c4b7ddd7", 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:34:22.218 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:34:22.219 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 msgr2=0x7f54540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 0x7f54540a3940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5468074c10 tx=0x7f54580098c0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 msgr2=0x7f54681a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f546000ef30 tx=0x7f546000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 shutdown_connections 2023-10-16T14:34:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f54540a1480 0x7f54540a3940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5468108db0 0x7f54681a03b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5468073910 0x7f5468198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 --2- 172.21.15.178:0/4184751753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5468074ef0 0x7f5468199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 >> 172.21.15.178:0/4184751753 conn(0x7f54680fc420 msgr2=0x7f54680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 shutdown_connections 2023-10-16T14:34:22.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:22.198+0000 7f54705ff700 1 -- 172.21.15.178:0/4184751753 wait complete. 2023-10-16T14:34:22.223 INFO:tasks.cephfs.filesystem:scrub status for tag:0c655c04-ecf5-493f-b202-aa41c4b7ddd7 - {'path': '/', 'tag': '0c655c04-ecf5-493f-b202-aa41c4b7ddd7', 'options': 'recursive,force'} 2023-10-16T14:34:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:34:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:22.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:34:22.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:34:22.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:22.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:22 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:34:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:34:23.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:34:23.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:22 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:34:23.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:34:23.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:34:23.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:22 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:23.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:23 smithi187 ceph-mon[135297]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:24.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:23 smithi178 ceph-mon[98495]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:23 smithi186 ceph-mon[134892]: pgmap v359: 129 pgs: 129 active+clean; 660 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:34:25.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:25 smithi187 ceph-mon[135297]: pgmap v360: 129 pgs: 129 active+clean; 665 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-10-16T14:34:26.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:25 smithi178 ceph-mon[98495]: pgmap v360: 129 pgs: 129 active+clean; 665 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-10-16T14:34:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:25 smithi186 ceph-mon[134892]: pgmap v360: 129 pgs: 129 active+clean; 665 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-10-16T14:34:27.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:27 smithi187 ceph-mon[135297]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-16T14:34:28.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:27 smithi178 ceph-mon[98495]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-16T14:34:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:27 smithi186 ceph-mon[134892]: pgmap v361: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-10-16T14:34:29.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:29 smithi187 ceph-mon[135297]: pgmap v362: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-10-16T14:34:30.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:29 smithi178 ceph-mon[98495]: pgmap v362: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-10-16T14:34:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:29 smithi186 ceph-mon[134892]: pgmap v362: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-10-16T14:34:31.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:31 smithi178 ceph-mon[98495]: pgmap v363: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:31 smithi186 ceph-mon[134892]: pgmap v363: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:31.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:31 smithi187 ceph-mon[135297]: pgmap v363: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:32.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:32 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:34:32.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:32 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:34:32.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:32 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:34:33.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:33 smithi178 ceph-mon[98495]: pgmap v364: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:33 smithi186 ceph-mon[134892]: pgmap v364: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:33.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:33 smithi187 ceph-mon[135297]: pgmap v364: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-10-16T14:34:36.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:35 smithi178 ceph-mon[98495]: pgmap v365: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-10-16T14:34:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:35 smithi186 ceph-mon[134892]: pgmap v365: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-10-16T14:34:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:35 smithi187 ceph-mon[135297]: pgmap v365: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 7 op/s 2023-10-16T14:34:37.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:36 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:34:37.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:36 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:34:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:36 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:34:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:36 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:34:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:36 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:34:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:36 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:34:38.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:37 smithi178 ceph-mon[98495]: pgmap v366: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:34:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:37 smithi186 ceph-mon[134892]: pgmap v366: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:34:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:37 smithi187 ceph-mon[135297]: pgmap v366: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2023-10-16T14:34:40.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:39 smithi178 ceph-mon[98495]: pgmap v367: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:39 smithi186 ceph-mon[134892]: pgmap v367: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:39 smithi187 ceph-mon[135297]: pgmap v367: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:41.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:41 smithi178 ceph-mon[98495]: pgmap v368: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:41 smithi186 ceph-mon[134892]: pgmap v368: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:41.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:41 smithi187 ceph-mon[135297]: pgmap v368: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:44.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:43 smithi178 ceph-mon[98495]: pgmap v369: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:44.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:43 smithi186 ceph-mon[134892]: pgmap v369: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:43 smithi187 ceph-mon[135297]: pgmap v369: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:46.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:45 smithi178 ceph-mon[98495]: pgmap v370: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:45 smithi186 ceph-mon[134892]: pgmap v370: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:45 smithi187 ceph-mon[135297]: pgmap v370: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:48.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:47 smithi178 ceph-mon[98495]: pgmap v371: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:47 smithi186 ceph-mon[134892]: pgmap v371: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:48.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:47 smithi187 ceph-mon[135297]: pgmap v371: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:34:50.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:49 smithi178 ceph-mon[98495]: pgmap v372: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:50.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:49 smithi186 ceph-mon[134892]: pgmap v372: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:49 smithi187 ceph-mon[135297]: pgmap v372: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:51.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:51 smithi178 ceph-mon[98495]: pgmap v373: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:51.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:51 smithi186 ceph-mon[134892]: pgmap v373: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:51.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:51 smithi187 ceph-mon[135297]: pgmap v373: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:34:52.224 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:34:52.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 -- 172.21.15.178:0/2204006870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 msgr2=0x7f345c1003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 --2- 172.21.15.178:0/2204006870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c1003c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f344c009990 tx=0x7f344c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 -- 172.21.15.178:0/2204006870 shutdown_connections 2023-10-16T14:34:52.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 --2- 172.21.15.178:0/2204006870 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 --2- 172.21.15.178:0/2204006870 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 --2- 172.21.15.178:0/2204006870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c1003c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.474 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 -- 172.21.15.178:0/2204006870 >> 172.21.15.178:0/2204006870 conn(0x7f345c0fb160 msgr2=0x7f345c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 -- 172.21.15.178:0/2204006870 shutdown_connections 2023-10-16T14:34:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.464+0000 7f346299d700 1 -- 172.21.15.178:0/2204006870 wait complete. 2023-10-16T14:34:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 Processor -- start 2023-10-16T14:34:52.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 -- start start 2023-10-16T14:34:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1a0740 con 0x7f345c0ff5e0 2023-10-16T14:34:52.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1a0880 con 0x7f345c100900 2023-10-16T14:34:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.466+0000 7f346299d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1a0a10 con 0x7f345c0fff40 2023-10-16T14:34:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37650/0 (socket says 172.21.15.178:37650) 2023-10-16T14:34:52.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 -- 172.21.15.178:0/1321144986 learned_addr learned my addr 172.21.15.178:0/1321144986 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346119a700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346219c700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 -- 172.21.15.178:0/1321144986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 msgr2=0x7f345c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 -- 172.21.15.178:0/1321144986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 msgr2=0x7f345c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346199b700 1 -- 172.21.15.178:0/1321144986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c0095f0 con 0x7f345c0ff5e0 2023-10-16T14:34:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.467+0000 7f346219c700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f346199b700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f345800b530 tx=0x7f345800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f3452ffd700 1 -- 172.21.15.178:0/1321144986 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458011830 con 0x7f345c0ff5e0 2023-10-16T14:34:52.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c1a0cf0 con 0x7f345c0ff5e0 2023-10-16T14:34:52.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c1a1220 con 0x7f345c0ff5e0 2023-10-16T14:34:52.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f3452ffd700 1 -- 172.21.15.178:0/1321144986 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3458011e50 con 0x7f345c0ff5e0 2023-10-16T14:34:52.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.468+0000 7f3452ffd700 1 -- 172.21.15.178:0/1321144986 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800f2e0 con 0x7f345c0ff5e0 2023-10-16T14:34:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.473+0000 7f3452ffd700 1 -- 172.21.15.178:0/1321144986 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3458011990 con 0x7f345c0ff5e0 2023-10-16T14:34:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.474+0000 7f347744bb80 1 Processor -- start 2023-10-16T14:34:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 -- start start 2023-10-16T14:34:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cc5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cc5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15c38c70 con 0x55aa15cc5800 2023-10-16T14:34:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15c1dc60 con 0x55aa15cc3fb0 2023-10-16T14:34:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.475+0000 7f347744bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15c1b580 con 0x55aa15cc4910 2023-10-16T14:34:52.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:60972/0 (socket says 172.21.15.178:60972) 2023-10-16T14:34:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346219c700 1 -- 172.21.15.178:0/2131343880 learned_addr learned my addr 172.21.15.178:0/2131343880 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346199b700 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cc5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346219c700 1 -- 172.21.15.178:0/2131343880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 msgr2=0x55aa15cc5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.476+0000 7f346219c700 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cc5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f3452ffd700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 0x7f34480a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f346219c700 1 -- 172.21.15.178:0/2131343880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 msgr2=0x55aa15cc5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f346219c700 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cc5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f346219c700 1 -- 172.21.15.178:0/2131343880 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa15bd43f0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f346219c700 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3454009440 tx=0x7f3454009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc17871868f26b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f346219c700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 0x7f34480a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f3452ffd700 1 -- 172.21.15.178:0/1321144986 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f34580ccb60 con 0x7f345c0ff5e0 2023-10-16T14:34:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.477+0000 7f3447fff700 1 -- 172.21.15.178:0/2131343880 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345401a070 con 0x55aa15cc3fb0 2023-10-16T14:34:52.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.478+0000 7f3447fff700 1 -- 172.21.15.178:0/2131343880 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f345400f040 con 0x55aa15cc3fb0 2023-10-16T14:34:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.478+0000 7f3447fff700 1 -- 172.21.15.178:0/2131343880 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400e070 con 0x55aa15cc3fb0 2023-10-16T14:34:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.478+0000 7f347744bb80 1 -- 172.21.15.178:0/2131343880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 msgr2=0x55aa15cc43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.478+0000 7f347744bb80 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3454009440 tx=0x7f3454009920 comp rx=0 tx=0).stop 2023-10-16T14:34:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f346219c700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 0x7f34480a39e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3454012fd0 tx=0x7f3454015d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 -- 172.21.15.178:0/2131343880 shutdown_connections 2023-10-16T14:34:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cc5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc43d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 --2- 172.21.15.178:0/2131343880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cc5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 -- 172.21.15.178:0/2131343880 >> 172.21.15.178:0/2131343880 conn(0x55aa15bf1dd0 msgr2=0x55aa15cc0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 -- 172.21.15.178:0/2131343880 shutdown_connections 2023-10-16T14:34:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.479+0000 7f347744bb80 1 -- 172.21.15.178:0/2131343880 wait complete. 2023-10-16T14:34:52.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.480+0000 7f347744bb80 1 -- start start 2023-10-16T14:34:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.480+0000 7f347744bb80 1 Processor -- start 2023-10-16T14:34:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc4910 0x55aa15cce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc5800 0x55aa15cd5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15c16e70 con 0x55aa15cc4910 2023-10-16T14:34:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15bb3710 con 0x55aa15cc5800 2023-10-16T14:34:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15bb33b0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f347744bb80 10 client.0 _ll_register_callbacks cb 0x55aa15a10800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f346199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f346199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43604/0 (socket says 172.21.15.178:43604) 2023-10-16T14:34:52.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f346199b700 1 -- 172.21.15.178:0/3483605678 learned_addr learned my addr 172.21.15.178:0/3483605678 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.481+0000 7f346219c700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc5800 0x55aa15cd5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 -- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc5800 msgr2=0x55aa15cd5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc5800 0x55aa15cd5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 -- 172.21.15.178:0/3483605678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc4910 msgr2=0x55aa15cce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc4910 0x55aa15cce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 -- 172.21.15.178:0/3483605678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34540090f0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f346199b700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3458099a40 tx=0x7f3458099900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2baf13bd37b61bbd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f34457fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:52.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458099410 con 0x55aa15cc3fb0 2023-10-16T14:34:52.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.482+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f34580d5050 con 0x55aa15cc3fb0 2023-10-16T14:34:52.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa15c1bef0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f345809a1b0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 10 client.25579 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aa15a41de0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aa15c1bef0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3458098fa0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 10 client.25579 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:52.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:52.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 20 client.25579 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:52.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15cdc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 4 client.25579 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:52.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.483+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa15c93c10 con 0x55aa15cd9d60 2023-10-16T14:34:52.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.484+0000 7f346119a700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15cdc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.484+0000 7f34457fa700 10 client.25579 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:52.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.484+0000 7f346119a700 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15cdc180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc213ffca5824f4 server_cookie=894e3715b6ce4519 in_seq=0 out_seq=0 2023-10-16T14:34:52.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.485+0000 7f34457fa700 1 -- 172.21.15.178:0/3483605678 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55aa15c93c10 con 0x55aa15cd9d60 2023-10-16T14:34:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.485+0000 7f34457fa700 10 client.25579 handle_command_reply: tid=0 2023-10-16T14:34:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.485+0000 7f347744bb80 1 client.25579 shutdown 2023-10-16T14:34:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 msgr2=0x55aa15ccdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3458099a40 tx=0x7f3458099900 comp rx=0 tx=0).stop 2023-10-16T14:34:52.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 shutdown_connections 2023-10-16T14:34:52.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15cdc180 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f34457fa700 0 client.25579 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:52.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc5800 0x55aa15cd5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15ccdeb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 --2- 172.21.15.178:0/3483605678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc4910 0x55aa15cce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 >> 172.21.15.178:0/3483605678 conn(0x55aa15bf1dd0 msgr2=0x55aa15cbed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 shutdown_connections 2023-10-16T14:34:52.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 1 -- 172.21.15.178:0/3483605678 wait complete. 2023-10-16T14:34:52.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.486+0000 7f347744bb80 20 client.25579 trim_cache size 0 max 16384 2023-10-16T14:34:52.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.503+0000 7f347744bb80 1 Processor -- start 2023-10-16T14:34:52.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 -- start start 2023-10-16T14:34:52.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15ccdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15ccda80 con 0x55aa15cc5800 2023-10-16T14:34:52.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15a97920 con 0x55aa15cc4910 2023-10-16T14:34:52.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.504+0000 7f347744bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15c1bef0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:60994/0 (socket says 172.21.15.178:60994) 2023-10-16T14:34:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 -- 172.21.15.178:0/3084645337 learned_addr learned my addr 172.21.15.178:0/3084645337 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346219c700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15ccdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 -- 172.21.15.178:0/3084645337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 msgr2=0x55aa15cc8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 -- 172.21.15.178:0/3084645337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 msgr2=0x55aa15ccdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15ccdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346119a700 1 -- 172.21.15.178:0/3084645337 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa15a2ac60 con 0x55aa15cc4910 2023-10-16T14:34:52.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346199b700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:52.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.505+0000 7f346219c700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15ccdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:52.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.506+0000 7f346119a700 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55aa15cc4fe0 tx=0x7f344c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dc9a9800e088353 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.506+0000 7f3450bff700 1 -- 172.21.15.178:0/3084645337 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c01e040 con 0x55aa15cc4910 2023-10-16T14:34:52.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.506+0000 7f3450bff700 1 -- 172.21.15.178:0/3084645337 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344c023dc0 con 0x55aa15cc4910 2023-10-16T14:34:52.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.506+0000 7f3450bff700 1 -- 172.21.15.178:0/3084645337 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c005540 con 0x55aa15cc4910 2023-10-16T14:34:52.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 -- 172.21.15.178:0/3084645337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 msgr2=0x55aa15cd53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x55aa15cc4fe0 tx=0x7f344c005a90 comp rx=0 tx=0).stop 2023-10-16T14:34:52.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 -- 172.21.15.178:0/3084645337 shutdown_connections 2023-10-16T14:34:52.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15ccdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc3fb0 0x55aa15cc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 --2- 172.21.15.178:0/3084645337 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc4910 0x55aa15cd53c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 -- 172.21.15.178:0/3084645337 >> 172.21.15.178:0/3084645337 conn(0x55aa15bf1dd0 msgr2=0x55aa15cc8450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 -- 172.21.15.178:0/3084645337 shutdown_connections 2023-10-16T14:34:52.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.507+0000 7f347744bb80 1 -- 172.21.15.178:0/3084645337 wait complete. 2023-10-16T14:34:52.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.508+0000 7f347744bb80 1 -- start start 2023-10-16T14:34:52.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.508+0000 7f347744bb80 1 Processor -- start 2023-10-16T14:34:52.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.508+0000 7f347744bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.508+0000 7f347744bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cd37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.508+0000 7f347744bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cd3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f347744bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15cd42b0 con 0x55aa15cc5800 2023-10-16T14:34:52.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f347744bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15cd43f0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f347744bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa15cd45b0 con 0x55aa15cc4910 2023-10-16T14:34:52.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f347744bb80 10 client.0 _ll_register_callbacks cb 0x7f3458010ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f346119a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cd37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f346199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f346199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32772/0 (socket says 172.21.15.178:32772) 2023-10-16T14:34:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f346199b700 1 -- 172.21.15.178:0/451145961 learned_addr learned my addr 172.21.15.178:0/451145961 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.509+0000 7f346219c700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cd3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 -- 172.21.15.178:0/451145961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 msgr2=0x55aa15cd37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cd37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 -- 172.21.15.178:0/451145961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 msgr2=0x55aa15cd3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:52.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cd3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 -- 172.21.15.178:0/451145961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa15a41de0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346219c700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cd3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:52.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f346199b700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34580997b0 tx=0x7f34580a1b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8278ac79f7269aa4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f34457fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:34:52.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.510+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458010880 con 0x55aa15cc3fb0 2023-10-16T14:34:52.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f345400f040 con 0x55aa15cc3fb0 2023-10-16T14:34:52.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f34580d5070 con 0x55aa15cc3fb0 2023-10-16T14:34:52.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f345800d040 con 0x55aa15cc3fb0 2023-10-16T14:34:52.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f347744bb80 10 client.25601 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:52.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f345c10bc80 con 0x55aa15cc3fb0 2023-10-16T14:34:52.529 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.511+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f345400f040 con 0x55aa15cc3fb0 2023-10-16T14:34:52.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34580958a0 con 0x55aa15cc3fb0 2023-10-16T14:34:52.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 10 client.25601 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:52.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:52.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 20 client.25601 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:52.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15d39900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 4 client.25601 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:34:52.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aa15c0fe50 con 0x55aa15cd9d60 2023-10-16T14:34:52.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.512+0000 7f346119a700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15d39900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f346119a700 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15d39900 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3075204ef16478d7 server_cookie=daf030ba3260f67a in_seq=0 out_seq=0 2023-10-16T14:34:52.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f34457fa700 10 client.25601 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:52.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f34457fa700 1 -- 172.21.15.178:0/451145961 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aa15c0fe50 con 0x55aa15cd9d60 2023-10-16T14:34:52.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f34457fa700 10 client.25601 handle_command_reply: tid=0 2023-10-16T14:34:52.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 client.25601 shutdown 2023-10-16T14:34:52.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 msgr2=0x55aa15cd5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f34580997b0 tx=0x7f34580a1b50 comp rx=0 tx=0).stop 2023-10-16T14:34:52.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 shutdown_connections 2023-10-16T14:34:52.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55aa15cd9d60 0x55aa15d39900 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55aa15cc5800 0x55aa15cd3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55aa15cc3fb0 0x55aa15cd5190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f34457fa700 0 client.25601 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:52.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 --2- 172.21.15.178:0/451145961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa15cc4910 0x55aa15cd37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.513+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 >> 172.21.15.178:0/451145961 conn(0x55aa15bf1dd0 msgr2=0x55aa15ccdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.514+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 shutdown_connections 2023-10-16T14:34:52.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.514+0000 7f347744bb80 1 -- 172.21.15.178:0/451145961 wait complete. 2023-10-16T14:34:52.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.514+0000 7f347744bb80 20 client.25601 trim_cache size 0 max 16384 2023-10-16T14:34:52.537 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:52.538 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:34:52.538 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:34:52.538 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:52.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 msgr2=0x7f34480a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 0x7f34480a39e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3454012fd0 tx=0x7f3454015d80 comp rx=0 tx=0).stop 2023-10-16T14:34:52.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 msgr2=0x7f345c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f345800b530 tx=0x7f345800b8f0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 shutdown_connections 2023-10-16T14:34:52.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f34480a1520 0x7f34480a39e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f345c100900 0x7f345c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f345c0ff5e0 0x7f345c198bc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 --2- 172.21.15.178:0/1321144986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f345c0fff40 0x7f345c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 >> 172.21.15.178:0/1321144986 conn(0x7f345c0fb160 msgr2=0x7f345c0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 shutdown_connections 2023-10-16T14:34:52.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.515+0000 7f346299d700 1 -- 172.21.15.178:0/1321144986 wait complete. 2023-10-16T14:34:52.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:34:52.544 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:34:52.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/2608002291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a00ff5b0 msgr2=0x7fd4a00ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/2608002291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a00ff9d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009920 tx=0x7fd49c009c30 comp rx=0 tx=0).stop 2023-10-16T14:34:52.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/2608002291 shutdown_connections 2023-10-16T14:34:52.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/2608002291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a01008d0 0x7fd4a0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/2608002291 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a00ff9d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/2608002291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00fff10 0x7fd4a0100390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/2608002291 >> 172.21.15.178:0/2608002291 conn(0x7fd4a00fb150 msgr2=0x7fd4a00fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:52.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/2608002291 shutdown_connections 2023-10-16T14:34:52.761 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.755+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/2608002291 wait complete. 2023-10-16T14:34:52.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 Processor -- start 2023-10-16T14:34:52.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 -- start start 2023-10-16T14:34:52.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a00fff10 0x7fd4a01991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a07e0 con 0x7fd4a00ff5b0 2023-10-16T14:34:52.766 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a0950 con 0x7fd4a01008d0 2023-10-16T14:34:52.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.757+0000 7fd4a6f9d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01a0b10 con 0x7fd4a00fff10 2023-10-16T14:34:52.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32796/0 (socket says 172.21.15.178:32796) 2023-10-16T14:34:52.767 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 -- 172.21.15.178:0/90967353 learned_addr learned my addr 172.21.15.178:0/90967353 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:52.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a5f9b700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a579a700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a00fff10 0x7fd4a01991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 -- 172.21.15.178:0/90967353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a00fff10 msgr2=0x7fd4a01991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a00fff10 0x7fd4a01991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 -- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 msgr2=0x7fd4a0198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:52.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a0198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:52.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a679c700 1 -- 172.21.15.178:0/90967353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498005e70 con 0x7fd4a01008d0 2023-10-16T14:34:52.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.758+0000 7fd4a5f9b700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:52.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd4a679c700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd49800ef30 tx=0x7fd49800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498018070 con 0x7fd4a01008d0 2023-10-16T14:34:52.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c009520 con 0x7fd4a01008d0 2023-10-16T14:34:52.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a01a1110 con 0x7fd4a01008d0 2023-10-16T14:34:52.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd49800f040 con 0x7fd4a01008d0 2023-10-16T14:34:52.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.759+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498007dd0 con 0x7fd4a01008d0 2023-10-16T14:34:52.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.761+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484005310 con 0x7fd4a01008d0 2023-10-16T14:34:52.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.764+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd49800caa0 con 0x7fd4a01008d0 2023-10-16T14:34:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.766+0000 7fd496ffd700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 0x7fd48c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.766+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd49809a7e0 con 0x7fd4a01008d0 2023-10-16T14:34:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.766+0000 7fd4a5f9b700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 0x7fd48c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:52.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.767+0000 7fd4a5f9b700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 0x7fd48c0a39c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00faa0 tx=0x7fd49c019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:52.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:52.772+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd498096bb0 con 0x7fd4a01008d0 2023-10-16T14:34:53.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.027+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4840055c0 con 0x7fd4a01008d0 2023-10-16T14:34:53.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.029+0000 7fd496ffd700 1 -- 172.21.15.178:0/90967353 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11340 (secure 0 0 0) 0x7fd498096d50 con 0x7fd4a01008d0 2023-10-16T14:34:53.033 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:34:53.033 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:34:30.913267+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:34:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 msgr2=0x7fd48c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 0x7fd48c0a39c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00faa0 tx=0x7fd49c019040 comp rx=0 tx=0).stop 2023-10-16T14:34:53.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 msgr2=0x7fd4a01a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd49800ef30 tx=0x7fd49800c450 comp rx=0 tx=0).stop 2023-10-16T14:34:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 shutdown_connections 2023-10-16T14:34:53.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd48c0a1500 0x7fd48c0a39c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd4a01008d0 0x7fd4a01a0210 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4a00ff5b0 0x7fd4a0198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 --2- 172.21.15.178:0/90967353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4a00fff10 0x7fd4a01991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.031+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 >> 172.21.15.178:0/90967353 conn(0x7fd4a00fb150 msgr2=0x7fd4a00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.032+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 shutdown_connections 2023-10-16T14:34:53.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.032+0000 7fd4a6f9d700 1 -- 172.21.15.178:0/90967353 wait complete. 2023-10-16T14:34:53.039 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 27 2023-10-16T14:34:53.056 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:34:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 -- 172.21.15.178:0/3913097275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 msgr2=0x7f47801003c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 --2- 172.21.15.178:0/3913097275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f47801003c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f47700099c0 tx=0x7f4770009cd0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 -- 172.21.15.178:0/3913097275 shutdown_connections 2023-10-16T14:34:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 --2- 172.21.15.178:0/3913097275 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4780100900 0x7f4780100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 --2- 172.21.15.178:0/3913097275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47800ff5e0 0x7f47800ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 --2- 172.21.15.178:0/3913097275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f47801003c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 -- 172.21.15.178:0/3913097275 >> 172.21.15.178:0/3913097275 conn(0x7f47800fb160 msgr2=0x7f47800fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 -- 172.21.15.178:0/3913097275 shutdown_connections 2023-10-16T14:34:53.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.274+0000 7f478759d700 1 -- 172.21.15.178:0/3913097275 wait complete. 2023-10-16T14:34:53.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.275+0000 7f478759d700 1 Processor -- start 2023-10-16T14:34:53.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.275+0000 7f478759d700 1 -- start start 2023-10-16T14:34:53.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.275+0000 7f478759d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 0x7f4780198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478759d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478759d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 0x7f47801a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478759d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47801a06c0 con 0x7f47800fff40 2023-10-16T14:34:53.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478759d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47801a0800 con 0x7f47800ff5e0 2023-10-16T14:34:53.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478759d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47801a0990 con 0x7f4780100900 2023-10-16T14:34:53.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37708/0 (socket says 172.21.15.178:37708) 2023-10-16T14:34:53.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 -- 172.21.15.178:0/4273633290 learned_addr learned my addr 172.21.15.178:0/4273633290 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f478659b700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 0x7f4780198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4786d9c700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 0x7f47801a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 -- 172.21.15.178:0/4273633290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 msgr2=0x7f47801a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 0x7f47801a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.276+0000 7f4785d9a700 1 -- 172.21.15.178:0/4273633290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 msgr2=0x7f4780198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f4785d9a700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 0x7f4780198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f4785d9a700 1 -- 172.21.15.178:0/4273633290 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700095f0 con 0x7f47800fff40 2023-10-16T14:34:53.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f4786d9c700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 0x7f47801a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:53.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f478659b700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 0x7f4780198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f4785d9a700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4770009990 tx=0x7f477000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f47777fe700 1 -- 172.21.15.178:0/4273633290 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477001d070 con 0x7f47800fff40 2023-10-16T14:34:53.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.277+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47801a0c10 con 0x7f47800fff40 2023-10-16T14:34:53.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.278+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47801a1090 con 0x7f47800fff40 2023-10-16T14:34:53.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.278+0000 7f47777fe700 1 -- 172.21.15.178:0/4273633290 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47700054f0 con 0x7f47800fff40 2023-10-16T14:34:53.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.278+0000 7f47777fe700 1 -- 172.21.15.178:0/4273633290 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770021720 con 0x7f47800fff40 2023-10-16T14:34:53.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.283+0000 7f47777fe700 1 -- 172.21.15.178:0/4273633290 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f47700037d0 con 0x7f47800fff40 2023-10-16T14:34:53.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.284+0000 7f479c07cb80 1 Processor -- start 2023-10-16T14:34:53.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 -- start start 2023-10-16T14:34:53.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f47777fe700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 0x7f476c0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f47777fe700 1 -- 172.21.15.178:0/4273633290 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f47700cd930 con 0x7f47800fff40 2023-10-16T14:34:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dafb0 0x5582192db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192db910 0x5582192dc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821924fc70 con 0x5582192dafb0 2023-10-16T14:34:53.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558219234c60 con 0x5582192db910 2023-10-16T14:34:53.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.285+0000 7f479c07cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558219232580 con 0x5582192dc800 2023-10-16T14:34:53.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 0x7f476c0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43652/0 (socket says 172.21.15.178:43652) 2023-10-16T14:34:53.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 -- 172.21.15.178:0/2738136659 learned_addr learned my addr 172.21.15.178:0/2738136659 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f478659b700 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dafb0 0x5582192db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 -- 172.21.15.178:0/2738136659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192db910 msgr2=0x5582192dc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192db910 0x5582192dc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 -- 172.21.15.178:0/2738136659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dafb0 msgr2=0x5582192db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dafb0 0x5582192db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.286+0000 7f4786d9c700 1 -- 172.21.15.178:0/2738136659 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582191eb3f0 con 0x5582192dc800 2023-10-16T14:34:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.287+0000 7f4786d9c700 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4778019600 tx=0x7f4778019910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ef6a6d98aafe58f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.287+0000 7f4786d9c700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 0x7f476c0a3a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4778019c60 tx=0x7f477801d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.287+0000 7f4774ff9700 1 -- 172.21.15.178:0/2738136659 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778025070 con 0x5582192dc800 2023-10-16T14:34:53.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.287+0000 7f4774ff9700 1 -- 172.21.15.178:0/2738136659 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477801aae0 con 0x5582192dc800 2023-10-16T14:34:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.287+0000 7f4774ff9700 1 -- 172.21.15.178:0/2738136659 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800e040 con 0x5582192dc800 2023-10-16T14:34:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 -- 172.21.15.178:0/2738136659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 msgr2=0x5582192dcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4778019600 tx=0x7f4778019910 comp rx=0 tx=0).stop 2023-10-16T14:34:53.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 -- 172.21.15.178:0/2738136659 shutdown_connections 2023-10-16T14:34:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dc800 0x5582192dcc80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dafb0 0x5582192db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 --2- 172.21.15.178:0/2738136659 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192db910 0x5582192dc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 -- 172.21.15.178:0/2738136659 >> 172.21.15.178:0/2738136659 conn(0x558219208dd0 msgr2=0x5582192d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 -- 172.21.15.178:0/2738136659 shutdown_connections 2023-10-16T14:34:53.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.288+0000 7f479c07cb80 1 -- 172.21.15.178:0/2738136659 wait complete. 2023-10-16T14:34:53.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.289+0000 7f479c07cb80 1 -- start start 2023-10-16T14:34:53.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.289+0000 7f479c07cb80 1 Processor -- start 2023-10-16T14:34:53.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55821922de70 con 0x5582192db910 2023-10-16T14:34:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582191ca710 con 0x5582192dc800 2023-10-16T14:34:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582191ca3b0 con 0x5582192dafb0 2023-10-16T14:34:53.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f479c07cb80 10 client.0 _ll_register_callbacks cb 0x558219027800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:53.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f4786d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f4785d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f4786d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32844/0 (socket says 172.21.15.178:32844) 2023-10-16T14:34:53.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f4786d9c700 1 -- 172.21.15.178:0/367028288 learned_addr learned my addr 172.21.15.178:0/367028288 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.290+0000 7f478659b700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 -- 172.21.15.178:0/367028288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 msgr2=0x5582192ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 -- 172.21.15.178:0/367028288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 msgr2=0x5582192edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 -- 172.21.15.178:0/367028288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4778012d40 con 0x5582192dc800 2023-10-16T14:34:53.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f478659b700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4786d9c700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47780094c0 tx=0x7f47780099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af462b0e92539eab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4769ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:34:53.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778025070 con 0x5582192dc800 2023-10-16T14:34:53.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558219232ef0 con 0x5582192dc800 2023-10-16T14:34:53.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.291+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4778007500 con 0x5582192dc800 2023-10-16T14:34:53.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47780076c0 con 0x5582192dc800 2023-10-16T14:34:53.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 10 client.25616 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:53.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558219058de0 con 0x5582192dc800 2023-10-16T14:34:53.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558219232ef0 con 0x5582192dc800 2023-10-16T14:34:53.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4778014040 con 0x5582192dc800 2023-10-16T14:34:53.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 10 client.25616 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:53.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 10 client.25616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:34:53.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 20 client.25616 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:53.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5582192f9520 0x5582192fb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 4 client.25616 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:53.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.292+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582192aac10 con 0x5582192f9520 2023-10-16T14:34:53.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.293+0000 7f478659b700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5582192f9520 0x5582192fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.293+0000 7f478659b700 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5582192f9520 0x5582192fb940 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=578dbec0d7974732 server_cookie=8f94ed25b6691c29 in_seq=0 out_seq=0 2023-10-16T14:34:53.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.293+0000 7f4769ffb700 10 client.25616 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:53.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f4769ffb700 1 -- 172.21.15.178:0/367028288 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5582192aac10 con 0x5582192f9520 2023-10-16T14:34:53.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f4769ffb700 10 client.25616 handle_command_reply: tid=0 2023-10-16T14:34:53.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 client.25616 shutdown 2023-10-16T14:34:53.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 msgr2=0x5582192f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f47780094c0 tx=0x7f47780099a0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 shutdown_connections 2023-10-16T14:34:53.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5582192f9520 0x5582192fb940 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f4cb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f4769ffb700 0 client.25616 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:53.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 --2- 172.21.15.178:0/367028288 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 >> 172.21.15.178:0/367028288 conn(0x558219208dd0 msgr2=0x5582192d5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 shutdown_connections 2023-10-16T14:34:53.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.294+0000 7f479c07cb80 1 -- 172.21.15.178:0/367028288 wait complete. 2023-10-16T14:34:53.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.295+0000 7f479c07cb80 20 client.25616 trim_cache size 0 max 16384 2023-10-16T14:34:53.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.308+0000 7f479c07cb80 1 Processor -- start 2023-10-16T14:34:53.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.308+0000 7f479c07cb80 1 -- start start 2023-10-16T14:34:53.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192db910 0x5582192f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dc800 0x5582192ed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582192d1070 con 0x5582192dc800 2023-10-16T14:34:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582190ae920 con 0x5582192dafb0 2023-10-16T14:34:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f479c07cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558219232ef0 con 0x5582192db910 2023-10-16T14:34:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f478659b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f4786d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dc800 0x5582192ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f478659b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32856/0 (socket says 172.21.15.178:32856) 2023-10-16T14:34:53.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f478659b700 1 -- 172.21.15.178:0/3698408472 learned_addr learned my addr 172.21.15.178:0/3698408472 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.309+0000 7f4785d9a700 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192db910 0x5582192f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 -- 172.21.15.178:0/3698408472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192db910 msgr2=0x5582192f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192db910 0x5582192f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 -- 172.21.15.178:0/3698408472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dc800 msgr2=0x5582192ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dc800 0x5582192ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 -- 172.21.15.178:0/3698408472 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558219041c60 con 0x5582192dafb0 2023-10-16T14:34:53.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f478659b700 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5582192fc1b0 tx=0x7f477c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d35dbab8139759 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.310+0000 7f47757fa700 1 -- 172.21.15.178:0/3698408472 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c016040 con 0x5582192dafb0 2023-10-16T14:34:53.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.311+0000 7f47757fa700 1 -- 172.21.15.178:0/3698408472 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477c017040 con 0x5582192dafb0 2023-10-16T14:34:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.311+0000 7f47757fa700 1 -- 172.21.15.178:0/3698408472 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c004b20 con 0x5582192dafb0 2023-10-16T14:34:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.311+0000 7f479c07cb80 1 -- 172.21.15.178:0/3698408472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 msgr2=0x5582192ecde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.311+0000 7f479c07cb80 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x5582192fc1b0 tx=0x7f477c009260 comp rx=0 tx=0).stop 2023-10-16T14:34:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 -- 172.21.15.178:0/3698408472 shutdown_connections 2023-10-16T14:34:53.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192dc800 0x5582192ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dafb0 0x5582192ecde0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 --2- 172.21.15.178:0/3698408472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192db910 0x5582192f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 -- 172.21.15.178:0/3698408472 >> 172.21.15.178:0/3698408472 conn(0x558219208dd0 msgr2=0x5582192ec340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 -- 172.21.15.178:0/3698408472 shutdown_connections 2023-10-16T14:34:53.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 -- 172.21.15.178:0/3698408472 wait complete. 2023-10-16T14:34:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.312+0000 7f479c07cb80 1 -- start start 2023-10-16T14:34:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 Processor -- start 2023-10-16T14:34:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582192f41c0 con 0x5582192db910 2023-10-16T14:34:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582192f4330 con 0x5582192dc800 2023-10-16T14:34:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582192f44f0 con 0x5582192dafb0 2023-10-16T14:34:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.313+0000 7f479c07cb80 10 client.0 _ll_register_callbacks cb 0x7f477c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:53.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43678/0 (socket says 172.21.15.178:43678) 2023-10-16T14:34:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 -- 172.21.15.178:0/58275533 learned_addr learned my addr 172.21.15.178:0/58275533 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f4785d9a700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 -- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 msgr2=0x5582192f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 -- 172.21.15.178:0/58275533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 msgr2=0x5582192f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f478659b700 1 -- 172.21.15.178:0/58275533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c009e20 con 0x5582192dafb0 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.314+0000 7f4785d9a700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f478659b700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f477c004800 tx=0x7f477c021f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14d91fa921fa4e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f4769ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c011c30 con 0x5582192dafb0 2023-10-16T14:34:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f477c011d90 con 0x5582192dafb0 2023-10-16T14:34:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f477801aae0 con 0x5582192dafb0 2023-10-16T14:34:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f477c0048a0 con 0x5582192dafb0 2023-10-16T14:34:53.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f479c07cb80 10 client.25609 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:53.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.315+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55821907f250 con 0x5582192dafb0 2023-10-16T14:34:53.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477801aae0 con 0x5582192dafb0 2023-10-16T14:34:53.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f477c0277a0 con 0x5582192dafb0 2023-10-16T14:34:53.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 10 client.25609 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:53.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 10 client.25609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:34:53.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 20 client.25609 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:53.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55821934e0e0 0x558219350500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 4 client.25609 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:34:53.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.316+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558219226e50 con 0x55821934e0e0 2023-10-16T14:34:53.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f4785d9a700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55821934e0e0 0x558219350500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f4785d9a700 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55821934e0e0 0x558219350500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3593891d7bd07548 server_cookie=19614e6e21439321 in_seq=0 out_seq=0 2023-10-16T14:34:53.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f4769ffb700 10 client.25609 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:53.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f4769ffb700 1 -- 172.21.15.178:0/58275533 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558219226e50 con 0x55821934e0e0 2023-10-16T14:34:53.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f4769ffb700 10 client.25609 handle_command_reply: tid=0 2023-10-16T14:34:53.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f479c07cb80 1 client.25609 shutdown 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 msgr2=0x5582192f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.317+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f477c004800 tx=0x7f477c021f00 comp rx=0 tx=0).stop 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 shutdown_connections 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55821934e0e0 0x558219350500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5582192dc800 0x5582192f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5582192dafb0 0x5582192f2730 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 --2- 172.21.15.178:0/58275533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5582192db910 0x5582192f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 >> 172.21.15.178:0/58275533 conn(0x558219208dd0 msgr2=0x5582192d7540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f4769ffb700 0 client.25609 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:34:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 shutdown_connections 2023-10-16T14:34:53.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 1 -- 172.21.15.178:0/58275533 wait complete. 2023-10-16T14:34:53.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.318+0000 7f479c07cb80 20 client.25609 trim_cache size 0 max 16384 2023-10-16T14:34:53.342 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:34:53.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 msgr2=0x7f476c0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 0x7f476c0a3a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4778019c60 tx=0x7f477801d000 comp rx=0 tx=0).stop 2023-10-16T14:34:53.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 msgr2=0x7f4780199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4770009990 tx=0x7f477000f690 comp rx=0 tx=0).stop 2023-10-16T14:34:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 shutdown_connections 2023-10-16T14:34:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f476c0a15a0 0x7f476c0a3a60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4780100900 0x7f47801a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f47800ff5e0 0x7f4780198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 --2- 172.21.15.178:0/4273633290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47800fff40 0x7f4780199080 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 >> 172.21.15.178:0/4273633290 conn(0x7f47800fb160 msgr2=0x7f47800fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 shutdown_connections 2023-10-16T14:34:53.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.319+0000 7f478759d700 1 -- 172.21.15.178:0/4273633290 wait complete. 2023-10-16T14:34:53.348 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:34:53.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.558+0000 7efc0c3ff700 1 -- 172.21.15.178:0/1556910817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04074ef0 msgr2=0x7efc04073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.558+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/1556910817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04074ef0 0x7efc04073340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efc00009990 tx=0x7efc00009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 -- 172.21.15.178:0/1556910817 shutdown_connections 2023-10-16T14:34:53.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/1556910817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 0x7efc0410b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/1556910817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04074ef0 0x7efc04073340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/1556910817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc04073910 0x7efc04073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 -- 172.21.15.178:0/1556910817 >> 172.21.15.178:0/1556910817 conn(0x7efc040fc400 msgr2=0x7efc040fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 -- 172.21.15.178:0/1556910817 shutdown_connections 2023-10-16T14:34:53.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.559+0000 7efc0c3ff700 1 -- 172.21.15.178:0/1556910817 wait complete. 2023-10-16T14:34:53.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 Processor -- start 2023-10-16T14:34:53.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 -- start start 2023-10-16T14:34:53.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc04074ef0 0x7efc04199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 0x7efc041a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a09c0 con 0x7efc04074ef0 2023-10-16T14:34:53.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a0b00 con 0x7efc04108d90 2023-10-16T14:34:53.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.560+0000 7efc0c3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc041a0c90 con 0x7efc04073910 2023-10-16T14:34:53.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43692/0 (socket says 172.21.15.178:43692) 2023-10-16T14:34:53.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 -- 172.21.15.178:0/470256321 learned_addr learned my addr 172.21.15.178:0/470256321 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a99c700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 0x7efc041a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 -- 172.21.15.178:0/470256321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 msgr2=0x7efc041a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 0x7efc041a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 -- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc04074ef0 msgr2=0x7efc04199380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:34:53.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc04074ef0 0x7efc04199380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 -- 172.21.15.178:0/470256321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc000095f0 con 0x7efc04073910 2023-10-16T14:34:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.561+0000 7efc0a19b700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc00004730 tx=0x7efc00005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.562+0000 7efbfb7fe700 1 -- 172.21.15.178:0/470256321 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0001d070 con 0x7efc04073910 2023-10-16T14:34:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.562+0000 7efbfb7fe700 1 -- 172.21.15.178:0/470256321 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc00004cd0 con 0x7efc04073910 2023-10-16T14:34:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.562+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc041a0f10 con 0x7efc04073910 2023-10-16T14:34:53.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.562+0000 7efbfb7fe700 1 -- 172.21.15.178:0/470256321 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc00005610 con 0x7efc04073910 2023-10-16T14:34:53.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.562+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc041a1280 con 0x7efc04073910 2023-10-16T14:34:53.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.565+0000 7efbfb7fe700 1 -- 172.21.15.178:0/470256321 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efc0000fc70 con 0x7efc04073910 2023-10-16T14:34:53.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.567+0000 7efbfb7fe700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 0x7efbf00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.567+0000 7efbfb7fe700 1 -- 172.21.15.178:0/470256321 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efc000ce4e0 con 0x7efc04073910 2023-10-16T14:34:53.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.567+0000 7efc0999a700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 0x7efbf00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.568+0000 7efc0999a700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 0x7efbf00a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005fd0 tx=0x7efbf4005f60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.568+0000 7efc1fdeab80 1 Processor -- start 2023-10-16T14:34:53.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.568+0000 7efc1fdeab80 1 -- start start 2023-10-16T14:34:53.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc99729c70 con 0x55dc997b6620 2023-10-16T14:34:53.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9970ec60 con 0x55dc997b4e20 2023-10-16T14:34:53.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc9970c580 con 0x55dc997b5780 2023-10-16T14:34:53.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32900/0 (socket says 172.21.15.178:32900) 2023-10-16T14:34:53.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.569+0000 7efc0a19b700 1 -- 172.21.15.178:0/3847076218 learned_addr learned my addr 172.21.15.178:0/3847076218 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0999a700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a99c700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 -- 172.21.15.178:0/3847076218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 msgr2=0x55dc997b6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997b6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 -- 172.21.15.178:0/3847076218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 msgr2=0x55dc997b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 -- 172.21.15.178:0/3847076218 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc996c53f0 con 0x55dc997b4e20 2023-10-16T14:34:53.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0999a700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:53.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efc0a19b700 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc0009b450 tx=0x7efc00005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d33552f5d6e1463f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efbf8bfe700 1 -- 172.21.15.178:0/3847076218 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc0009b4c0 con 0x55dc997b4e20 2023-10-16T14:34:53.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efbf8bfe700 1 -- 172.21.15.178:0/3847076218 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc000ce070 con 0x55dc997b4e20 2023-10-16T14:34:53.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.570+0000 7efbf8bfe700 1 -- 172.21.15.178:0/3847076218 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc000998f0 con 0x55dc997b4e20 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3847076218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 msgr2=0x55dc997b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc0009b450 tx=0x7efc00005a90 comp rx=0 tx=0).stop 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3847076218 shutdown_connections 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b5240 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3847076218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997b6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3847076218 >> 172.21.15.178:0/3847076218 conn(0x55dc996e2dd0 msgr2=0x55dc997affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3847076218 shutdown_connections 2023-10-16T14:34:53.580 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.571+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3847076218 wait complete. 2023-10-16T14:34:53.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.572+0000 7efc1fdeab80 1 -- start start 2023-10-16T14:34:53.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.572+0000 7efc1fdeab80 1 Processor -- start 2023-10-16T14:34:53.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc99707e70 con 0x55dc997b6620 2023-10-16T14:34:53.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc996a4710 con 0x55dc997b4e20 2023-10-16T14:34:53.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc996a43b0 con 0x55dc997b5780 2023-10-16T14:34:53.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc1fdeab80 10 client.0 _ll_register_callbacks cb 0x55dc99501800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:53.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc0a99c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32914/0 (socket says 172.21.15.178:32914) 2023-10-16T14:34:53.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.573+0000 7efc0a19b700 1 -- 172.21.15.178:0/2027061188 learned_addr learned my addr 172.21.15.178:0/2027061188 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0999a700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 -- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 msgr2=0x55dc997c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 -- 172.21.15.178:0/2027061188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 msgr2=0x55dc997ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 -- 172.21.15.178:0/2027061188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc041a1280 con 0x55dc997b4e20 2023-10-16T14:34:53.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a99c700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:53.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc0a19b700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc0009bb20 tx=0x7efc0009af50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be738bba2de9a3f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efbedffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:34:53.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc000a3330 con 0x55dc997b4e20 2023-10-16T14:34:53.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efc000a3490 con 0x55dc997b4e20 2023-10-16T14:34:53.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.574+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc9970cef0 con 0x55dc997b4e20 2023-10-16T14:34:53.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc0009f170 con 0x55dc997b4e20 2023-10-16T14:34:53.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 10 client.25631 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:53.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dc99532de0 con 0x55dc997b4e20 2023-10-16T14:34:53.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dc9970cef0 con 0x55dc997b4e20 2023-10-16T14:34:53.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc00084af0 con 0x55dc997b4e20 2023-10-16T14:34:53.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 10 client.25631 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:53.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 10 client.25631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:34:53.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 20 client.25631 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:53.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc997d2b30 0x55dc997d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 4 client.25631 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:53.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.575+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc99784c10 con 0x55dc997d2b30 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.576+0000 7efc0999a700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc997d2b30 0x55dc997d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.576+0000 7efbedffb700 10 client.25631 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.576+0000 7efc0999a700 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc997d2b30 0x55dc997d4f50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82f3dc90245dc37e server_cookie=89380f08d9e80a40 in_seq=0 out_seq=0 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efbedffb700 1 -- 172.21.15.178:0/2027061188 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55dc99784c10 con 0x55dc997d2b30 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efbedffb700 10 client.25631 handle_command_reply: tid=0 2023-10-16T14:34:53.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 client.25631 shutdown 2023-10-16T14:34:53.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 msgr2=0x55dc997c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efc0009bb20 tx=0x7efc0009af50 comp rx=0 tx=0).stop 2023-10-16T14:34:53.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 shutdown_connections 2023-10-16T14:34:53.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc997d2b30 0x55dc997d4f50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997c7260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/2027061188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efbedffb700 0 client.25631 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:53.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.577+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 >> 172.21.15.178:0/2027061188 conn(0x55dc996e2dd0 msgr2=0x55dc997afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.578+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 shutdown_connections 2023-10-16T14:34:53.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.578+0000 7efc1fdeab80 1 -- 172.21.15.178:0/2027061188 wait complete. 2023-10-16T14:34:53.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.578+0000 7efc1fdeab80 20 client.25631 trim_cache size 0 max 16384 2023-10-16T14:34:53.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.603+0000 7efc1fdeab80 1 Processor -- start 2023-10-16T14:34:53.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.603+0000 7efc1fdeab80 1 -- start start 2023-10-16T14:34:53.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc99588920 con 0x55dc997b6620 2023-10-16T14:34:53.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc997b48a0 con 0x55dc997b4e20 2023-10-16T14:34:53.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc997bdf00 con 0x55dc997b5780 2023-10-16T14:34:53.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc0999a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32918/0 (socket says 172.21.15.178:32918) 2023-10-16T14:34:53.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.604+0000 7efc0a19b700 1 -- 172.21.15.178:0/3819406111 learned_addr learned my addr 172.21.15.178:0/3819406111 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0a99c700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 -- 172.21.15.178:0/3819406111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 msgr2=0x55dc997b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 -- 172.21.15.178:0/3819406111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 msgr2=0x55dc997cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 -- 172.21.15.178:0/3819406111 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc9951bc60 con 0x55dc997b5780 2023-10-16T14:34:53.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0a19b700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0a99c700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:53.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efc0999a700 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997ce7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55dc997d2f10 tx=0x7efbf4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=368db48cc85b1304 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.605+0000 7efbf93ff700 1 -- 172.21.15.178:0/3819406111 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4029070 con 0x55dc997b5780 2023-10-16T14:34:53.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.606+0000 7efbf93ff700 1 -- 172.21.15.178:0/3819406111 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf40044f0 con 0x55dc997b5780 2023-10-16T14:34:53.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.606+0000 7efbf93ff700 1 -- 172.21.15.178:0/3819406111 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4024040 con 0x55dc997b5780 2023-10-16T14:34:53.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.606+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3819406111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 msgr2=0x55dc997ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.606+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997ce7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55dc997d2f10 tx=0x7efbf4012000 comp rx=0 tx=0).stop 2023-10-16T14:34:53.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3819406111 shutdown_connections 2023-10-16T14:34:53.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b6620 0x55dc997cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b4e20 0x55dc997b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/3819406111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b5780 0x55dc997ce7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3819406111 >> 172.21.15.178:0/3819406111 conn(0x55dc996e2dd0 msgr2=0x55dc997bd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3819406111 shutdown_connections 2023-10-16T14:34:53.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 -- 172.21.15.178:0/3819406111 wait complete. 2023-10-16T14:34:53.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.607+0000 7efc1fdeab80 1 -- start start 2023-10-16T14:34:53.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 Processor -- start 2023-10-16T14:34:53.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b5780 0x55dc997ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b6620 0x55dc997cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc997ce090 con 0x55dc997b5780 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc997ce200 con 0x55dc997b6620 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc997ce3c0 con 0x55dc997b4e20 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.608+0000 7efc1fdeab80 10 client.0 _ll_register_callbacks cb 0x55dc99501800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43724/0 (socket says 172.21.15.178:43724) 2023-10-16T14:34:53.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 -- 172.21.15.178:0/1780302336 learned_addr learned my addr 172.21.15.178:0/1780302336 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0999a700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b5780 0x55dc997ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 -- 172.21.15.178:0/1780302336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b6620 msgr2=0x55dc997cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b6620 0x55dc997cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 -- 172.21.15.178:0/1780302336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b5780 msgr2=0x55dc997ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b5780 0x55dc997ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 -- 172.21.15.178:0/1780302336 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc99532de0 con 0x55dc997b4e20 2023-10-16T14:34:53.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efc0a19b700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc0009b680 tx=0x7efc00009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=351883ec6498e1a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.609+0000 7efbedffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc000a1760 con 0x55dc997b4e20 2023-10-16T14:34:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efc00004ed0 con 0x55dc997b4e20 2023-10-16T14:34:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc000ce070 con 0x55dc997b4e20 2023-10-16T14:34:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc000a15e0 con 0x55dc997b4e20 2023-10-16T14:34:53.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 10 client.25624 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7efbf400da00 con 0x55dc997b4e20 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc000ce070 con 0x55dc997b4e20 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efc000a3330 con 0x55dc997b4e20 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 10 client.25624 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 10 client.25624 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:34:53.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 20 client.25624 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:53.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc99827590 0x55dc998299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 4 client.25624 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:34:53.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.610+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc99700e50 con 0x55dc99827590 2023-10-16T14:34:53.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efc0999a700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc99827590 0x55dc998299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efc0999a700 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc99827590 0x55dc998299b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2e28040a51a33d2e server_cookie=23e3c3e66d15fa98 in_seq=0 out_seq=0 2023-10-16T14:34:53.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efbedffb700 10 client.25624 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:53.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efbedffb700 1 -- 172.21.15.178:0/1780302336 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc99700e50 con 0x55dc99827590 2023-10-16T14:34:53.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efbedffb700 10 client.25624 handle_command_reply: tid=0 2023-10-16T14:34:53.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.611+0000 7efc1fdeab80 1 client.25624 shutdown 2023-10-16T14:34:53.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 msgr2=0x55dc997cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efc0009b680 tx=0x7efc00009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 shutdown_connections 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55dc99827590 0x55dc998299b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55dc997b6620 0x55dc997cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc997b4e20 0x55dc997cc600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 --2- 172.21.15.178:0/1780302336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55dc997b5780 0x55dc997ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 >> 172.21.15.178:0/1780302336 conn(0x55dc996e2dd0 msgr2=0x55dc997cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efbedffb700 0 client.25624 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:34:53.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 shutdown_connections 2023-10-16T14:34:53.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 1 -- 172.21.15.178:0/1780302336 wait complete. 2023-10-16T14:34:53.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.612+0000 7efc1fdeab80 20 client.25624 trim_cache size 0 max 16384 2023-10-16T14:34:53.629 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:34:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 msgr2=0x7efbf00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 0x7efbf00a39c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005fd0 tx=0x7efbf4005f60 comp rx=0 tx=0).stop 2023-10-16T14:34:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 msgr2=0x7efc04198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7efc00004730 tx=0x7efc00005ef0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 shutdown_connections 2023-10-16T14:34:53.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efbf00a1500 0x7efbf00a39c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efc04108d90 0x7efc041a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc04073910 0x7efc04198e40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 --2- 172.21.15.178:0/470256321 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc04074ef0 0x7efc04199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 >> 172.21.15.178:0/470256321 conn(0x7efc040fc400 msgr2=0x7efc040fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.613+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 shutdown_connections 2023-10-16T14:34:53.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.614+0000 7efc0c3ff700 1 -- 172.21.15.178:0/470256321 wait complete. 2023-10-16T14:34:53.637 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:34:53.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 -- 172.21.15.178:0/2550734384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 msgr2=0x7fe9781024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/2550734384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781024e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009990 tx=0x7fe96c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 -- 172.21.15.178:0/2550734384 shutdown_connections 2023-10-16T14:34:53.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/2550734384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978102a20 0x7fe978109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/2550734384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978100e80 0x7fe9781012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/2550734384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781024e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 -- 172.21.15.178:0/2550734384 >> 172.21.15.178:0/2550734384 conn(0x7fe9780fc400 msgr2=0x7fe9780fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 -- 172.21.15.178:0/2550734384 shutdown_connections 2023-10-16T14:34:53.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.946+0000 7fe97c9ff700 1 -- 172.21.15.178:0/2550734384 wait complete. 2023-10-16T14:34:53.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.948+0000 7fe97c9ff700 1 Processor -- start 2023-10-16T14:34:53.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.948+0000 7fe97c9ff700 1 -- start start 2023-10-16T14:34:53.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.948+0000 7fe97c9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97c9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97c9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 0x7fe97819e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97c9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97819e800 con 0x7fe978102a20 2023-10-16T14:34:53.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97c9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97819e970 con 0x7fe978100e80 2023-10-16T14:34:53.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97c9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97819eb30 con 0x7fe978102060 2023-10-16T14:34:53.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe976d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 0x7fe97819e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe975d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32950/0 (socket says 172.21.15.178:32950) 2023-10-16T14:34:53.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.949+0000 7fe97659c700 1 -- 172.21.15.178:0/3746523738 learned_addr learned my addr 172.21.15.178:0/3746523738 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 -- 172.21.15.178:0/3746523738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 msgr2=0x7fe978196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 -- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 msgr2=0x7fe97819e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 0x7fe97819e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 -- 172.21.15.178:0/3746523738 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0095f0 con 0x7fe978102060 2023-10-16T14:34:53.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe97659c700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe976d9d700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 0x7fe97819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe975d9b700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781971d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0098e0 tx=0x7fe96c0057b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe9677fe700 1 -- 172.21.15.178:0/3746523738 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c01d070 con 0x7fe978102060 2023-10-16T14:34:53.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe9677fe700 1 -- 172.21.15.178:0/3746523738 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c022470 con 0x7fe978102060 2023-10-16T14:34:53.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9780731a0 con 0x7fe978102060 2023-10-16T14:34:53.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.950+0000 7fe9677fe700 1 -- 172.21.15.178:0/3746523738 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c017650 con 0x7fe978102060 2023-10-16T14:34:53.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.951+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe978073650 con 0x7fe978102060 2023-10-16T14:34:53.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.956+0000 7fe9677fe700 1 -- 172.21.15.178:0/3746523738 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe96c022800 con 0x7fe978102060 2023-10-16T14:34:53.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.957+0000 7fe9903a5b80 1 Processor -- start 2023-10-16T14:34:53.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.957+0000 7fe9903a5b80 1 -- start start 2023-10-16T14:34:53.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 0x563b24cd0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24c3bc70 con 0x563b24cd0e30 2023-10-16T14:34:53.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24c20c60 con 0x563b24ccf630 2023-10-16T14:34:53.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24c1e580 con 0x563b24ccff90 2023-10-16T14:34:53.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe976d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe97659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 0x563b24cd0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe9677fe700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 0x7fe9600a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe976d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32960/0 (socket says 172.21.15.178:32960) 2023-10-16T14:34:53.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.958+0000 7fe976d9d700 1 -- 172.21.15.178:0/2022408497 learned_addr learned my addr 172.21.15.178:0/2022408497 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe9677fe700 1 -- 172.21.15.178:0/3746523738 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe96c0ce5f0 con 0x7fe978102060 2023-10-16T14:34:53.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 0x7fe9600a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 -- 172.21.15.178:0/2022408497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 msgr2=0x563b24cd0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe975d9b700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 0x563b24cd0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 -- 172.21.15.178:0/2022408497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 msgr2=0x563b24cd12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe976d9d700 1 -- 172.21.15.178:0/2022408497 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b24bd73f0 con 0x563b24ccf630 2023-10-16T14:34:53.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe97659c700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 0x563b24cd0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.959+0000 7fe975d9b700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:34:53.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.960+0000 7fe976d9d700 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe970009550 tx=0x7fe9700175d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d8c4a3094a92e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.960+0000 7fe976d9d700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 0x7fe9600a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe97001c000 tx=0x7fe97001d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.960+0000 7fe964ff9700 1 -- 172.21.15.178:0/2022408497 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970017d40 con 0x563b24ccf630 2023-10-16T14:34:53.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.960+0000 7fe964ff9700 1 -- 172.21.15.178:0/2022408497 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97001e070 con 0x563b24ccf630 2023-10-16T14:34:53.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.960+0000 7fe964ff9700 1 -- 172.21.15.178:0/2022408497 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe970004290 con 0x563b24ccf630 2023-10-16T14:34:53.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 -- 172.21.15.178:0/2022408497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 msgr2=0x563b24ccfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe970009550 tx=0x7fe9700175d0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 -- 172.21.15.178:0/2022408497 shutdown_connections 2023-10-16T14:34:53.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccf630 0x563b24ccfa50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/2022408497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccff90 0x563b24cd0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 -- 172.21.15.178:0/2022408497 >> 172.21.15.178:0/2022408497 conn(0x563b24bf4dd0 msgr2=0x563b24cc1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.961+0000 7fe9903a5b80 1 -- 172.21.15.178:0/2022408497 shutdown_connections 2023-10-16T14:34:53.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.962+0000 7fe9903a5b80 1 -- 172.21.15.178:0/2022408497 wait complete. 2023-10-16T14:34:53.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.962+0000 7fe9903a5b80 1 -- start start 2023-10-16T14:34:53.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 Processor -- start 2023-10-16T14:34:53.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccff90 0x563b24cd9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24c19e70 con 0x563b24ccff90 2023-10-16T14:34:53.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24bb6710 con 0x563b24cd0e30 2023-10-16T14:34:53.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.963+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24bb63b0 con 0x563b24ccf630 2023-10-16T14:34:53.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe9903a5b80 10 client.0 _ll_register_callbacks cb 0x563b24a13800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:53.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe976d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe976d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32966/0 (socket says 172.21.15.178:32966) 2023-10-16T14:34:53.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe976d9d700 1 -- 172.21.15.178:0/1495551927 learned_addr learned my addr 172.21.15.178:0/1495551927 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:53.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe975d9b700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccff90 0x563b24cd9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe97659c700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.964+0000 7fe976d9d700 1 -- 172.21.15.178:0/1495551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 msgr2=0x563b24cd94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe976d9d700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe976d9d700 1 -- 172.21.15.178:0/1495551927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccff90 msgr2=0x563b24cd9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:53.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe976d9d700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccff90 0x563b24cd9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe976d9d700 1 -- 172.21.15.178:0/1495551927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe968009d50 con 0x563b24cd0e30 2023-10-16T14:34:53.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe97659c700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:53.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe976d9d700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe970009fd0 tx=0x7fe9700077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794e59c6289be2b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:53.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.965+0000 7fe95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:34:53.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97002c680 con 0x563b24cd0e30 2023-10-16T14:34:53.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b24c1eef0 con 0x563b24cd0e30 2023-10-16T14:34:53.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe97002c7e0 con 0x563b24cd0e30 2023-10-16T14:34:53.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe970003b10 con 0x563b24cd0e30 2023-10-16T14:34:53.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe9903a5b80 10 client.25646 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:53.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe970007a40 con 0x563b24cd0e30 2023-10-16T14:34:53.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.966+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b24c1eef0 con 0x563b24cd0e30 2023-10-16T14:34:53.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe970012050 con 0x563b24cd0e30 2023-10-16T14:34:53.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 10 client.25646 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 20 client.25646 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24ce5170 0x563b24ce7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 4 client.25646 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b24c96c10 con 0x563b24ce5170 2023-10-16T14:34:53.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.967+0000 7fe97659c700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24ce5170 0x563b24ce7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:53.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.968+0000 7fe97659c700 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24ce5170 0x563b24ce7590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617d7a0563b780b0 server_cookie=84accbebc64ec835 in_seq=0 out_seq=0 2023-10-16T14:34:53.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.968+0000 7fe95dffb700 10 client.25646 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:53.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe95dffb700 1 -- 172.21.15.178:0/1495551927 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563b24c96c10 con 0x563b24ce5170 2023-10-16T14:34:53.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe95dffb700 10 client.25646 handle_command_reply: tid=0 2023-10-16T14:34:53.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 client.25646 shutdown 2023-10-16T14:34:53.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 msgr2=0x563b24ce0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:53.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe970009fd0 tx=0x7fe9700077e0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 shutdown_connections 2023-10-16T14:34:53.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24ce5170 0x563b24ce7590 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24cd0e30 0x563b24ce0a40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/1495551927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccff90 0x563b24cd9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 >> 172.21.15.178:0/1495551927 conn(0x563b24bf4dd0 msgr2=0x563b24cc1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.969+0000 7fe95dffb700 0 client.25646 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.970+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 shutdown_connections 2023-10-16T14:34:53.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.970+0000 7fe9903a5b80 1 -- 172.21.15.178:0/1495551927 wait complete. 2023-10-16T14:34:53.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.970+0000 7fe9903a5b80 20 client.25646 trim_cache size 0 max 16384 2023-10-16T14:34:53.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.995+0000 7fe9903a5b80 1 Processor -- start 2023-10-16T14:34:53.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.995+0000 7fe9903a5b80 1 -- start start 2023-10-16T14:34:53.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24ce0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24a9a920 con 0x563b24cd0e30 2023-10-16T14:34:54.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24cd06e0 con 0x563b24ccff90 2023-10-16T14:34:54.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24cd7bc0 con 0x563b24ccf630 2023-10-16T14:34:54.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe97659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe97659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43770/0 (socket says 172.21.15.178:43770) 2023-10-16T14:34:54.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe97659c700 1 -- 172.21.15.178:0/4200337281 learned_addr learned my addr 172.21.15.178:0/4200337281 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:54.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe975d9b700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24ce0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.996+0000 7fe976d9d700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 -- 172.21.15.178:0/4200337281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 msgr2=0x563b24cd3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:54.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 -- 172.21.15.178:0/4200337281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 msgr2=0x563b24cd4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:54.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 -- 172.21.15.178:0/4200337281 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b24a44de0 con 0x563b24ccff90 2023-10-16T14:34:54.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe976d9d700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:54.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe975d9b700 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24ce0a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x563b24cd0610 tx=0x7fe96c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e8277ed554e614 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:54.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe9657fa700 1 -- 172.21.15.178:0/4200337281 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c09bcd0 con 0x563b24ccff90 2023-10-16T14:34:54.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe9657fa700 1 -- 172.21.15.178:0/4200337281 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe96c09b5d0 con 0x563b24ccff90 2023-10-16T14:34:54.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.997+0000 7fe9657fa700 1 -- 172.21.15.178:0/4200337281 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c099a80 con 0x563b24ccff90 2023-10-16T14:34:54.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.998+0000 7fe9903a5b80 1 -- 172.21.15.178:0/4200337281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 msgr2=0x563b24ce0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:54.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.998+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24ce0a00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x563b24cd0610 tx=0x7fe96c005fd0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 -- 172.21.15.178:0/4200337281 shutdown_connections 2023-10-16T14:34:54.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24cd0e30 0x563b24cd4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24ccf630 0x563b24cd3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/4200337281 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24ce0a00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 -- 172.21.15.178:0/4200337281 >> 172.21.15.178:0/4200337281 conn(0x563b24bf4dd0 msgr2=0x563b24cd80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:54.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 -- 172.21.15.178:0/4200337281 shutdown_connections 2023-10-16T14:34:54.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 -- 172.21.15.178:0/4200337281 wait complete. 2023-10-16T14:34:54.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:53.999+0000 7fe9903a5b80 1 -- start start 2023-10-16T14:34:54.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 Processor -- start 2023-10-16T14:34:54.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccf630 0x563b24cde260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24cd0e30 0x563b24cdf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24cdfcf0 con 0x563b24ccf630 2023-10-16T14:34:54.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24cdfe60 con 0x563b24ccff90 2023-10-16T14:34:54.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b24ce0020 con 0x563b24cd0e30 2023-10-16T14:34:54.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.000+0000 7fe9903a5b80 10 client.0 _ll_register_callbacks cb 0x7fe968009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:54.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:32990/0 (socket says 172.21.15.178:32990) 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 -- 172.21.15.178:0/962351130 learned_addr learned my addr 172.21.15.178:0/962351130 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe97659c700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccf630 0x563b24cde260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe976d9d700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24cd0e30 0x563b24cdf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 -- 172.21.15.178:0/962351130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24cd0e30 msgr2=0x563b24cdf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24cd0e30 0x563b24cdf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 -- 172.21.15.178:0/962351130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccf630 msgr2=0x563b24cde260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:54.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccf630 0x563b24cde260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.001+0000 7fe975d9b700 1 -- 172.21.15.178:0/962351130 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe970007a40 con 0x563b24ccff90 2023-10-16T14:34:54.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe975d9b700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe96c086010 tx=0x7fe96c09ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ee47b0579011a15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:54.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe95dffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:34:54.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c09f720 con 0x563b24ccff90 2023-10-16T14:34:54.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe96c0a38d0 con 0x563b24ccff90 2023-10-16T14:34:54.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe97001e070 con 0x563b24ccff90 2023-10-16T14:34:54.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe96c09bcd0 con 0x563b24ccff90 2023-10-16T14:34:54.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe9903a5b80 10 client.25652 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:54.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.002+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fe978073650 con 0x563b24ccff90 2023-10-16T14:34:54.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe97001e070 con 0x563b24ccff90 2023-10-16T14:34:54.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe96c00fad0 con 0x563b24ccff90 2023-10-16T14:34:54.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 10 client.25652 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:54.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 10 client.25652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:54.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 20 client.25652 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24d396a0 0x563b24d3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 4 client.25652 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b24c12e50 con 0x563b24d396a0 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.003+0000 7fe97659c700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24d396a0 0x563b24d3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe97659c700 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24d396a0 0x563b24d3bac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747d26f1371c960b server_cookie=1b966cbef3218a29 in_seq=0 out_seq=0 2023-10-16T14:34:54.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe95dffb700 10 client.25652 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:54.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe95dffb700 1 -- 172.21.15.178:0/962351130 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b24c12e50 con 0x563b24d396a0 2023-10-16T14:34:54.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe95dffb700 10 client.25652 handle_command_reply: tid=0 2023-10-16T14:34:54.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 client.25652 shutdown 2023-10-16T14:34:54.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 msgr2=0x563b24cde7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:54.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe96c086010 tx=0x7fe96c09ae50 comp rx=0 tx=0).stop 2023-10-16T14:34:54.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 shutdown_connections 2023-10-16T14:34:54.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563b24d396a0 0x563b24d3bac0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b24cd0e30 0x563b24cdf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563b24ccf630 0x563b24cde260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 --2- 172.21.15.178:0/962351130 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563b24ccff90 0x563b24cde7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.004+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 >> 172.21.15.178:0/962351130 conn(0x563b24bf4dd0 msgr2=0x563b24cd3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:54.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.005+0000 7fe95dffb700 0 client.25652 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:54.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.005+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 shutdown_connections 2023-10-16T14:34:54.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.005+0000 7fe9903a5b80 1 -- 172.21.15.178:0/962351130 wait complete. 2023-10-16T14:34:54.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.005+0000 7fe9903a5b80 20 client.25652 trim_cache size 0 max 16384 2023-10-16T14:34:54.020 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:34:54.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 msgr2=0x7fe9600a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:54.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 0x7fe9600a3940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe97001c000 tx=0x7fe97001d000 comp rx=0 tx=0).stop 2023-10-16T14:34:54.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 msgr2=0x7fe9781971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:54.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781971d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe96c0098e0 tx=0x7fe96c0057b0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 shutdown_connections 2023-10-16T14:34:54.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9600a1480 0x7fe9600a3940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe978102a20 0x7fe97819e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe978100e80 0x7fe978196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 --2- 172.21.15.178:0/3746523738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe978102060 0x7fe9781971d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:54.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 >> 172.21.15.178:0/3746523738 conn(0x7fe9780fc400 msgr2=0x7fe9780fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:54.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.006+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 shutdown_connections 2023-10-16T14:34:54.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:54.007+0000 7fe97c9ff700 1 -- 172.21.15.178:0/3746523738 wait complete. 2023-10-16T14:34:54.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:53 smithi178 ceph-mon[98495]: pgmap v374: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:54.034 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:53 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/90967353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:53 smithi186 ceph-mon[134892]: pgmap v374: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:54.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:53 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/90967353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:54.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:53 smithi187 ceph-mon[135297]: pgmap v374: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:34:54.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:53 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/90967353' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:34:55.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:34:55.038 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:34:55.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.266+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3338496865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c0073910 msgr2=0x7fd2c0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.266+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3338496865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c0073910 0x7fd2c0073d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009990 tx=0x7fd2b0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.266+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3338496865 shutdown_connections 2023-10-16T14:34:55.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3338496865 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c010d4a0 0x7fd2c010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3338496865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c0073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3338496865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c0073910 0x7fd2c0073d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3338496865 >> 172.21.15.178:0/3338496865 conn(0x7fd2c00fc400 msgr2=0x7fd2c00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3338496865 shutdown_connections 2023-10-16T14:34:55.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.267+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3338496865 wait complete. 2023-10-16T14:34:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.268+0000 7fd2c85ff700 1 Processor -- start 2023-10-16T14:34:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.268+0000 7fd2c85ff700 1 -- start start 2023-10-16T14:34:55.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 0x7fd2c01a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a4da0 con 0x7fd2c0074ef0 2023-10-16T14:34:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a4ee0 con 0x7fd2c0073910 2023-10-16T14:34:55.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c85ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c01a5070 con 0x7fd2c010d4a0 2023-10-16T14:34:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c639b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c639b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:33006/0 (socket says 172.21.15.178:33006) 2023-10-16T14:34:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c639b700 1 -- 172.21.15.178:0/3847864248 learned_addr learned my addr 172.21.15.178:0/3847864248 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.269+0000 7fd2c639b700 1 -- 172.21.15.178:0/3847864248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 msgr2=0x7fd2c01a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 0x7fd2c01a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c639b700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 0x7fd2c01a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c639b700 1 -- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 msgr2=0x7fd2c019d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c639b700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c019d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c639b700 1 -- 172.21.15.178:0/3847864248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00095f0 con 0x7fd2c0073910 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 0x7fd2c01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c639b700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e9a0 tx=0x7fd2bc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2b77fe700 1 -- 172.21.15.178:0/3847864248 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc009790 con 0x7fd2c0073910 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c01a5350 con 0x7fd2c0073910 2023-10-16T14:34:55.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2b77fe700 1 -- 172.21.15.178:0/3847864248 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2bc00cbd0 con 0x7fd2c0073910 2023-10-16T14:34:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2b77fe700 1 -- 172.21.15.178:0/3847864248 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc018680 con 0x7fd2c0073910 2023-10-16T14:34:55.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.270+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c01a5860 con 0x7fd2c0073910 2023-10-16T14:34:55.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.275+0000 7fd2b77fe700 1 -- 172.21.15.178:0/3847864248 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd2bc00cd40 con 0x7fd2c0073910 2023-10-16T14:34:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 Processor -- start 2023-10-16T14:34:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 -- start start 2023-10-16T14:34:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d429244cf0 con 0x55d4292d16a0 2023-10-16T14:34:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d429229ce0 con 0x55d4292cfea0 2023-10-16T14:34:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.276+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d429227600 con 0x55d4292d0800 2023-10-16T14:34:55.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c639b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c639b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43810/0 (socket says 172.21.15.178:43810) 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c639b700 1 -- 172.21.15.178:0/1166520650 learned_addr learned my addr 172.21.15.178:0/1166520650 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/1166520650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 msgr2=0x55d4292d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/1166520650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 msgr2=0x55d4292d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.277+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/1166520650 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4291e0470 con 0x55d4292cfea0 2023-10-16T14:34:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2b77fe700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 0x7fd2ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d02c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009420 tx=0x7fd2b8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e14f1ab69c239f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2b77fe700 1 -- 172.21.15.178:0/3847864248 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2bc014070 con 0x7fd2c0073910 2023-10-16T14:34:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2c639b700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 0x7fd2ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2b4bfe700 1 -- 172.21.15.178:0/1166520650 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8014610 con 0x55d4292cfea0 2023-10-16T14:34:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2b4bfe700 1 -- 172.21.15.178:0/1166520650 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b800f070 con 0x55d4292cfea0 2023-10-16T14:34:55.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.278+0000 7fd2b4bfe700 1 -- 172.21.15.178:0/1166520650 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80139e0 con 0x55d4292cfea0 2023-10-16T14:34:55.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/1166520650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 msgr2=0x55d4292d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2c639b700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 0x7fd2ac0a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0caaa0 tx=0x7fd2bc0d0bb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d02c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8009420 tx=0x7fd2b8009920 comp rx=0 tx=0).stop 2023-10-16T14:34:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/1166520650 shutdown_connections 2023-10-16T14:34:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d02c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/1166520650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/1166520650 >> 172.21.15.178:0/1166520650 conn(0x55d4291fde50 msgr2=0x55d4292cb060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/1166520650 shutdown_connections 2023-10-16T14:34:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.279+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/1166520650 wait complete. 2023-10-16T14:34:55.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.280+0000 7fd2dbf73b80 1 -- start start 2023-10-16T14:34:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.280+0000 7fd2dbf73b80 1 Processor -- start 2023-10-16T14:34:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292da200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d429222ef0 con 0x55d4292d16a0 2023-10-16T14:34:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4291bf790 con 0x55d4292cfea0 2023-10-16T14:34:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4291bf430 con 0x55d4292d0800 2023-10-16T14:34:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2dbf73b80 10 client.0 _ll_register_callbacks cb 0x55d42901c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:33036/0 (socket says 172.21.15.178:33036) 2023-10-16T14:34:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.281+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/25630386 learned_addr learned my addr 172.21.15.178:0/25630386 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292da200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 -- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 msgr2=0x55d4292d9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 -- 172.21.15.178:0/25630386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 msgr2=0x55d4292e1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c5b9a700 1 -- 172.21.15.178:0/25630386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b80090d0 con 0x55d4292d0800 2023-10-16T14:34:55.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.282+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:55.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.283+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292da200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b00053b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48846903b804767e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.283+0000 7fd2a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:55.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.283+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001d070 con 0x55d4292d0800 2023-10-16T14:34:55.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.283+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d429227f70 con 0x55d4292d0800 2023-10-16T14:34:55.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.283+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2b00049d0 con 0x55d4292d0800 2023-10-16T14:34:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b0004b90 con 0x55d4292d0800 2023-10-16T14:34:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 10 client.25651 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d42904de60 con 0x55d4292d0800 2023-10-16T14:34:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d429227f70 con 0x55d4292d0800 2023-10-16T14:34:55.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2b0024450 con 0x55d4292d0800 2023-10-16T14:34:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 10 client.25651 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 10 client.25651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 20 client.25651 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d4292e5b60 0x55d4292e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 4 client.25651 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.284+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d42929fc90 con 0x55d4292e5b60 2023-10-16T14:34:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.285+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d4292e5b60 0x55d4292e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.285+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d4292e5b60 0x55d4292e7f80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217a65ff4f939e59 server_cookie=d54fc02cf867ff4d in_seq=0 out_seq=0 2023-10-16T14:34:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.285+0000 7fd2a9ffb700 10 client.25651 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.286+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/25630386 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d42929fc90 con 0x55d4292e5b60 2023-10-16T14:34:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.286+0000 7fd2a9ffb700 10 client.25651 handle_command_reply: tid=0 2023-10-16T14:34:55.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.286+0000 7fd2dbf73b80 1 client.25651 shutdown 2023-10-16T14:34:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.286+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 msgr2=0x55d4292da200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292da200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00098e0 tx=0x7fd2b00053b0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 shutdown_connections 2023-10-16T14:34:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d4292e5b60 0x55d4292e7f80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2a9ffb700 0 client.25651 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/25630386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292da200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 >> 172.21.15.178:0/25630386 conn(0x55d4291fde50 msgr2=0x55d4292cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 shutdown_connections 2023-10-16T14:34:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/25630386 wait complete. 2023-10-16T14:34:55.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.287+0000 7fd2dbf73b80 20 client.25651 trim_cache size 0 max 16384 2023-10-16T14:34:55.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.304+0000 7fd2dbf73b80 1 Processor -- start 2023-10-16T14:34:55.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 -- start start 2023-10-16T14:34:55.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4290a39a0 con 0x55d4292d16a0 2023-10-16T14:34:55.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4292d9170 con 0x55d4292cfea0 2023-10-16T14:34:55.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4292d9830 con 0x55d4292d0800 2023-10-16T14:34:55.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:33052/0 (socket says 172.21.15.178:33052) 2023-10-16T14:34:55.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/630257384 learned_addr learned my addr 172.21.15.178:0/630257384 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.305+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/630257384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 msgr2=0x55d4292d4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/630257384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 msgr2=0x55d4292e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/630257384 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4292ecbb0 con 0x55d4292cfea0 2023-10-16T14:34:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8016010 tx=0x7fd2b8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a480893d5364988f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2b53ff700 1 -- 172.21.15.178:0/630257384 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8026040 con 0x55d4292cfea0 2023-10-16T14:34:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.306+0000 7fd2b53ff700 1 -- 172.21.15.178:0/630257384 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b802edf0 con 0x55d4292cfea0 2023-10-16T14:34:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2b53ff700 1 -- 172.21.15.178:0/630257384 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b802e3b0 con 0x55d4292cfea0 2023-10-16T14:34:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/630257384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 msgr2=0x55d4292d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8016010 tx=0x7fd2b8009920 comp rx=0 tx=0).stop 2023-10-16T14:34:55.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/630257384 shutdown_connections 2023-10-16T14:34:55.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d4292e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292cfea0 0x55d4292d4570 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/630257384 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292d0800 0x55d4292d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/630257384 >> 172.21.15.178:0/630257384 conn(0x55d4291fde50 msgr2=0x55d4292d9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/630257384 shutdown_connections 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.307+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/630257384 wait complete. 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.308+0000 7fd2dbf73b80 1 -- start start 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.308+0000 7fd2dbf73b80 1 Processor -- start 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 0x55d4292ddf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 0x55d4292de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d429339980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4292dea50 con 0x55d4292d16a0 2023-10-16T14:34:55.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4292debc0 con 0x55d4292d0800 2023-10-16T14:34:55.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d429339ef0 con 0x55d4292cfea0 2023-10-16T14:34:55.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2dbf73b80 10 client.0 _ll_register_callbacks cb 0x7fd2b000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:55.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 0x55d4292ddf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2c6b9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 0x55d4292ddf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43824/0 (socket says 172.21.15.178:43824) 2023-10-16T14:34:55.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/2939988753 learned_addr learned my addr 172.21.15.178:0/2939988753 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d429339980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.309+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 0x55d4292de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/2939988753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 msgr2=0x55d4292ddf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 0x55d4292ddf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 msgr2=0x55d4292de4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 0x55d4292de4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 -- 172.21.15.178:0/2939988753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b8029040 con 0x55d4292d16a0 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 0x55d4292de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:34:55.320 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2c6b9c700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d429339980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8000f80 tx=0x7fd2b8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3548bffc21462ccc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.310+0000 7fd2a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:34:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8026070 con 0x55d4292d16a0 2023-10-16T14:34:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2b800f070 con 0x55d4292d16a0 2023-10-16T14:34:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd2b8004a70 con 0x55d4292d16a0 2023-10-16T14:34:55.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2b8004c30 con 0x55d4292d16a0 2023-10-16T14:34:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2dbf73b80 10 client.15621 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d429036ce0 con 0x55d4292d16a0 2023-10-16T14:34:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.311+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2b800f070 con 0x55d4292d16a0 2023-10-16T14:34:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd2b8037050 con 0x55d4292d16a0 2023-10-16T14:34:55.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 20 client.15621 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d42933b720 0x55d42933db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 4 client.15621 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:34:55.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d42921bed0 con 0x55d42933b720 2023-10-16T14:34:55.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.312+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d42933b720 0x55d42933db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.313+0000 7fd2a9ffb700 10 client.15621 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.313+0000 7fd2c5b9a700 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d42933b720 0x55d42933db40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446abba464e0762e server_cookie=2aa76a98fff1ca47 in_seq=0 out_seq=0 2023-10-16T14:34:55.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2a9ffb700 1 -- 172.21.15.178:0/2939988753 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d42921bed0 con 0x55d42933b720 2023-10-16T14:34:55.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2a9ffb700 10 client.15621 handle_command_reply: tid=0 2023-10-16T14:34:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 client.15621 shutdown 2023-10-16T14:34:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 msgr2=0x55d429339980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d429339980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8000f80 tx=0x7fd2b8009f60 comp rx=0 tx=0).stop 2023-10-16T14:34:55.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 shutdown_connections 2023-10-16T14:34:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d42933b720 0x55d42933db40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4292d16a0 0x55d429339980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4292cfea0 0x55d4292ddf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 --2- 172.21.15.178:0/2939988753 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4292d0800 0x55d4292de4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 >> 172.21.15.178:0/2939988753 conn(0x55d4291fde50 msgr2=0x55d4292d9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.314+0000 7fd2a9ffb700 0 client.15621 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.315+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 shutdown_connections 2023-10-16T14:34:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.315+0000 7fd2dbf73b80 1 -- 172.21.15.178:0/2939988753 wait complete. 2023-10-16T14:34:55.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.315+0000 7fd2dbf73b80 20 client.15621 trim_cache size 0 max 16384 2023-10-16T14:34:55.329 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:55.329 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:34:55.330 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "2e213273-1b23-48f1-976a-65b10b34cde8", 2023-10-16T14:34:55.330 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:34:55.330 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:55.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 msgr2=0x7fd2ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 0x7fd2ac0a3940 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0caaa0 tx=0x7fd2bc0d0bb0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 msgr2=0x7fd2c019d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00e9a0 tx=0x7fd2bc00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 shutdown_connections 2023-10-16T14:34:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd2ac0a1480 0x7fd2ac0a3940 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2c010d4a0 0x7fd2c01a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd2c0073910 0x7fd2c019d220 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 --2- 172.21.15.178:0/3847864248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2c0074ef0 0x7fd2c019d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 >> 172.21.15.178:0/3847864248 conn(0x7fd2c00fc400 msgr2=0x7fd2c00fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 shutdown_connections 2023-10-16T14:34:55.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.316+0000 7fd2c85ff700 1 -- 172.21.15.178:0/3847864248 wait complete. 2023-10-16T14:34:55.338 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:34:55.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 -- 172.21.15.178:0/3571759752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 msgr2=0x7fc0cc0732c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/3571759752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 0x7fc0cc0732c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0098f0 tx=0x7fc0bc009c00 comp rx=0 tx=0).stop 2023-10-16T14:34:55.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 -- 172.21.15.178:0/3571759752 shutdown_connections 2023-10-16T14:34:55.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/3571759752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc108c00 0x7fc0cc10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/3571759752 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 0x7fc0cc0732c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/3571759752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0cc073890 0x7fc0cc073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.623+0000 7fc0cb59e700 1 -- 172.21.15.178:0/3571759752 >> 172.21.15.178:0/3571759752 conn(0x7fc0cc0fc280 msgr2=0x7fc0cc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.624+0000 7fc0cb59e700 1 -- 172.21.15.178:0/3571759752 shutdown_connections 2023-10-16T14:34:55.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.624+0000 7fc0cb59e700 1 -- 172.21.15.178:0/3571759752 wait complete. 2023-10-16T14:34:55.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 Processor -- start 2023-10-16T14:34:55.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 -- start start 2023-10-16T14:34:55.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 0x7fc0cc1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0cc108c00 0x7fc0cc1a0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1a06f0 con 0x7fc0cc108c00 2023-10-16T14:34:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1a0830 con 0x7fc0cc074e70 2023-10-16T14:34:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0cb59e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc1a09c0 con 0x7fc0cc073890 2023-10-16T14:34:55.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43838/0 (socket says 172.21.15.178:43838) 2023-10-16T14:34:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.625+0000 7fc0ca59c700 1 -- 172.21.15.178:0/4131704264 learned_addr learned my addr 172.21.15.178:0/4131704264 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 -- 172.21.15.178:0/4131704264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 msgr2=0x7fc0cc1990b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:34:55.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 0x7fc0cc1990b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 -- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0cc108c00 msgr2=0x7fc0cc1a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0cc108c00 0x7fc0cc1a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 -- 172.21.15.178:0/4131704264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009520 con 0x7fc0cc073890 2023-10-16T14:34:55.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009fd0 tx=0x7fc0bc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0bb7fe700 1 -- 172.21.15.178:0/4131704264 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc01c070 con 0x7fc0cc073890 2023-10-16T14:34:55.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc1a0c40 con 0x7fc0cc073890 2023-10-16T14:34:55.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.626+0000 7fc0bb7fe700 1 -- 172.21.15.178:0/4131704264 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc00f820 con 0x7fc0cc073890 2023-10-16T14:34:55.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.627+0000 7fc0bb7fe700 1 -- 172.21.15.178:0/4131704264 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc004c70 con 0x7fc0cc073890 2023-10-16T14:34:55.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.627+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc1a10c0 con 0x7fc0cc073890 2023-10-16T14:34:55.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.631+0000 7fc0bb7fe700 1 -- 172.21.15.178:0/4131704264 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc0bc020070 con 0x7fc0cc073890 2023-10-16T14:34:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.632+0000 7fc0e497cb80 1 Processor -- start 2023-10-16T14:34:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.632+0000 7fc0e497cb80 1 -- start start 2023-10-16T14:34:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef516be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef517a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef482c70 con 0x5564ef517120 2023-10-16T14:34:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef467c60 con 0x5564ef518010 2023-10-16T14:34:55.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef465580 con 0x5564ef5167c0 2023-10-16T14:34:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0bb7fe700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 0x7fc0b40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0bb7fe700 1 -- 172.21.15.178:0/4131704264 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0bc0cdf50 con 0x7fc0cc073890 2023-10-16T14:34:55.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0c9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0c9d9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:33082/0 (socket says 172.21.15.178:33082) 2023-10-16T14:34:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.633+0000 7fc0c9d9b700 1 -- 172.21.15.178:0/548653483 learned_addr learned my addr 172.21.15.178:0/548653483 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef517a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 0x7fc0b40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 -- 172.21.15.178:0/548653483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 msgr2=0x5564ef516be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef516be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef516be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 -- 172.21.15.178:0/548653483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 msgr2=0x5564ef517a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef517a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 -- 172.21.15.178:0/548653483 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564ef41e3f0 con 0x5564ef518010 2023-10-16T14:34:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef517a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef516be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:34:55.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009b50 tx=0x7fc0c0009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b30a04f5e18bbad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.634+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 0x7fc0b40a3a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009690 tx=0x7fc0c400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.635+0000 7fc0b8ff9700 1 -- 172.21.15.178:0/548653483 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000eae0 con 0x5564ef518010 2023-10-16T14:34:55.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.635+0000 7fc0b8ff9700 1 -- 172.21.15.178:0/548653483 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c0004cf0 con 0x5564ef518010 2023-10-16T14:34:55.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.635+0000 7fc0b8ff9700 1 -- 172.21.15.178:0/548653483 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0005620 con 0x5564ef518010 2023-10-16T14:34:55.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- 172.21.15.178:0/548653483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 msgr2=0x5564ef518490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009b50 tx=0x7fc0c0009e60 comp rx=0 tx=0).stop 2023-10-16T14:34:55.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- 172.21.15.178:0/548653483 shutdown_connections 2023-10-16T14:34:55.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef518490 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef516be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/548653483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef517a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- 172.21.15.178:0/548653483 >> 172.21.15.178:0/548653483 conn(0x5564ef43bdd0 msgr2=0x5564ef50a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- 172.21.15.178:0/548653483 shutdown_connections 2023-10-16T14:34:55.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- 172.21.15.178:0/548653483 wait complete. 2023-10-16T14:34:55.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.636+0000 7fc0e497cb80 1 -- start start 2023-10-16T14:34:55.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 Processor -- start 2023-10-16T14:34:55.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef520b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef527b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef460e70 con 0x5564ef517120 2023-10-16T14:34:55.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef3fd710 con 0x5564ef518010 2023-10-16T14:34:55.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef3fd3b0 con 0x5564ef5167c0 2023-10-16T14:34:55.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.637+0000 7fc0e497cb80 10 client.0 _ll_register_callbacks cb 0x5564ef25a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:55.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0c9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef520b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43854/0 (socket says 172.21.15.178:43854) 2023-10-16T14:34:55.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 -- 172.21.15.178:0/2233462162 learned_addr learned my addr 172.21.15.178:0/2233462162 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef527b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 -- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 msgr2=0x5564ef527b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef527b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 -- 172.21.15.178:0/2233462162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 msgr2=0x5564ef520b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef520b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 -- 172.21.15.178:0/2233462162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c0009800 con 0x5564ef5167c0 2023-10-16T14:34:55.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef527b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:34:55.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.638+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0979a0 tx=0x7fc0bc09a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b94676e89d3a6597 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:55.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc09f080 con 0x5564ef5167c0 2023-10-16T14:34:55.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0bc0a3230 con 0x5564ef5167c0 2023-10-16T14:34:55.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564ef465ef0 con 0x5564ef5167c0 2023-10-16T14:34:55.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0bc0cb790 con 0x5564ef5167c0 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0e497cb80 10 client.25666 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564ef28bde0 con 0x5564ef5167c0 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.639+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0bc0b7020 con 0x5564ef5167c0 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0bc0a2e30 con 0x5564ef5167c0 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 10 client.25666 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:55.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 10 client.25666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:55.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 20 client.25666 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:55.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef52bf70 0x5564ef52e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 4 client.25666 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:34:55.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564ef465ef0 con 0x5564ef52bf70 2023-10-16T14:34:55.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.640+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef52bf70 0x5564ef52e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.641+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef52bf70 0x5564ef52e3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e7749a99958dfd server_cookie=77d9996dfe6b66fa in_seq=0 out_seq=0 2023-10-16T14:34:55.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.641+0000 7fc0b1ffb700 10 client.25666 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/2233462162 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5564ef465ef0 con 0x5564ef52bf70 2023-10-16T14:34:55.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0b1ffb700 10 client.25666 handle_command_reply: tid=0 2023-10-16T14:34:55.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 client.25666 shutdown 2023-10-16T14:34:55.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 msgr2=0x5564ef5205e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0979a0 tx=0x7fc0bc09a910 comp rx=0 tx=0).stop 2023-10-16T14:34:55.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 shutdown_connections 2023-10-16T14:34:55.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef52bf70 0x5564ef52e3d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef527b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0b1ffb700 0 client.25666 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef5205e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/2233462162 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef520b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 >> 172.21.15.178:0/2233462162 conn(0x5564ef43bdd0 msgr2=0x5564ef508d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 shutdown_connections 2023-10-16T14:34:55.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.642+0000 7fc0e497cb80 1 -- 172.21.15.178:0/2233462162 wait complete. 2023-10-16T14:34:55.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.643+0000 7fc0e497cb80 20 client.25666 trim_cache size 0 max 16384 2023-10-16T14:34:55.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.660+0000 7fc0e497cb80 1 Processor -- start 2023-10-16T14:34:55.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 -- start start 2023-10-16T14:34:55.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef508cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef527af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef509200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef504070 con 0x5564ef517120 2023-10-16T14:34:55.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef2e1920 con 0x5564ef518010 2023-10-16T14:34:55.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef465ef0 con 0x5564ef5167c0 2023-10-16T14:34:55.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.661+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef508cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0ca59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef508cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43860/0 (socket says 172.21.15.178:43860) 2023-10-16T14:34:55.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0ca59c700 1 -- 172.21.15.178:0/3686686821 learned_addr learned my addr 172.21.15.178:0/3686686821 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef527af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef509200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 -- 172.21.15.178:0/3686686821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 msgr2=0x5564ef508cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef508cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 -- 172.21.15.178:0/3686686821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 msgr2=0x5564ef527af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef527af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 -- 172.21.15.178:0/3686686821 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564ef274c60 con 0x5564ef518010 2023-10-16T14:34:55.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.662+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef509200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40060b0 tx=0x7fc0c4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758fbd5989734934 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.663+0000 7fc0b97fa700 1 -- 172.21.15.178:0/3686686821 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4007b70 con 0x5564ef518010 2023-10-16T14:34:55.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.663+0000 7fc0b97fa700 1 -- 172.21.15.178:0/3686686821 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c400f040 con 0x5564ef518010 2023-10-16T14:34:55.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.663+0000 7fc0b97fa700 1 -- 172.21.15.178:0/3686686821 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4014670 con 0x5564ef518010 2023-10-16T14:34:55.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.663+0000 7fc0e497cb80 1 -- 172.21.15.178:0/3686686821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 msgr2=0x5564ef509200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.663+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef509200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc0c40060b0 tx=0x7fc0c4013000 comp rx=0 tx=0).stop 2023-10-16T14:34:55.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 -- 172.21.15.178:0/3686686821 shutdown_connections 2023-10-16T14:34:55.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef518010 0x5564ef509200 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef5167c0 0x5564ef508cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/3686686821 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef517120 0x5564ef527af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 -- 172.21.15.178:0/3686686821 >> 172.21.15.178:0/3686686821 conn(0x5564ef43bdd0 msgr2=0x5564ef51e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 -- 172.21.15.178:0/3686686821 shutdown_connections 2023-10-16T14:34:55.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 -- 172.21.15.178:0/3686686821 wait complete. 2023-10-16T14:34:55.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.664+0000 7fc0e497cb80 1 -- start start 2023-10-16T14:34:55.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 Processor -- start 2023-10-16T14:34:55.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef5167c0 0x5564ef5278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 0x5564ef525eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef526970 con 0x5564ef5167c0 2023-10-16T14:34:55.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef526ae0 con 0x5564ef517120 2023-10-16T14:34:55.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564ef526ca0 con 0x5564ef518010 2023-10-16T14:34:55.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.665+0000 7fc0e497cb80 10 client.0 _ll_register_callbacks cb 0x7fc0bc09adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:34:55.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43872/0 (socket says 172.21.15.178:43872) 2023-10-16T14:34:55.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 -- 172.21.15.178:0/363021843 learned_addr learned my addr 172.21.15.178:0/363021843 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:34:55.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef5167c0 0x5564ef5278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 0x5564ef525eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 -- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 msgr2=0x5564ef525eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 0x5564ef525eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 -- 172.21.15.178:0/363021843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef5167c0 msgr2=0x5564ef5278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:34:55.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef5167c0 0x5564ef5278a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0cad9d700 1 -- 172.21.15.178:0/363021843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c401d040 con 0x5564ef518010 2023-10-16T14:34:55.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.666+0000 7fc0c9d9b700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 0x5564ef525eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:34:55.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.667+0000 7fc0cad9d700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009510 tx=0x7fc0c4007d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997204f4b38136c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:34:55.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.667+0000 7fc0b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:34:55.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.667+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4022690 con 0x5564ef518010 2023-10-16T14:34:55.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.667+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc0c40227f0 con 0x5564ef518010 2023-10-16T14:34:55.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.667+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0c0004cf0 con 0x5564ef518010 2023-10-16T14:34:55.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.668+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0c4003a00 con 0x5564ef518010 2023-10-16T14:34:55.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.668+0000 7fc0e497cb80 10 client.25672 fetch_fsmap learned FSMap version 27 2023-10-16T14:34:55.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.668+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564ef2b2250 con 0x5564ef518010 2023-10-16T14:34:55.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.668+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0c0004cf0 con 0x5564ef518010 2023-10-16T14:34:55.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.668+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc0c4014410 con 0x5564ef518010 2023-10-16T14:34:55.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 10 client.25672 fetch_fsmap finished waiting for FSMap version 27 2023-10-16T14:34:55.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 10 client.25672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:34:55.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 20 client.25672 populate_metadata read hostname 'smithi178' 2023-10-16T14:34:55.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef50f3a0 0x5564ef5117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:34:55.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 4 client.25672 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:34:55.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5564ef4ddc10 con 0x5564ef50f3a0 2023-10-16T14:34:55.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef50f3a0 0x5564ef5117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:34:55.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0ca59c700 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef50f3a0 0x5564ef5117c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be0329328b12beb server_cookie=6991fd41b8cd6df7 in_seq=0 out_seq=0 2023-10-16T14:34:55.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.669+0000 7fc0b1ffb700 10 client.25672 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0b1ffb700 1 -- 172.21.15.178:0/363021843 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5564ef4ddc10 con 0x5564ef50f3a0 2023-10-16T14:34:55.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0b1ffb700 10 client.25672 handle_command_reply: tid=0 2023-10-16T14:34:55.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 client.25672 shutdown 2023-10-16T14:34:55.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 msgr2=0x5564ef5263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009510 tx=0x7fc0c4007d80 comp rx=0 tx=0).stop 2023-10-16T14:34:55.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 shutdown_connections 2023-10-16T14:34:55.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5564ef50f3a0 0x5564ef5117c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564ef518010 0x5564ef5263f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5564ef5167c0 0x5564ef5278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 --2- 172.21.15.178:0/363021843 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5564ef517120 0x5564ef525eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0b1ffb700 0 client.25672 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 >> 172.21.15.178:0/363021843 conn(0x5564ef43bdd0 msgr2=0x5564ef51e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 shutdown_connections 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 1 -- 172.21.15.178:0/363021843 wait complete. 2023-10-16T14:34:55.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.670+0000 7fc0e497cb80 20 client.25672 trim_cache size 0 max 16384 2023-10-16T14:34:55.705 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:34:55.705 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-16T14:34:55.705 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:34:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: "2e213273-1b23-48f1-976a-65b10b34cde8": { 2023-10-16T14:34:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:34:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "2e213273-1b23-48f1-976a-65b10b34cde8", 2023-10-16T14:34:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:34:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:34:55.707 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:34:55.707 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:34:55.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 msgr2=0x7fc0b40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 0x7fc0b40a3a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009690 tx=0x7fc0c400b040 comp rx=0 tx=0).stop 2023-10-16T14:34:55.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 msgr2=0x7fc0cc198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc009fd0 tx=0x7fc0bc00b840 comp rx=0 tx=0).stop 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 shutdown_connections 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc0b40a15a0 0x7fc0b40a3a60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0cc108c00 0x7fc0cc1a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0cc073890 0x7fc0cc198b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 --2- 172.21.15.178:0/4131704264 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc0cc074e70 0x7fc0cc1990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:34:55.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 >> 172.21.15.178:0/4131704264 conn(0x7fc0cc0fc280 msgr2=0x7fc0cc0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:34:55.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 shutdown_connections 2023-10-16T14:34:55.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:34:55.671+0000 7fc0cb59e700 1 -- 172.21.15.178:0/4131704264 wait complete. 2023-10-16T14:34:55.710 INFO:tasks.cephfs.filesystem:scrub status for tag:2e213273-1b23-48f1-976a-65b10b34cde8 - {'path': '/', 'tag': '2e213273-1b23-48f1-976a-65b10b34cde8', 'options': 'recursive,force'} 2023-10-16T14:34:56.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:55 smithi178 ceph-mon[98495]: pgmap v375: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:56.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:55 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:34:56.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:55 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:55 smithi186 ceph-mon[134892]: pgmap v375: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:55 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:34:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:55 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:56.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:55 smithi187 ceph-mon[135297]: pgmap v375: 129 pgs: 129 active+clean; 668 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:34:56.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:55 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:34:56.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:55 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:56 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:34:57.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:56 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:56 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:34:57.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:56 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:56 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:34:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:56 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:56 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:34:57.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:56 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:56 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:34:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:56 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:56 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:34:57.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:56 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:34:58.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:57 smithi178 ceph-mon[98495]: pgmap v376: 129 pgs: 129 active+clean; 670 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:34:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:57 smithi186 ceph-mon[134892]: pgmap v376: 129 pgs: 129 active+clean; 670 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:34:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:57 smithi187 ceph-mon[135297]: pgmap v376: 129 pgs: 129 active+clean; 670 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:35:00.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:34:59 smithi178 ceph-mon[98495]: pgmap v377: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:35:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:34:59 smithi186 ceph-mon[134892]: pgmap v377: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:35:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:34:59 smithi187 ceph-mon[135297]: pgmap v377: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:35:01.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:01 smithi178 ceph-mon[98495]: pgmap v378: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:01.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:01 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:35:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:01 smithi186 ceph-mon[134892]: pgmap v378: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:01.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:01 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:35:01.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:01 smithi187 ceph-mon[135297]: pgmap v378: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:01.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:01 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:35:02.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:02 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:35:02.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:02 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:35:02.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:02 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:35:03.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:03 smithi178 ceph-mon[98495]: pgmap v379: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:03.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:03 smithi186 ceph-mon[134892]: pgmap v379: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:03.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:03 smithi187 ceph-mon[135297]: pgmap v379: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2023-10-16T14:35:05.446 INFO:tasks.workunit.client.0.smithi178.stderr:Note: switching to '33be745ec4531df08cd1bb70b4c9c1f89be58078'. 2023-10-16T14:35:05.446 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.447 INFO:tasks.workunit.client.0.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-16T14:35:05.447 INFO:tasks.workunit.client.0.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-16T14:35:05.447 INFO:tasks.workunit.client.0.smithi178.stderr:state without impacting any branches by switching back to a branch. 2023-10-16T14:35:05.448 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.448 INFO:tasks.workunit.client.0.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-16T14:35:05.449 INFO:tasks.workunit.client.0.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-16T14:35:05.449 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.449 INFO:tasks.workunit.client.0.smithi178.stderr: git switch -c 2023-10-16T14:35:05.450 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.450 INFO:tasks.workunit.client.0.smithi178.stderr:Or undo this operation with: 2023-10-16T14:35:05.450 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.451 INFO:tasks.workunit.client.0.smithi178.stderr: git switch - 2023-10-16T14:35:05.451 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.452 INFO:tasks.workunit.client.0.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-16T14:35:05.452 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:05.452 INFO:tasks.workunit.client.0.smithi178.stderr:HEAD is now at 33be745ec45 Merge remote-tracking branches 'temp2/adding-useful-print', 'milind/mds-enqueue-all-child-frags-for-scrub-for-a-given-fragset', 'temp3/fix-mds-metadata-lost', 'rishabh/before-fs-rename-fail-fs' and 'temp1/origin/tmp2' into rishabh-2023oct13-b2 2023-10-16T14:35:05.461 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-16T14:35:05.515 INFO:tasks.workunit.client.0.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-16T14:35:05.519 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-16T14:35:05.519 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-16T14:35:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:05 smithi178 ceph-mon[98495]: pgmap v380: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:05 smithi186 ceph-mon[134892]: pgmap v380: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:06.118 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-16T14:35:06.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:05 smithi187 ceph-mon[135297]: pgmap v380: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:06.231 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-16T14:35:06.296 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-16T14:35:06.300 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-16T14:35:06.300 INFO:tasks.workunit.client.0.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-16T14:35:06.364 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-16T14:35:06.371 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:35:06.371 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-16T14:35:06.415 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2023-10-16T14:35:06.416 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2023-10-16T14:35:06.417 DEBUG:teuthology.orchestra.run.smithi178:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=33be745ec4531df08cd1bb70b4c9c1f89be58078 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2023-10-16T14:35:06.514 INFO:tasks.workunit.client.0.smithi178.stdout:getting blogbench 2023-10-16T14:35:06.514 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'getting blogbench' 2023-10-16T14:35:06.514 INFO:tasks.workunit.client.0.smithi178.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-16T14:35:06.558 INFO:tasks.workunit.client.0.smithi178.stderr:--2023-10-16 14:35:06-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2023-10-16T14:35:06.559 INFO:tasks.workunit.client.0.smithi178.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2023-10-16T14:35:06.580 INFO:tasks.workunit.client.0.smithi178.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2023-10-16T14:35:06.601 INFO:tasks.workunit.client.0.smithi178.stderr:HTTP request sent, awaiting response... 200 OK 2023-10-16T14:35:06.601 INFO:tasks.workunit.client.0.smithi178.stderr:Length: 103355 (101K) [application/octet-stream] 2023-10-16T14:35:06.602 INFO:tasks.workunit.client.0.smithi178.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2023-10-16T14:35:06.606 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:06.642 INFO:tasks.workunit.client.0.smithi178.stderr: 0K .......... .......... .......... .......... .......... 49% 1.24M 0s 2023-10-16T14:35:06.683 INFO:tasks.workunit.client.0.smithi178.stderr: 50K .......... .......... .......... .......... .......... 99% 2.38M 0s 2023-10-16T14:35:06.683 INFO:tasks.workunit.client.0.smithi178.stderr: 100K 100% 46.5K=0.08s 2023-10-16T14:35:06.683 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:06.684 INFO:tasks.workunit.client.0.smithi178.stderr:2023-10-16 14:35:06 (1.23 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2023-10-16T14:35:06.684 INFO:tasks.workunit.client.0.smithi178.stderr: 2023-10-16T14:35:06.684 INFO:tasks.workunit.client.0.smithi178.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2023-10-16T14:35:06.727 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/ 2023-10-16T14:35:06.729 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/m4/ 2023-10-16T14:35:06.730 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/m4/acx_pthread.m4 2023-10-16T14:35:06.731 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/man/ 2023-10-16T14:35:06.733 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/man/blogbench.8 2023-10-16T14:35:06.734 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/man/Makefile.am 2023-10-16T14:35:06.735 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/man/Makefile.in 2023-10-16T14:35:06.736 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/ 2023-10-16T14:35:06.737 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/mysnprintf.c 2023-10-16T14:35:06.739 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/mysnprintf.h 2023-10-16T14:35:06.740 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/blogbench.c 2023-10-16T14:35:06.741 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/blogbench.h 2023-10-16T14:35:06.742 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/fakesnprintf.c 2023-10-16T14:35:06.743 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/fakesnprintf.h 2023-10-16T14:35:06.744 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/blogbench_p.h 2023-10-16T14:35:06.745 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/bsd-getopt_long.c 2023-10-16T14:35:06.746 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/bsd-getopt_long.h 2023-10-16T14:35:06.746 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/Makefile.am 2023-10-16T14:35:06.747 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/Makefile.in 2023-10-16T14:35:06.748 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/process.c 2023-10-16T14:35:06.749 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/globals.h 2023-10-16T14:35:06.750 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/commenter.c 2023-10-16T14:35:06.751 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/rewriter.c 2023-10-16T14:35:06.752 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/reader.c 2023-10-16T14:35:06.753 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/writer.c 2023-10-16T14:35:06.754 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/src/helpers.c 2023-10-16T14:35:06.755 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/NEWS 2023-10-16T14:35:06.757 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/compile 2023-10-16T14:35:06.757 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/depcomp 2023-10-16T14:35:06.758 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/aclocal.m4 2023-10-16T14:35:06.759 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/README 2023-10-16T14:35:06.760 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/configure 2023-10-16T14:35:06.764 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/configure.ac 2023-10-16T14:35:06.765 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/config.guess 2023-10-16T14:35:06.766 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/install-sh 2023-10-16T14:35:06.767 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/config.sub 2023-10-16T14:35:06.768 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/missing 2023-10-16T14:35:06.769 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/mkinstalldirs 2023-10-16T14:35:06.770 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/Makefile.am 2023-10-16T14:35:06.771 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/Makefile.in 2023-10-16T14:35:06.771 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/config.h.in 2023-10-16T14:35:06.773 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/AUTHORS 2023-10-16T14:35:06.774 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/INSTALL 2023-10-16T14:35:06.775 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/ChangeLog 2023-10-16T14:35:06.775 INFO:tasks.workunit.client.0.smithi178.stdout:blogbench-1.0/COPYING 2023-10-16T14:35:06.777 INFO:tasks.workunit.client.0.smithi178.stderr:+ cd blogbench-1.0/ 2023-10-16T14:35:06.778 INFO:tasks.workunit.client.0.smithi178.stdout:making blogbench 2023-10-16T14:35:06.778 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'making blogbench' 2023-10-16T14:35:06.778 INFO:tasks.workunit.client.0.smithi178.stderr:+ ./configure 2023-10-16T14:35:07.052 INFO:tasks.workunit.client.0.smithi178.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-16T14:35:08.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:07 smithi178 ceph-mon[98495]: pgmap v381: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:08.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:35:08.064 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether build environment is sane... yes 2023-10-16T14:35:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:07 smithi186 ceph-mon[134892]: pgmap v381: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:35:08.082 INFO:tasks.workunit.client.0.smithi178.stdout:checking for gawk... gawk 2023-10-16T14:35:08.117 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether make sets $(MAKE)... yes 2023-10-16T14:35:08.137 INFO:tasks.workunit.client.0.smithi178.stdout:checking for gcc... gcc 2023-10-16T14:35:08.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:07 smithi187 ceph-mon[135297]: pgmap v381: 129 pgs: 129 active+clean; 676 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-10-16T14:35:08.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:35:08.229 INFO:tasks.workunit.client.0.smithi178.stdout:checking for C compiler default output file name... a.out 2023-10-16T14:35:08.234 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether the C compiler works... yes 2023-10-16T14:35:08.238 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether we are cross compiling... no 2023-10-16T14:35:08.300 INFO:tasks.workunit.client.0.smithi178.stdout:checking for suffix of executables... 2023-10-16T14:35:08.365 INFO:tasks.workunit.client.0.smithi178.stdout:checking for suffix of object files... o 2023-10-16T14:35:08.428 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether we are using the GNU C compiler... yes 2023-10-16T14:35:08.500 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether gcc accepts -g... yes 2023-10-16T14:35:08.586 INFO:tasks.workunit.client.0.smithi178.stdout:checking for gcc option to accept ANSI C... none needed 2023-10-16T14:35:08.653 INFO:tasks.workunit.client.0.smithi178.stdout:checking for style of include used by make... GNU 2023-10-16T14:35:08.786 INFO:tasks.workunit.client.0.smithi178.stdout:checking dependency style of gcc... gcc3 2023-10-16T14:35:08.786 INFO:tasks.workunit.client.0.smithi178.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2023-10-16T14:35:08.869 INFO:tasks.workunit.client.0.smithi178.stdout:checking how to run the C preprocessor... gcc -E 2023-10-16T14:35:08.946 INFO:tasks.workunit.client.0.smithi178.stdout:checking for egrep... grep -E 2023-10-16T14:35:08.964 INFO:tasks.workunit.client.0.smithi178.stdout:checking for AIX... no 2023-10-16T14:35:09.065 INFO:tasks.workunit.client.0.smithi178.stdout:checking for library containing strerror... none required 2023-10-16T14:35:09.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:35:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:35:09.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:08 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.298 INFO:tasks.workunit.client.0.smithi178.stdout:checking for ANSI C header files... yes 2023-10-16T14:35:09.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:35:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:35:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:08 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:35:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:35:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:08 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:35:09.371 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether time.h and sys/time.h may both be included... yes 2023-10-16T14:35:09.459 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/types.h... yes 2023-10-16T14:35:09.551 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/stat.h... yes 2023-10-16T14:35:09.645 INFO:tasks.workunit.client.0.smithi178.stdout:checking for stdlib.h... yes 2023-10-16T14:35:09.738 INFO:tasks.workunit.client.0.smithi178.stdout:checking for string.h... yes 2023-10-16T14:35:09.832 INFO:tasks.workunit.client.0.smithi178.stdout:checking for memory.h... yes 2023-10-16T14:35:09.928 INFO:tasks.workunit.client.0.smithi178.stdout:checking for strings.h... yes 2023-10-16T14:35:10.024 INFO:tasks.workunit.client.0.smithi178.stdout:checking for inttypes.h... yes 2023-10-16T14:35:10.118 INFO:tasks.workunit.client.0.smithi178.stdout:checking for stdint.h... yes 2023-10-16T14:35:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:09 smithi187 ceph-mon[135297]: pgmap v382: 129 pgs: 129 active+clean; 676 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-10-16T14:35:10.213 INFO:tasks.workunit.client.0.smithi178.stdout:checking for unistd.h... yes 2023-10-16T14:35:10.237 INFO:tasks.workunit.client.0.smithi178.stdout:checking for memory.h... (cached) yes 2023-10-16T14:35:10.253 INFO:tasks.workunit.client.0.smithi178.stdout:checking for strings.h... (cached) yes 2023-10-16T14:35:10.269 INFO:tasks.workunit.client.0.smithi178.stdout:checking for unistd.h... (cached) yes 2023-10-16T14:35:10.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:09 smithi178 ceph-mon[98495]: pgmap v382: 129 pgs: 129 active+clean; 676 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-10-16T14:35:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:09 smithi186 ceph-mon[134892]: pgmap v382: 129 pgs: 129 active+clean; 676 KiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 938 B/s wr, 7 op/s 2023-10-16T14:35:10.357 INFO:tasks.workunit.client.0.smithi178.stdout:checking locale.h usability... yes 2023-10-16T14:35:10.391 INFO:tasks.workunit.client.0.smithi178.stdout:checking locale.h presence... yes 2023-10-16T14:35:10.393 INFO:tasks.workunit.client.0.smithi178.stdout:checking for locale.h... yes 2023-10-16T14:35:10.420 INFO:tasks.workunit.client.0.smithi178.stdout:checking for stdlib.h... (cached) yes 2023-10-16T14:35:10.436 INFO:tasks.workunit.client.0.smithi178.stdout:checking for string.h... (cached) yes 2023-10-16T14:35:10.527 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/socket.h usability... yes 2023-10-16T14:35:10.564 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/socket.h presence... yes 2023-10-16T14:35:10.566 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/socket.h... yes 2023-10-16T14:35:10.663 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/time.h usability... yes 2023-10-16T14:35:10.698 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/time.h presence... yes 2023-10-16T14:35:10.700 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/time.h... yes 2023-10-16T14:35:10.796 INFO:tasks.workunit.client.0.smithi178.stdout:checking fcntl.h usability... yes 2023-10-16T14:35:10.832 INFO:tasks.workunit.client.0.smithi178.stdout:checking fcntl.h presence... yes 2023-10-16T14:35:10.834 INFO:tasks.workunit.client.0.smithi178.stdout:checking for fcntl.h... yes 2023-10-16T14:35:10.931 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/fcntl.h usability... yes 2023-10-16T14:35:10.968 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/fcntl.h presence... yes 2023-10-16T14:35:10.970 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/fcntl.h... yes 2023-10-16T14:35:11.092 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/param.h usability... yes 2023-10-16T14:35:11.133 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/param.h presence... yes 2023-10-16T14:35:11.135 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/param.h... yes 2023-10-16T14:35:11.214 INFO:tasks.workunit.client.0.smithi178.stdout:checking ioctl.h usability... no 2023-10-16T14:35:11.251 INFO:tasks.workunit.client.0.smithi178.stdout:checking ioctl.h presence... no 2023-10-16T14:35:11.253 INFO:tasks.workunit.client.0.smithi178.stdout:checking for ioctl.h... no 2023-10-16T14:35:11.343 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/ioctl.h usability... yes 2023-10-16T14:35:11.378 INFO:tasks.workunit.client.0.smithi178.stdout:checking sys/ioctl.h presence... yes 2023-10-16T14:35:11.380 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sys/ioctl.h... yes 2023-10-16T14:35:11.470 INFO:tasks.workunit.client.0.smithi178.stdout:checking return type of signal handlers... void 2023-10-16T14:35:11.552 INFO:tasks.workunit.client.0.smithi178.stdout:checking for size_t... yes 2023-10-16T14:35:11.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:11 smithi178 ceph-mon[98495]: pgmap v383: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-10-16T14:35:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:11 smithi186 ceph-mon[134892]: pgmap v383: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-10-16T14:35:11.622 INFO:tasks.workunit.client.0.smithi178.stdout:checking for dev_t... yes 2023-10-16T14:35:11.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:11 smithi187 ceph-mon[135297]: pgmap v383: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 8 op/s 2023-10-16T14:35:11.744 INFO:tasks.workunit.client.0.smithi178.stdout:checking for short... yes 2023-10-16T14:35:11.892 INFO:tasks.workunit.client.0.smithi178.stdout:checking size of short... 2 2023-10-16T14:35:11.977 INFO:tasks.workunit.client.0.smithi178.stdout:checking for int... yes 2023-10-16T14:35:12.088 INFO:tasks.workunit.client.0.smithi178.stdout:checking size of int... 4 2023-10-16T14:35:12.174 INFO:tasks.workunit.client.0.smithi178.stdout:checking for long... yes 2023-10-16T14:35:12.282 INFO:tasks.workunit.client.0.smithi178.stdout:checking size of long... 8 2023-10-16T14:35:12.369 INFO:tasks.workunit.client.0.smithi178.stdout:checking for long long... yes 2023-10-16T14:35:12.474 INFO:tasks.workunit.client.0.smithi178.stdout:checking size of long long... 8 2023-10-16T14:35:12.570 INFO:tasks.workunit.client.0.smithi178.stdout:checking for ssize_t... yes 2023-10-16T14:35:12.666 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sig_atomic_t... yes 2023-10-16T14:35:12.742 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether gcc needs -traditional... no 2023-10-16T14:35:12.809 INFO:tasks.workunit.client.0.smithi178.stdout:checking for an ANSI C-conforming const... yes 2023-10-16T14:35:12.862 INFO:tasks.workunit.client.0.smithi178.stdout:checking for inline... inline 2023-10-16T14:35:12.959 INFO:tasks.workunit.client.0.smithi178.stdout:checking for working alloca.h... yes 2023-10-16T14:35:13.048 INFO:tasks.workunit.client.0.smithi178.stdout:checking for alloca... yes 2023-10-16T14:35:13.160 INFO:tasks.workunit.client.0.smithi178.stdout:checking for working memcmp... yes 2023-10-16T14:35:13.260 INFO:tasks.workunit.client.0.smithi178.stdout:checking for strftime... yes 2023-10-16T14:35:13.381 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2023-10-16T14:35:13.482 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether stat accepts an empty string... no 2023-10-16T14:35:13.582 INFO:tasks.workunit.client.0.smithi178.stdout:checking for vprintf... yes 2023-10-16T14:35:13.683 INFO:tasks.workunit.client.0.smithi178.stdout:checking for _doprnt... no 2023-10-16T14:35:13.740 INFO:tasks.workunit.client.0.smithi178.stdout:checking build system type... x86_64-unknown-linux-gnu 2023-10-16T14:35:13.774 INFO:tasks.workunit.client.0.smithi178.stdout:checking host system type... x86_64-unknown-linux-gnu 2023-10-16T14:35:13.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:13 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:13 smithi178 ceph-mon[98495]: pgmap v384: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 7 op/s 2023-10-16T14:35:13.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:13 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:13.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:13 smithi186 ceph-mon[134892]: pgmap v384: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 7 op/s 2023-10-16T14:35:13.889 INFO:tasks.workunit.client.0.smithi178.stdout:checking for the pthreads library -lpthreads... no 2023-10-16T14:35:13.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:13 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:13.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:13 smithi187 ceph-mon[135297]: pgmap v384: 129 pgs: 129 active+clean; 1.5 MiB data, 331 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 71 KiB/s wr, 7 op/s 2023-10-16T14:35:13.980 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether pthreads work without any flags... no 2023-10-16T14:35:14.012 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether pthreads work with -Kthread... no 2023-10-16T14:35:14.045 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether pthreads work with -kthread... no 2023-10-16T14:35:14.121 INFO:tasks.workunit.client.0.smithi178.stdout:checking for the pthreads library -llthread... no 2023-10-16T14:35:14.215 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether pthreads work with -pthread... yes 2023-10-16T14:35:14.301 INFO:tasks.workunit.client.0.smithi178.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2023-10-16T14:35:14.301 INFO:tasks.workunit.client.0.smithi178.stdout:checking if more special flags are required for pthreads... no 2023-10-16T14:35:14.301 INFO:tasks.workunit.client.0.smithi178.stdout:checking for cc_r... gcc 2023-10-16T14:35:14.432 INFO:tasks.workunit.client.0.smithi178.stdout:checking for setrlimit... yes 2023-10-16T14:35:14.540 INFO:tasks.workunit.client.0.smithi178.stdout:checking for setproctitle... no 2023-10-16T14:35:14.645 INFO:tasks.workunit.client.0.smithi178.stdout:checking for getopt_long... yes 2023-10-16T14:35:14.757 INFO:tasks.workunit.client.0.smithi178.stdout:checking for strtoull... yes 2023-10-16T14:35:14.868 INFO:tasks.workunit.client.0.smithi178.stdout:checking for strtoq... yes 2023-10-16T14:35:14.981 INFO:tasks.workunit.client.0.smithi178.stdout:checking for snprintf... yes 2023-10-16T14:35:15.093 INFO:tasks.workunit.client.0.smithi178.stdout:checking for vsnprintf... yes 2023-10-16T14:35:15.205 INFO:tasks.workunit.client.0.smithi178.stdout:checking for vfprintf... yes 2023-10-16T14:35:15.316 INFO:tasks.workunit.client.0.smithi178.stdout:checking for gethostname... yes 2023-10-16T14:35:15.428 INFO:tasks.workunit.client.0.smithi178.stdout:checking for setlocale... yes 2023-10-16T14:35:15.542 INFO:tasks.workunit.client.0.smithi178.stdout:checking for sync... yes 2023-10-16T14:35:15.653 INFO:tasks.workunit.client.0.smithi178.stdout:checking whether snprintf is C99 conformant... done 2023-10-16T14:35:15.678 INFO:tasks.workunit.client.0.smithi178.stdout:configure: creating ./config.status 2023-10-16T14:35:15.988 INFO:tasks.workunit.client.0.smithi178.stdout:config.status: creating Makefile 2023-10-16T14:35:16.036 INFO:tasks.workunit.client.0.smithi178.stdout:config.status: creating src/Makefile 2023-10-16T14:35:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:15 smithi178 ceph-mon[98495]: pgmap v385: 129 pgs: 129 active+clean; 1.5 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-10-16T14:35:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:15 smithi186 ceph-mon[134892]: pgmap v385: 129 pgs: 129 active+clean; 1.5 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-10-16T14:35:16.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:15 smithi187 ceph-mon[135297]: pgmap v385: 129 pgs: 129 active+clean; 1.5 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 86 KiB/s wr, 16 op/s 2023-10-16T14:35:18.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:17 smithi178 ceph-mon[98495]: pgmap v386: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 297 KiB/s wr, 35 op/s 2023-10-16T14:35:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:17 smithi186 ceph-mon[134892]: pgmap v386: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 297 KiB/s wr, 35 op/s 2023-10-16T14:35:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:17 smithi187 ceph-mon[135297]: pgmap v386: 129 pgs: 129 active+clean; 3.8 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 297 KiB/s wr, 35 op/s 2023-10-16T14:35:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:19 smithi178 ceph-mon[98495]: pgmap v387: 129 pgs: 129 active+clean; 3.8 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 298 KiB/s wr, 35 op/s 2023-10-16T14:35:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:19 smithi186 ceph-mon[134892]: pgmap v387: 129 pgs: 129 active+clean; 3.8 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 298 KiB/s wr, 35 op/s 2023-10-16T14:35:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:19 smithi187 ceph-mon[135297]: pgmap v387: 129 pgs: 129 active+clean; 3.8 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 298 KiB/s wr, 35 op/s 2023-10-16T14:35:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:21 smithi178 ceph-mon[98495]: pgmap v388: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 317 KiB/s wr, 36 op/s 2023-10-16T14:35:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:21 smithi186 ceph-mon[134892]: pgmap v388: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 317 KiB/s wr, 36 op/s 2023-10-16T14:35:21.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:21 smithi187 ceph-mon[135297]: pgmap v388: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 317 KiB/s wr, 36 op/s 2023-10-16T14:35:22.001 INFO:tasks.workunit.client.0.smithi178.stdout:config.status: creating man/Makefile 2023-10-16T14:35:24.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:23 smithi178 ceph-mon[98495]: pgmap v389: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 247 KiB/s wr, 34 op/s 2023-10-16T14:35:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:23 smithi186 ceph-mon[134892]: pgmap v389: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 247 KiB/s wr, 34 op/s 2023-10-16T14:35:24.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:23 smithi187 ceph-mon[135297]: pgmap v389: 129 pgs: 129 active+clean; 4.0 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 247 KiB/s wr, 34 op/s 2023-10-16T14:35:25.712 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:35:25.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 -- 172.21.15.178:0/138994442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102a40 msgr2=0x7f5b54109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 --2- 172.21.15.178:0/138994442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102a40 0x7f5b54109360 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009f90 tx=0x7f5b4c00b330 comp rx=0 tx=0).stop 2023-10-16T14:35:25.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 -- 172.21.15.178:0/138994442 shutdown_connections 2023-10-16T14:35:25.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 --2- 172.21.15.178:0/138994442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102a40 0x7f5b54109360 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 --2- 172.21.15.178:0/138994442 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b541012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 --2- 172.21.15.178:0/138994442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102080 0x7f5b54102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 -- 172.21.15.178:0/138994442 >> 172.21.15.178:0/138994442 conn(0x7f5b540fc420 msgr2=0x7f5b540fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 -- 172.21.15.178:0/138994442 shutdown_connections 2023-10-16T14:35:25.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.919+0000 7f5b5adff700 1 -- 172.21.15.178:0/138994442 wait complete. 2023-10-16T14:35:25.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.920+0000 7f5b5adff700 1 Processor -- start 2023-10-16T14:35:25.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.920+0000 7f5b5adff700 1 -- start start 2023-10-16T14:35:25.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.920+0000 7f5b5adff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b54198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.920+0000 7f5b5adff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5adff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 0x7f5b541a04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5adff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a0a90 con 0x7f5b54102080 2023-10-16T14:35:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5adff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a0bd0 con 0x7f5b54100ea0 2023-10-16T14:35:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5adff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b541a0d60 con 0x7f5b54102a40 2023-10-16T14:35:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b54198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39274/0 (socket says 172.21.15.178:39274) 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 -- 172.21.15.178:0/757422322 learned_addr learned my addr 172.21.15.178:0/757422322 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5939c700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 0x7f5b541a04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 -- 172.21.15.178:0/757422322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 msgr2=0x7f5b541a04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 0x7f5b541a04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 -- 172.21.15.178:0/757422322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 msgr2=0x7f5b54198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b54198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 -- 172.21.15.178:0/757422322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480095f0 con 0x7f5b54102080 2023-10-16T14:35:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b54198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b5939c700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 0x7f5b541a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.921+0000 7f5b53fff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e970 tx=0x7f5b4000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.922+0000 7f5b51ffb700 1 -- 172.21.15.178:0/757422322 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40009790 con 0x7f5b54102080 2023-10-16T14:35:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.922+0000 7f5b51ffb700 1 -- 172.21.15.178:0/757422322 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4000c7e0 con 0x7f5b54102080 2023-10-16T14:35:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.922+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c009c40 con 0x7f5b54102080 2023-10-16T14:35:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.922+0000 7f5b51ffb700 1 -- 172.21.15.178:0/757422322 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40003c70 con 0x7f5b54102080 2023-10-16T14:35:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.922+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a1380 con 0x7f5b54102080 2023-10-16T14:35:25.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.925+0000 7f5b51ffb700 1 -- 172.21.15.178:0/757422322 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5b4000cc50 con 0x7f5b54102080 2023-10-16T14:35:25.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.926+0000 7f5b6e6adb80 1 Processor -- start 2023-10-16T14:35:25.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.926+0000 7f5b51ffb700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 0x7f5b440a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.926+0000 7f5b6e6adb80 1 -- start start 2023-10-16T14:35:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.926+0000 7f5b51ffb700 1 -- 172.21.15.178:0/757422322 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5b40014070 con 0x7f5b54102080 2023-10-16T14:35:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.926+0000 7f5b5939c700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 0x7f5b440a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c255c70 con 0x56361c2e1780 2023-10-16T14:35:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c23ac60 con 0x56361c2e0e20 2023-10-16T14:35:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c238580 con 0x56361c2e2620 2023-10-16T14:35:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55460/0 (socket says 172.21.15.178:55460) 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43590/0 (socket says 172.21.15.178:43590) 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 -- 172.21.15.178:0/1010704318 learned_addr learned my addr 172.21.15.178:0/1010704318 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b5939c700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 0x7f5b440a39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c007900 tx=0x7f5b4c0096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 -- 172.21.15.178:0/1010704318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 msgr2=0x56361c2e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 -- 172.21.15.178:0/1010704318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 msgr2=0x56361c2e2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2e2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b58b9b700 1 -- 172.21.15.178:0/1010704318 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361c1f13f0 con 0x56361c2e0e20 2023-10-16T14:35:25.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b5939c700 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:25.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.927+0000 7f5b53fff700 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:25.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.928+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011810 tx=0x7f5b48011b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd77dedc5be52c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.928+0000 7f5b3effd700 1 -- 172.21.15.178:0/1010704318 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004830 con 0x56361c2e0e20 2023-10-16T14:35:25.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.928+0000 7f5b3effd700 1 -- 172.21.15.178:0/1010704318 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b48004e50 con 0x56361c2e0e20 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.928+0000 7f5b3effd700 1 -- 172.21.15.178:0/1010704318 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800f860 con 0x56361c2e0e20 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/1010704318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 msgr2=0x56361c2e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011810 tx=0x7f5b48011b20 comp rx=0 tx=0).stop 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/1010704318 shutdown_connections 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e2620 0x56361c2e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e0e20 0x56361c2e1240 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/1010704318 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/1010704318 >> 172.21.15.178:0/1010704318 conn(0x56361c20edd0 msgr2=0x56361c2dbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/1010704318 shutdown_connections 2023-10-16T14:35:25.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/1010704318 wait complete. 2023-10-16T14:35:25.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.929+0000 7f5b6e6adb80 1 -- start start 2023-10-16T14:35:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 Processor -- start 2023-10-16T14:35:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2fa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c233e70 con 0x56361c2e0e20 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c1d0710 con 0x56361c2e2620 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.930+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c1d03b0 con 0x56361c2e1780 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b6e6adb80 10 client.0 _ll_register_callbacks cb 0x56361c02d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43606/0 (socket says 172.21.15.178:43606) 2023-10-16T14:35:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 -- 172.21.15.178:0/2739923077 learned_addr learned my addr 172.21.15.178:0/2739923077 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b5939c700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2fa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 -- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 msgr2=0x56361c2fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2fa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 -- 172.21.15.178:0/2739923077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 msgr2=0x56361c2f3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 -- 172.21.15.178:0/2739923077 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b54109c90 con 0x56361c2e1780 2023-10-16T14:35:25.946 INFO:tasks.workunit.client.0.smithi178.stdout:config.status: creating config.h 2023-10-16T14:35:25.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b53fff700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000efd0 tx=0x7f5b40099b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f25a8ed26546cb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.931+0000 7f5b33fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:35:25.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4009a5d0 con 0x56361c2e1780 2023-10-16T14:35:25.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361c238ef0 con 0x56361c2e1780 2023-10-16T14:35:25.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5b4009a170 con 0x56361c2e1780 2023-10-16T14:35:25.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4009e2c0 con 0x56361c2e1780 2023-10-16T14:35:25.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 10 client.25684 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:25.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5b48011440 con 0x56361c2e1780 2023-10-16T14:35:25.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56361c238ef0 con 0x56361c2e1780 2023-10-16T14:35:25.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5b40099660 con 0x56361c2e1780 2023-10-16T14:35:25.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 10 client.25684 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 20 client.25684 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c2ff270 0x56361c301690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 4 client.25684 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.932+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361c2b0c10 con 0x56361c2ff270 2023-10-16T14:35:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.933+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c2ff270 0x56361c301690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.933+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c2ff270 0x56361c301690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385c0fee66c4638d server_cookie=36d11f895b9735c6 in_seq=0 out_seq=0 2023-10-16T14:35:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.933+0000 7f5b33fff700 10 client.25684 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b33fff700 1 -- 172.21.15.178:0/2739923077 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56361c2b0c10 con 0x56361c2ff270 2023-10-16T14:35:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b33fff700 10 client.25684 handle_command_reply: tid=0 2023-10-16T14:35:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 client.25684 shutdown 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 msgr2=0x56361c2f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000efd0 tx=0x7f5b40099b40 comp rx=0 tx=0).stop 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 shutdown_connections 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c2ff270 0x56361c301690 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2fa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b33fff700 0 client.25684 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:25.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2739923077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2f3960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 >> 172.21.15.178:0/2739923077 conn(0x56361c20edd0 msgr2=0x56361c2dbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 shutdown_connections 2023-10-16T14:35:25.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2739923077 wait complete. 2023-10-16T14:35:25.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.934+0000 7f5b6e6adb80 20 client.25684 trim_cache size 0 max 16384 2023-10-16T14:35:25.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.953+0000 7f5b6e6adb80 1 Processor -- start 2023-10-16T14:35:25.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.953+0000 7f5b6e6adb80 1 -- start start 2023-10-16T14:35:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c0b4920 con 0x56361c2e0e20 2023-10-16T14:35:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c238ef0 con 0x56361c2e2620 2023-10-16T14:35:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c2e24a0 con 0x56361c2e1780 2023-10-16T14:35:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43608/0 (socket says 172.21.15.178:43608) 2023-10-16T14:35:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55484/0 (socket says 172.21.15.178:55484) 2023-10-16T14:35:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b53fff700 1 -- 172.21.15.178:0/2832376246 learned_addr learned my addr 172.21.15.178:0/2832376246 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 -- 172.21.15.178:0/2832376246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 msgr2=0x56361c2fa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 -- 172.21.15.178:0/2832376246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 msgr2=0x56361c2f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.954+0000 7f5b5939c700 1 -- 172.21.15.178:0/2832376246 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361c05ede0 con 0x56361c2e2620 2023-10-16T14:35:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.955+0000 7f5b53fff700 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.955+0000 7f5b5939c700 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009790 tx=0x7f5b4c026000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e8f002382699c5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.955+0000 7f5b3f7fe700 1 -- 172.21.15.178:0/2832376246 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c022040 con 0x56361c2e2620 2023-10-16T14:35:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.955+0000 7f5b3f7fe700 1 -- 172.21.15.178:0/2832376246 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b4c003e90 con 0x56361c2e2620 2023-10-16T14:35:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.955+0000 7f5b3f7fe700 1 -- 172.21.15.178:0/2832376246 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00e040 con 0x56361c2e2620 2023-10-16T14:35:25.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2832376246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 msgr2=0x56361c2f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009790 tx=0x7f5b4c026000 comp rx=0 tx=0).stop 2023-10-16T14:35:25.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2832376246 shutdown_connections 2023-10-16T14:35:25.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f3710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e0e20 0x56361c2f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/2832376246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e1780 0x56361c2fa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2832376246 >> 172.21.15.178:0/2832376246 conn(0x56361c20edd0 msgr2=0x56361c2fb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2832376246 shutdown_connections 2023-10-16T14:35:25.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/2832376246 wait complete. 2023-10-16T14:35:25.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.956+0000 7f5b6e6adb80 1 -- start start 2023-10-16T14:35:25.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 Processor -- start 2023-10-16T14:35:25.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e0e20 0x56361c2fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2f8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c2f9780 con 0x56361c2e1780 2023-10-16T14:35:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c2f98f0 con 0x56361c2e2620 2023-10-16T14:35:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361c2f9ab0 con 0x56361c2e0e20 2023-10-16T14:35:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b6e6adb80 10 client.0 _ll_register_callbacks cb 0x56361c02d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b5939c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55494/0 (socket says 172.21.15.178:55494) 2023-10-16T14:35:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b53fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2f8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b58b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e0e20 0x56361c2fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.957+0000 7f5b5939c700 1 -- 172.21.15.178:0/4110213707 learned_addr learned my addr 172.21.15.178:0/4110213707 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 -- 172.21.15.178:0/4110213707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e0e20 msgr2=0x56361c2fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e0e20 0x56361c2fa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 -- 172.21.15.178:0/4110213707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 msgr2=0x56361c2f8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2f8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 -- 172.21.15.178:0/4110213707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b48011440 con 0x56361c2e2620 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b5939c700 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005010 tx=0x7f5b4c0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56d64a0ac20e4f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b33fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:35:25.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4c00e040 con 0x56361c2e2620 2023-10-16T14:35:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b48004e50 con 0x56361c2e2620 2023-10-16T14:35:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5b4c022040 con 0x56361c2e2620 2023-10-16T14:35:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.958+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b4c00be00 con 0x56361c2e2620 2023-10-16T14:35:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 10 client.25700 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:25.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f5b4c007650 con 0x56361c2e2620 2023-10-16T14:35:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b48004e50 con 0x56361c2e2620 2023-10-16T14:35:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5b4c0284a0 con 0x56361c2e2620 2023-10-16T14:35:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 10 client.25700 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 10 client.25700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 20 client.25700 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c354600 0x56361c356a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 4 client.25700 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:35:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56361c22ce50 con 0x56361c354600 2023-10-16T14:35:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.959+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c354600 0x56361c356a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b58b9b700 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c354600 0x56361c356a20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd1db4b8de3c3c2 server_cookie=1de1cb0eae2dbfa4 in_seq=0 out_seq=0 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b33fff700 10 client.25700 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b33fff700 1 -- 172.21.15.178:0/4110213707 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56361c22ce50 con 0x56361c354600 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b33fff700 10 client.25700 handle_command_reply: tid=0 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 client.25700 shutdown 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 msgr2=0x56361c2f91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c005010 tx=0x7f5b4c0058d0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 shutdown_connections 2023-10-16T14:35:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56361c354600 0x56361c356a20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56361c2e2620 0x56361c2f91f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b33fff700 0 client.25700 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56361c2e0e20 0x56361c2fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 --2- 172.21.15.178:0/4110213707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56361c2e1780 0x56361c2f8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.960+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 >> 172.21.15.178:0/4110213707 conn(0x56361c20edd0 msgr2=0x56361c2f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 shutdown_connections 2023-10-16T14:35:25.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b6e6adb80 1 -- 172.21.15.178:0/4110213707 wait complete. 2023-10-16T14:35:25.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b6e6adb80 20 client.25700 trim_cache size 0 max 16384 2023-10-16T14:35:25.979 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:35:25.979 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:35:25.979 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:35:25.979 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:35:25.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 msgr2=0x7f5b440a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 0x7f5b440a39c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c007900 tx=0x7f5b4c0096c0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 msgr2=0x7f5b54199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.961+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e970 tx=0x7f5b4000ec80 comp rx=0 tx=0).stop 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 shutdown_connections 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5b440a1500 0x7f5b440a39c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b54102a40 0x7f5b541a04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5b54100ea0 0x7f5b54198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 --2- 172.21.15.178:0/757422322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b54102080 0x7f5b54199450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:25.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 >> 172.21.15.178:0/757422322 conn(0x7f5b540fc420 msgr2=0x7f5b540fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:25.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 shutdown_connections 2023-10-16T14:35:25.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:25.962+0000 7f5b5adff700 1 -- 172.21.15.178:0/757422322 wait complete. 2023-10-16T14:35:25.983 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:25 smithi178 ceph-mon[98495]: pgmap v390: 129 pgs: 129 active+clean; 4.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 263 KiB/s wr, 40 op/s 2023-10-16T14:35:25.984 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:35:25.984 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:35:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:25 smithi186 ceph-mon[134892]: pgmap v390: 129 pgs: 129 active+clean; 4.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 263 KiB/s wr, 40 op/s 2023-10-16T14:35:26.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:25 smithi187 ceph-mon[135297]: pgmap v390: 129 pgs: 129 active+clean; 4.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 263 KiB/s wr, 40 op/s 2023-10-16T14:35:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 -- 172.21.15.178:0/3500577669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102090 msgr2=0x7ff760102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 --2- 172.21.15.178:0/3500577669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102090 0x7ff760102510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff748009990 tx=0x7ff748009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 -- 172.21.15.178:0/3500577669 shutdown_connections 2023-10-16T14:35:26.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 --2- 172.21.15.178:0/3500577669 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760102a50 0x7ff760109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 --2- 172.21.15.178:0/3500577669 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760100eb0 0x7ff7601012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 --2- 172.21.15.178:0/3500577669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102090 0x7ff760102510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 -- 172.21.15.178:0/3500577669 >> 172.21.15.178:0/3500577669 conn(0x7ff7600fc430 msgr2=0x7ff7600fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 -- 172.21.15.178:0/3500577669 shutdown_connections 2023-10-16T14:35:26.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.181+0000 7ff7669ff700 1 -- 172.21.15.178:0/3500577669 wait complete. 2023-10-16T14:35:26.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.182+0000 7ff7669ff700 1 Processor -- start 2023-10-16T14:35:26.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.182+0000 7ff7669ff700 1 -- start start 2023-10-16T14:35:26.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760102090 0x7ff7600732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102a50 0x7ff7600737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760073f80 con 0x7ff760100eb0 2023-10-16T14:35:26.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600740f0 con 0x7ff760102090 2023-10-16T14:35:26.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff7669ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601a0be0 con 0x7ff760102a50 2023-10-16T14:35:26.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39336/0 (socket says 172.21.15.178:39336) 2023-10-16T14:35:26.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 -- 172.21.15.178:0/400865227 learned_addr learned my addr 172.21.15.178:0/400865227 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff75ffff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760102090 0x7ff7600732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff764f9c700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102a50 0x7ff7600737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 -- 172.21.15.178:0/400865227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102a50 msgr2=0x7ff7600737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102a50 0x7ff7600737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 -- 172.21.15.178:0/400865227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760102090 msgr2=0x7ff7600732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760102090 0x7ff7600732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.183+0000 7ff757fff700 1 -- 172.21.15.178:0/400865227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480095f0 con 0x7ff760100eb0 2023-10-16T14:35:26.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff757fff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e970 tx=0x7ff75000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750009790 con 0x7ff760100eb0 2023-10-16T14:35:26.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff75000c7e0 con 0x7ff760100eb0 2023-10-16T14:35:26.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7601a0ec0 con 0x7ff760100eb0 2023-10-16T14:35:26.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750003c70 con 0x7ff760100eb0 2023-10-16T14:35:26.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.184+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7601a12e0 con 0x7ff760100eb0 2023-10-16T14:35:26.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.185+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76010c7e0 con 0x7ff760100eb0 2023-10-16T14:35:26.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.187+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff75000cc50 con 0x7ff760100eb0 2023-10-16T14:35:26.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.189+0000 7ff75dffb700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 0x7ff74c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.189+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff750014070 con 0x7ff760100eb0 2023-10-16T14:35:26.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.189+0000 7ff75ffff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 0x7ff74c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.190+0000 7ff75ffff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 0x7ff74c0a39e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b2e0 tx=0x7ff748005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.192+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff750096640 con 0x7ff760100eb0 2023-10-16T14:35:26.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.439+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff760107d90 con 0x7ff760100eb0 2023-10-16T14:35:26.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.440+0000 7ff75dffb700 1 -- 172.21.15.178:0/400865227 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+11341 (secure 0 0 0) 0x7ff750003dd0 con 0x7ff760100eb0 2023-10-16T14:35:26.444 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:35:26.444 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:35:12.443822+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:35:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 msgr2=0x7ff74c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 0x7ff74c0a39e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff74800b2e0 tx=0x7ff748005fb0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 msgr2=0x7ff760074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff75000e970 tx=0x7ff75000ec80 comp rx=0 tx=0).stop 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 shutdown_connections 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff74c0a1520 0x7ff74c0a39e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff760102a50 0x7ff7600737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff760100eb0 0x7ff760074c60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 --2- 172.21.15.178:0/400865227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff760102090 0x7ff7600732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 >> 172.21.15.178:0/400865227 conn(0x7ff7600fc430 msgr2=0x7ff7600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 shutdown_connections 2023-10-16T14:35:26.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.442+0000 7ff7669ff700 1 -- 172.21.15.178:0/400865227 wait complete. 2023-10-16T14:35:26.448 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 28 2023-10-16T14:35:26.465 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:35:26.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 -- 172.21.15.178:0/2914980832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102a40 msgr2=0x7f4cac109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/2914980832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102a40 0x7f4cac109360 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b0a0 tx=0x7f4ca400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 -- 172.21.15.178:0/2914980832 shutdown_connections 2023-10-16T14:35:26.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/2914980832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102a40 0x7f4cac109360 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/2914980832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 0x7f4cac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/2914980832 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102080 0x7f4cac102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 -- 172.21.15.178:0/2914980832 >> 172.21.15.178:0/2914980832 conn(0x7f4cac0fc420 msgr2=0x7f4cac0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 -- 172.21.15.178:0/2914980832 shutdown_connections 2023-10-16T14:35:26.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.663+0000 7f4cb2dff700 1 -- 172.21.15.178:0/2914980832 wait complete. 2023-10-16T14:35:26.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 Processor -- start 2023-10-16T14:35:26.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 -- start start 2023-10-16T14:35:26.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 0x7f4cac196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 0x7f4cac19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac19e880 con 0x7f4cac102080 2023-10-16T14:35:26.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac19e9f0 con 0x7f4cac102a40 2023-10-16T14:35:26.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cb2dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cac19ebb0 con 0x7f4cac100ea0 2023-10-16T14:35:26.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.664+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39354/0 (socket says 172.21.15.178:39354) 2023-10-16T14:35:26.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 -- 172.21.15.178:0/1430268933 learned_addr learned my addr 172.21.15.178:0/1430268933 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 0x7f4cac196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cb139c700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 0x7f4cac19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 -- 172.21.15.178:0/1430268933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 msgr2=0x7f4cac196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 0x7f4cac196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 -- 172.21.15.178:0/1430268933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 msgr2=0x7f4cac19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 0x7f4cac19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 -- 172.21.15.178:0/1430268933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009520 con 0x7f4cac102080 2023-10-16T14:35:26.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cb139c700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 0x7f4cac19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:26.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cabfff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800ea50 tx=0x7f4c9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4ca9ffb700 1 -- 172.21.15.178:0/1430268933 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9800cbd0 con 0x7f4cac102080 2023-10-16T14:35:26.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4ca9ffb700 1 -- 172.21.15.178:0/1430268933 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c9800cd30 con 0x7f4cac102080 2023-10-16T14:35:26.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca4009d00 con 0x7f4cac102080 2023-10-16T14:35:26.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.665+0000 7f4ca9ffb700 1 -- 172.21.15.178:0/1430268933 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980186c0 con 0x7f4cac102080 2023-10-16T14:35:26.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.666+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac19f1b0 con 0x7f4cac102080 2023-10-16T14:35:26.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.668+0000 7f4ca9ffb700 1 -- 172.21.15.178:0/1430268933 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4c98018820 con 0x7f4cac102080 2023-10-16T14:35:26.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 Processor -- start 2023-10-16T14:35:26.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 -- start start 2023-10-16T14:35:26.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4ca9ffb700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 0x7f4c9c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4ca9ffb700 1 -- 172.21.15.178:0/1430268933 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4c98014070 con 0x7f4cac102080 2023-10-16T14:35:26.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bb2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.695 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03bb2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03b21c70 con 0x558a03bb1230 2023-10-16T14:35:26.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03b06c60 con 0x558a03bb1b90 2023-10-16T14:35:26.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03b04580 con 0x558a03bb2a30 2023-10-16T14:35:26.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.670+0000 7f4cb139c700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 0x7f4c9c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39360/0 (socket says 172.21.15.178:39360) 2023-10-16T14:35:26.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2006662356 learned_addr learned my addr 172.21.15.178:0/2006662356 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bb2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb139c700 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03bb2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2006662356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 msgr2=0x558a03bb2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03bb2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2006662356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 msgr2=0x558a03bb2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bb2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2006662356 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a03abd3f0 con 0x558a03bb1230 2023-10-16T14:35:26.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cabfff700 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bb2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb139c700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 0x7f4c9c0a39c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40062a0 tx=0x7f4ca40077a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009cf0 tx=0x7f4ca0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85376506dab7db91 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4c96ffd700 1 -- 172.21.15.178:0/2006662356 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011d80 con 0x558a03bb1230 2023-10-16T14:35:26.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4c96ffd700 1 -- 172.21.15.178:0/2006662356 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca00044e0 con 0x558a03bb1230 2023-10-16T14:35:26.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.671+0000 7f4c96ffd700 1 -- 172.21.15.178:0/2006662356 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0005240 con 0x558a03bb1230 2023-10-16T14:35:26.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2006662356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 msgr2=0x558a03bb1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009cf0 tx=0x7f4ca0011440 comp rx=0 tx=0).stop 2023-10-16T14:35:26.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2006662356 shutdown_connections 2023-10-16T14:35:26.700 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03bb2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1650 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2006662356 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bb2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2006662356 >> 172.21.15.178:0/2006662356 conn(0x558a03adadd0 msgr2=0x558a03ba7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2006662356 shutdown_connections 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.672+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2006662356 wait complete. 2023-10-16T14:35:26.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.673+0000 7f4cc67e3b80 1 -- start start 2023-10-16T14:35:26.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.673+0000 7f4cc67e3b80 1 Processor -- start 2023-10-16T14:35:26.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.673+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03affe70 con 0x558a03bb1b90 2023-10-16T14:35:26.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03a9c710 con 0x558a03bb2a30 2023-10-16T14:35:26.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03a9c3b0 con 0x558a03bb1230 2023-10-16T14:35:26.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cc67e3b80 10 client.0 _ll_register_callbacks cb 0x558a038f9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:26.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43670/0 (socket says 172.21.15.178:43670) 2023-10-16T14:35:26.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cabfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39370/0 (socket says 172.21.15.178:39370) 2023-10-16T14:35:26.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/3239248307 learned_addr learned my addr 172.21.15.178:0/3239248307 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb139c700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 msgr2=0x558a03bc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/3239248307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 msgr2=0x558a03bbb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/3239248307 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cac19f1b0 con 0x558a03bb1230 2023-10-16T14:35:26.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.674+0000 7f4cabfff700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009310 tx=0x7f4ca0011db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b8b434ed9b8b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4c8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:35:26.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0005200 con 0x558a03bb1230 2023-10-16T14:35:26.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a03b04ef0 con 0x558a03bb1230 2023-10-16T14:35:26.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4ca0005360 con 0x558a03bb1230 2023-10-16T14:35:26.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ca0005520 con 0x558a03bb1230 2023-10-16T14:35:26.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4cc67e3b80 10 client.25708 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:26.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.675+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f4ca0009920 con 0x558a03bb1230 2023-10-16T14:35:26.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a03b04ef0 con 0x558a03bb1230 2023-10-16T14:35:26.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4ca0019430 con 0x558a03bb1230 2023-10-16T14:35:26.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 10 client.25708 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:26.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 10 client.25708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:35:26.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 20 client.25708 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:26.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03bc6e40 0x558a03bc9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 4 client.25708 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:26.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a03b7cc10 con 0x558a03bc6e40 2023-10-16T14:35:26.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.676+0000 7f4cabfff700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03bc6e40 0x558a03bc9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.677+0000 7f4cabfff700 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03bc6e40 0x558a03bc9260 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d934decc574e7be7 server_cookie=bf3fbaa664f1f8fe in_seq=0 out_seq=0 2023-10-16T14:35:26.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.677+0000 7f4c8bfff700 10 client.25708 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:35:26.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.677+0000 7f4c8bfff700 1 -- 172.21.15.178:0/3239248307 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558a03b7cc10 con 0x558a03bc6e40 2023-10-16T14:35:26.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.677+0000 7f4c8bfff700 10 client.25708 handle_command_reply: tid=0 2023-10-16T14:35:26.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.677+0000 7f4cc67e3b80 1 client.25708 shutdown 2023-10-16T14:35:26.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 msgr2=0x558a03bbb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009310 tx=0x7f4ca0011db0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 shutdown_connections 2023-10-16T14:35:26.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03bc6e40 0x558a03bc9260 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1230 0x558a03bbb040 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/3239248307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1b90 0x558a03bbb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4c8bfff700 0 client.25708 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:35:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 >> 172.21.15.178:0/3239248307 conn(0x558a03adadd0 msgr2=0x558a03ba7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 shutdown_connections 2023-10-16T14:35:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/3239248307 wait complete. 2023-10-16T14:35:26.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.678+0000 7f4cc67e3b80 20 client.25708 trim_cache size 0 max 16384 2023-10-16T14:35:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.691+0000 7f4cc67e3b80 1 Processor -- start 2023-10-16T14:35:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.691+0000 7f4cc67e3b80 1 -- start start 2023-10-16T14:35:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bc2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03ba67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03980920 con 0x558a03bb1230 2023-10-16T14:35:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03bc6930 con 0x558a03bb1b90 2023-10-16T14:35:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03ba9fb0 con 0x558a03bb2a30 2023-10-16T14:35:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39372/0 (socket says 172.21.15.178:39372) 2023-10-16T14:35:26.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/840124926 learned_addr learned my addr 172.21.15.178:0/840124926 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cabfff700 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bc2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb139c700 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03ba67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/840124926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 msgr2=0x558a03ba67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.721 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03ba67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/840124926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 msgr2=0x558a03bc2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bc2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/840124926 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0392ade0 con 0x558a03bb1230 2023-10-16T14:35:26.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.692+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0011480 tx=0x7f4ca0005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29c7911b333004 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4c977fe700 1 -- 172.21.15.178:0/840124926 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011e20 con 0x558a03bb1230 2023-10-16T14:35:26.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4c977fe700 1 -- 172.21.15.178:0/840124926 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca0019430 con 0x558a03bb1230 2023-10-16T14:35:26.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4c977fe700 1 -- 172.21.15.178:0/840124926 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0011020 con 0x558a03bb1230 2023-10-16T14:35:26.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/840124926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 msgr2=0x558a03bb1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0011480 tx=0x7f4ca0005e00 comp rx=0 tx=0).stop 2023-10-16T14:35:26.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/840124926 shutdown_connections 2023-10-16T14:35:26.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb2a30 0x558a03ba67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bb1610 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/840124926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb1b90 0x558a03bc2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/840124926 >> 172.21.15.178:0/840124926 conn(0x558a03adadd0 msgr2=0x558a03bb9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/840124926 shutdown_connections 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.693+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/840124926 wait complete. 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 -- start start 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 Processor -- start 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1b90 0x558a03bc0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.725 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03bc1780 con 0x558a03bb1230 2023-10-16T14:35:26.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03bc18f0 con 0x558a03bb2a30 2023-10-16T14:35:26.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.694+0000 7f4cc67e3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a03bc1ab0 con 0x558a03bb1b90 2023-10-16T14:35:26.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cc67e3b80 10 client.0 _ll_register_callbacks cb 0x7f4c98096110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:26.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39388/0 (socket says 172.21.15.178:39388) 2023-10-16T14:35:26.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2324972965 learned_addr learned my addr 172.21.15.178:0/2324972965 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb139c700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cabfff700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1b90 0x558a03bc0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2324972965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1b90 msgr2=0x558a03bc0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1b90 0x558a03bc0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 msgr2=0x558a03bc1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 -- 172.21.15.178:0/2324972965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca0009920 con 0x558a03bb1230 2023-10-16T14:35:26.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb139c700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4cb0b9b700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009340 tx=0x7f4ca0004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6708d56b4dfdd03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.695+0000 7f4c8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:26.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca0005640 con 0x558a03bb1230 2023-10-16T14:35:26.729 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ca00044e0 con 0x558a03bb1230 2023-10-16T14:35:26.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4ca00057a0 con 0x558a03bb1230 2023-10-16T14:35:26.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ca0011020 con 0x558a03bb1230 2023-10-16T14:35:26.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 10 client.15678 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:26.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x558a03913c60 con 0x558a03bb1230 2023-10-16T14:35:26.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ca00044e0 con 0x558a03bb1230 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f4ca0024860 con 0x558a03bb1230 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 20 client.15678 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03c1b0a0 0x558a03c1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 4 client.15678 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:35:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.696+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a03af8e50 con 0x558a03c1b0a0 2023-10-16T14:35:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.697+0000 7f4cabfff700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03c1b0a0 0x558a03c1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.697+0000 7f4c8bfff700 10 client.15678 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:35:26.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.697+0000 7f4cabfff700 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03c1b0a0 0x558a03c1d490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=832777981b9dac03 server_cookie=1ed383e525a2f21d in_seq=0 out_seq=0 2023-10-16T14:35:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4c8bfff700 1 -- 172.21.15.178:0/2324972965 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a03af8e50 con 0x558a03c1b0a0 2023-10-16T14:35:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4c8bfff700 10 client.15678 handle_command_reply: tid=0 2023-10-16T14:35:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 client.15678 shutdown 2023-10-16T14:35:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 msgr2=0x558a03bbfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0009340 tx=0x7f4ca0004a60 comp rx=0 tx=0).stop 2023-10-16T14:35:26.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 shutdown_connections 2023-10-16T14:35:26.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x558a03c1b0a0 0x558a03c1d490 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558a03bb2a30 0x558a03bc1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558a03bb1230 0x558a03bbfcf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4c8bfff700 0 client.15678 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:35:26.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 --2- 172.21.15.178:0/2324972965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558a03bb1b90 0x558a03bc0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 >> 172.21.15.178:0/2324972965 conn(0x558a03adadd0 msgr2=0x558a03bb8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 shutdown_connections 2023-10-16T14:35:26.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 1 -- 172.21.15.178:0/2324972965 wait complete. 2023-10-16T14:35:26.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cc67e3b80 20 client.15678 trim_cache size 0 max 16384 2023-10-16T14:35:26.736 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:35:26.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 msgr2=0x7f4c9c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.698+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 0x7f4c9c0a39c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40062a0 tx=0x7f4ca40077a0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 msgr2=0x7f4cac197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800ea50 tx=0x7f4c9800ed60 comp rx=0 tx=0).stop 2023-10-16T14:35:26.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 shutdown_connections 2023-10-16T14:35:26.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4c9c0a1500 0x7f4c9c0a39c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cac102a40 0x7f4cac19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cac100ea0 0x7f4cac196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 --2- 172.21.15.178:0/1430268933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cac102080 0x7f4cac197280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 >> 172.21.15.178:0/1430268933 conn(0x7f4cac0fc420 msgr2=0x7f4cac0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 shutdown_connections 2023-10-16T14:35:26.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.699+0000 7f4cb2dff700 1 -- 172.21.15.178:0/1430268933 wait complete. 2023-10-16T14:35:26.740 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:26 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/400865227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:35:26.741 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:35:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 -- 172.21.15.178:0/174985700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908073910 msgr2=0x7f2908073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 --2- 172.21.15.178:0/174985700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908073910 0x7f2908073d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0098f0 tx=0x7f28fc009c00 comp rx=0 tx=0).stop 2023-10-16T14:35:26.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 -- 172.21.15.178:0/174985700 shutdown_connections 2023-10-16T14:35:26.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 --2- 172.21.15.178:0/174985700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908108d90 0x7f290810b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 --2- 172.21.15.178:0/174985700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f2908073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 --2- 172.21.15.178:0/174985700 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908073910 0x7f2908073d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 -- 172.21.15.178:0/174985700 >> 172.21.15.178:0/174985700 conn(0x7f29080fc400 msgr2=0x7f29080fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 -- 172.21.15.178:0/174985700 shutdown_connections 2023-10-16T14:35:26.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.940+0000 7f290dbff700 1 -- 172.21.15.178:0/174985700 wait complete. 2023-10-16T14:35:26.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.941+0000 7f290dbff700 1 Processor -- start 2023-10-16T14:35:26.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.941+0000 7f290dbff700 1 -- start start 2023-10-16T14:35:26.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908073910 0x7f2908198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081a09f0 con 0x7f2908074ef0 2023-10-16T14:35:26.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081a0b30 con 0x7f2908108d90 2023-10-16T14:35:26.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f290dbff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081a0cc0 con 0x7f2908073910 2023-10-16T14:35:26.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f2907fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f2907fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55582/0 (socket says 172.21.15.178:55582) 2023-10-16T14:35:26.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39404/0 (socket says 172.21.15.178:39404) 2023-10-16T14:35:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f29077fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908073910 0x7f2908198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.942+0000 7f2907fff700 1 -- 172.21.15.178:0/3161993810 learned_addr learned my addr 172.21.15.178:0/3161993810 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 -- 172.21.15.178:0/3161993810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908073910 msgr2=0x7f2908198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908073910 0x7f2908198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 -- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 msgr2=0x7f29081993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 -- 172.21.15.178:0/3161993810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009520 con 0x7f2908108d90 2023-10-16T14:35:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2907fff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f290000ef30 tx=0x7f290000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2904ff9700 1 -- 172.21.15.178:0/3161993810 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900018070 con 0x7f2908108d90 2023-10-16T14:35:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2904ff9700 1 -- 172.21.15.178:0/3161993810 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f290000f040 con 0x7f2908108d90 2023-10-16T14:35:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29081a0f40 con 0x7f2908108d90 2023-10-16T14:35:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f2904ff9700 1 -- 172.21.15.178:0/3161993810 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000ccf0 con 0x7f2908108d90 2023-10-16T14:35:26.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.943+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081a1370 con 0x7f2908108d90 2023-10-16T14:35:26.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.946+0000 7f2904ff9700 1 -- 172.21.15.178:0/3161993810 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f2900003a00 con 0x7f2908108d90 2023-10-16T14:35:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.948+0000 7f29215d4b80 1 Processor -- start 2023-10-16T14:35:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.948+0000 7f29215d4b80 1 -- start start 2023-10-16T14:35:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2904ff9700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 0x7f28f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674d8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55906744bc70 con 0x5590674d8620 2023-10-16T14:35:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x559067430c60 con 0x5590674d7780 2023-10-16T14:35:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29215d4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55906742e580 con 0x5590674d6e20 2023-10-16T14:35:26.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2904ff9700 1 -- 172.21.15.178:0/3161993810 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f29000ccc00 con 0x7f2908108d90 2023-10-16T14:35:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2907fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43726/0 (socket says 172.21.15.178:43726) 2023-10-16T14:35:26.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 -- 172.21.15.178:0/2933147260 learned_addr learned my addr 172.21.15.178:0/2933147260 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 0x7f28f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2906ffd700 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 -- 172.21.15.178:0/2933147260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 msgr2=0x5590674d8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674d8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 -- 172.21.15.178:0/2933147260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 msgr2=0x5590674d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674d8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f29077fe700 1 -- 172.21.15.178:0/2933147260 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590673e73f0 con 0x5590674d6e20 2023-10-16T14:35:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.949+0000 7f2907fff700 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.950+0000 7f29077fe700 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d60 tx=0x7f28f8011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed989e3ac7197242 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.950+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 0x7f28f00a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.950+0000 7f28edffb700 1 -- 172.21.15.178:0/2933147260 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8019630 con 0x5590674d6e20 2023-10-16T14:35:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.950+0000 7f28edffb700 1 -- 172.21.15.178:0/2933147260 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28f8019c50 con 0x5590674d6e20 2023-10-16T14:35:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.950+0000 7f28edffb700 1 -- 172.21.15.178:0/2933147260 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800ebe0 con 0x5590674d6e20 2023-10-16T14:35:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- 172.21.15.178:0/2933147260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 msgr2=0x5590674d7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d60 tx=0x7f28f8011720 comp rx=0 tx=0).stop 2023-10-16T14:35:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- 172.21.15.178:0/2933147260 shutdown_connections 2023-10-16T14:35:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674d7240 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 --2- 172.21.15.178:0/2933147260 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674d8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- 172.21.15.178:0/2933147260 >> 172.21.15.178:0/2933147260 conn(0x559067404dd0 msgr2=0x5590674d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- 172.21.15.178:0/2933147260 shutdown_connections 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- 172.21.15.178:0/2933147260 wait complete. 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.951+0000 7f29215d4b80 1 -- start start 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 Processor -- start 2023-10-16T14:35:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559067429e70 con 0x5590674d8620 2023-10-16T14:35:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590673c6710 con 0x5590674d7780 2023-10-16T14:35:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590673c63b0 con 0x5590674d6e20 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29215d4b80 10 client.0 _ll_register_callbacks cb 0x559067223800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f2907fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55598/0 (socket says 172.21.15.178:55598) 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f2906ffd700 1 -- 172.21.15.178:0/3992825772 learned_addr learned my addr 172.21.15.178:0/3992825772 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.952+0000 7f29077fe700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 -- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 msgr2=0x5590674e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 -- 172.21.15.178:0/3992825772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 msgr2=0x5590674e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674e81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 -- 172.21.15.178:0/3992825772 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081a1370 con 0x5590674d7780 2023-10-16T14:35:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2907fff700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f2906ffd700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28fc005e90 tx=0x7f28fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f4bfa2decfe4a3d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f28e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:35:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02c070 con 0x5590674d7780 2023-10-16T14:35:26.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55906742eef0 con 0x5590674d7780 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.953+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28fc005040 con 0x5590674d7780 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28fc00bc00 con 0x5590674d7780 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 10 client.25736 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f28f80099c0 con 0x5590674d7780 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55906742eef0 con 0x5590674d7780 2023-10-16T14:35:26.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28fc03a070 con 0x5590674d7780 2023-10-16T14:35:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 10 client.25736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:35:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 20 client.25736 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x5590674ecb40 0x5590674eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 4 client.25736 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590674a6c10 con 0x5590674ecb40 2023-10-16T14:35:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.954+0000 7f29077fe700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x5590674ecb40 0x5590674eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.955+0000 7f28e6ffd700 10 client.25736 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:35:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.955+0000 7f29077fe700 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x5590674ecb40 0x5590674eef60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f843ddf07e4058f server_cookie=a7705d5966428150 in_seq=0 out_seq=0 2023-10-16T14:35:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f28e6ffd700 1 -- 172.21.15.178:0/3992825772 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5590674a6c10 con 0x5590674ecb40 2023-10-16T14:35:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f28e6ffd700 10 client.25736 handle_command_reply: tid=0 2023-10-16T14:35:26.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 client.25736 shutdown 2023-10-16T14:35:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 msgr2=0x5590674e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f28fc005e90 tx=0x7f28fc009c00 comp rx=0 tx=0).stop 2023-10-16T14:35:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 shutdown_connections 2023-10-16T14:35:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x5590674ecb40 0x5590674eef60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d8620 0x5590674e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d6e20 0x5590674e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 --2- 172.21.15.178:0/3992825772 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d7780 0x5590674e11b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f28e6ffd700 0 client.25736 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:35:26.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 >> 172.21.15.178:0/3992825772 conn(0x559067404dd0 msgr2=0x5590674d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 shutdown_connections 2023-10-16T14:35:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 1 -- 172.21.15.178:0/3992825772 wait complete. 2023-10-16T14:35:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.956+0000 7f29215d4b80 20 client.25736 trim_cache size 0 max 16384 2023-10-16T14:35:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.969+0000 7f29215d4b80 1 Processor -- start 2023-10-16T14:35:26.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 -- start start 2023-10-16T14:35:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674d1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590672aa920 con 0x5590674d7780 2023-10-16T14:35:26.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590674d7ed0 con 0x5590674d6e20 2023-10-16T14:35:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29215d4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590674d0e20 con 0x5590674d8620 2023-10-16T14:35:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39438/0 (socket says 172.21.15.178:39438) 2023-10-16T14:35:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 -- 172.21.15.178:0/783056415 learned_addr learned my addr 172.21.15.178:0/783056415 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29077fe700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2907fff700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674d1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 -- 172.21.15.178:0/783056415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 msgr2=0x5590674d1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674d1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 -- 172.21.15.178:0/783056415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 msgr2=0x5590674d8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674d8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2906ffd700 1 -- 172.21.15.178:0/783056415 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559067254de0 con 0x5590674d7780 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f2907fff700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.970+0000 7f29077fe700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f2906ffd700 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009230 tx=0x7f28fc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9499910f89a140f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f28ee7fc700 1 -- 172.21.15.178:0/783056415 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02c070 con 0x5590674d7780 2023-10-16T14:35:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f28ee7fc700 1 -- 172.21.15.178:0/783056415 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28fc004b70 con 0x5590674d7780 2023-10-16T14:35:26.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f28ee7fc700 1 -- 172.21.15.178:0/783056415 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0056f0 con 0x5590674d7780 2023-10-16T14:35:26.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f29215d4b80 1 -- 172.21.15.178:0/783056415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 msgr2=0x5590674e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.971+0000 7f29215d4b80 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009230 tx=0x7f28fc00bc80 comp rx=0 tx=0).stop 2023-10-16T14:35:26.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 -- 172.21.15.178:0/783056415 shutdown_connections 2023-10-16T14:35:26.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674d1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 --2- 172.21.15.178:0/783056415 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e81b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 -- 172.21.15.178:0/783056415 >> 172.21.15.178:0/783056415 conn(0x559067404dd0 msgr2=0x5590674df950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:26.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 -- 172.21.15.178:0/783056415 shutdown_connections 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 -- 172.21.15.178:0/783056415 wait complete. 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 -- start start 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.972+0000 7f29215d4b80 1 Processor -- start 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590674e7650 con 0x5590674d7780 2023-10-16T14:35:26.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590674e77c0 con 0x5590674d6e20 2023-10-16T14:35:26.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590674e7980 con 0x5590674d8620 2023-10-16T14:35:26.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29215d4b80 10 client.0 _ll_register_callbacks cb 0x7f28fc00f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:26.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f29077fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f2906ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39452/0 (socket says 172.21.15.178:39452) 2023-10-16T14:35:26.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f2906ffd700 1 -- 172.21.15.178:0/4289292205 learned_addr learned my addr 172.21.15.178:0/4289292205 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:26.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.973+0000 7f2907fff700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 -- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 msgr2=0x5590674e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 -- 172.21.15.178:0/4289292205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 msgr2=0x5590674e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 -- 172.21.15.178:0/4289292205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f80099c0 con 0x5590674d7780 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f2906ffd700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0092a0 tx=0x7f28fc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d481217983db5d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:26.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f28e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:26.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02c030 con 0x5590674d7780 2023-10-16T14:35:26.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28fc02d070 con 0x5590674d7780 2023-10-16T14:35:26.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f8019c50 con 0x5590674d7780 2023-10-16T14:35:26.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28fc027480 con 0x5590674d7780 2023-10-16T14:35:26.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f29215d4b80 10 client.15696 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:26.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.974+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55906723dc60 con 0x5590674d7780 2023-10-16T14:35:26.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28fc052020 con 0x5590674d7780 2023-10-16T14:35:26.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f28fc015020 con 0x5590674d7780 2023-10-16T14:35:26.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:26.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 10 client.15696 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:35:26.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 20 client.15696 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:26.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559067540f60 0x559067543350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:26.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 4 client.15696 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:35:26.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559067422e50 con 0x559067540f60 2023-10-16T14:35:26.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.975+0000 7f29077fe700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559067540f60 0x559067543350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:26.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29077fe700 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559067540f60 0x559067543350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b2cd8ac2dd90005 server_cookie=795501059219e0ca in_seq=0 out_seq=0 2023-10-16T14:35:26.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f28e6ffd700 10 client.15696 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:35:26.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f28e6ffd700 1 -- 172.21.15.178:0/4289292205 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559067422e50 con 0x559067540f60 2023-10-16T14:35:26.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f28e6ffd700 10 client.15696 handle_command_reply: tid=0 2023-10-16T14:35:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 client.15696 shutdown 2023-10-16T14:35:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 msgr2=0x5590674e40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0092a0 tx=0x7f28fc010fd0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 shutdown_connections 2023-10-16T14:35:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x559067540f60 0x559067543350 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590674d8620 0x5590674e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5590674d6e20 0x5590674e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 --2- 172.21.15.178:0/4289292205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5590674d7780 0x5590674e40f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:26.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f28e6ffd700 0 client.15696 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:35:26.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 >> 172.21.15.178:0/4289292205 conn(0x559067404dd0 msgr2=0x5590674db4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 shutdown_connections 2023-10-16T14:35:27.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.976+0000 7f29215d4b80 1 -- 172.21.15.178:0/4289292205 wait complete. 2023-10-16T14:35:27.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f29215d4b80 20 client.15696 trim_cache size 0 max 16384 2023-10-16T14:35:27.000 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:35:27.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 msgr2=0x7f28f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 0x7f28f00a3940 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009fd0 tx=0x7f28fc019040 comp rx=0 tx=0).stop 2023-10-16T14:35:27.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 msgr2=0x7f29081a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f290000ef30 tx=0x7f290000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 shutdown_connections 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f28f00a1480 0x7f28f00a3940 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2908108d90 0x7f29081a0420 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2908073910 0x7f2908198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 --2- 172.21.15.178:0/3161993810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2908074ef0 0x7f29081993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 >> 172.21.15.178:0/3161993810 conn(0x7f29080fc400 msgr2=0x7f29080fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 shutdown_connections 2023-10-16T14:35:27.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:26.977+0000 7f290dbff700 1 -- 172.21.15.178:0/3161993810 wait complete. 2023-10-16T14:35:27.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:35:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:26 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/400865227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:35:27.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:26 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/400865227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:35:27.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 -- 172.21.15.178:0/1560836485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc100ea0 msgr2=0x7fc2cc1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/1560836485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc1012c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009990 tx=0x7fc2bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 -- 172.21.15.178:0/1560836485 shutdown_connections 2023-10-16T14:35:27.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/1560836485 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/1560836485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc1012c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/1560836485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc102080 0x7fc2cc102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 -- 172.21.15.178:0/1560836485 >> 172.21.15.178:0/1560836485 conn(0x7fc2cc0fc420 msgr2=0x7fc2cc0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 -- 172.21.15.178:0/1560836485 shutdown_connections 2023-10-16T14:35:27.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.202+0000 7fc2d25ff700 1 -- 172.21.15.178:0/1560836485 wait complete. 2023-10-16T14:35:27.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 Processor -- start 2023-10-16T14:35:27.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 -- start start 2023-10-16T14:35:27.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a0a50 con 0x7fc2cc102080 2023-10-16T14:35:27.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a0b90 con 0x7fc2cc102a40 2023-10-16T14:35:27.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.203+0000 7fc2d25ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc1a0d20 con 0x7fc2cc100ea0 2023-10-16T14:35:27.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39476/0 (socket says 172.21.15.178:39476) 2023-10-16T14:35:27.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/3406198817 learned_addr learned my addr 172.21.15.178:0/3406198817 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/3406198817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc100ea0 msgr2=0x7fc2cc198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/3406198817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 msgr2=0x7fc2cc1a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc1a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/3406198817 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0095f0 con 0x7fc2cc102080 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d980 tx=0x7fc2c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2c97fa700 1 -- 172.21.15.178:0/3406198817 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0004020 con 0x7fc2cc102080 2023-10-16T14:35:27.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2c97fa700 1 -- 172.21.15.178:0/3406198817 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c0009730 con 0x7fc2cc102080 2023-10-16T14:35:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.204+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc1a0fa0 con 0x7fc2cc102080 2023-10-16T14:35:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.205+0000 7fc2c97fa700 1 -- 172.21.15.178:0/3406198817 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003a80 con 0x7fc2cc102080 2023-10-16T14:35:27.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.205+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc1a14d0 con 0x7fc2cc102080 2023-10-16T14:35:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.207+0000 7fc2c97fa700 1 -- 172.21.15.178:0/3406198817 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc2c00098a0 con 0x7fc2cc102080 2023-10-16T14:35:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2c97fa700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 0x7fc2b40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 Processor -- start 2023-10-16T14:35:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 -- start start 2023-10-16T14:35:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2c97fa700 1 -- 172.21.15.178:0/3406198817 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2c00ccd10 con 0x7fc2cc102080 2023-10-16T14:35:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 0x7fc2b40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adb5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ad29c70 con 0x56400adb4e20 2023-10-16T14:35:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ad0ec60 con 0x56400adb6620 2023-10-16T14:35:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.209+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ad0c580 con 0x56400adb5780 2023-10-16T14:35:27.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adb5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43772/0 (socket says 172.21.15.178:43772) 2023-10-16T14:35:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/199271044 learned_addr learned my addr 172.21.15.178:0/199271044 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adb6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 0x7fc2b40a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b5c0 tx=0x7fc2bc00be00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/199271044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 msgr2=0x56400adb6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adb6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/199271044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 msgr2=0x56400adb5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adb5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 -- 172.21.15.178:0/199271044 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56400acc53f0 con 0x56400adb5780 2023-10-16T14:35:27.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adb5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.210+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0099bf0 tx=0x7fc2c0099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faec179c5857bc42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2b27fc700 1 -- 172.21.15.178:0/199271044 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c009ddb0 con 0x56400adb5780 2023-10-16T14:35:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2b27fc700 1 -- 172.21.15.178:0/199271044 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c00a1ff0 con 0x56400adb5780 2023-10-16T14:35:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2b27fc700 1 -- 172.21.15.178:0/199271044 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0099150 con 0x56400adb5780 2023-10-16T14:35:27.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/199271044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 msgr2=0x56400adb6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0099bf0 tx=0x7fc2c0099690 comp rx=0 tx=0).stop 2023-10-16T14:35:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/199271044 shutdown_connections 2023-10-16T14:35:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adb5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/199271044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adb6040 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/199271044 >> 172.21.15.178:0/199271044 conn(0x56400ace2dd0 msgr2=0x56400adaffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/199271044 shutdown_connections 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.211+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/199271044 wait complete. 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.212+0000 7fc2e5fa6b80 1 -- start start 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.212+0000 7fc2e5fa6b80 1 Processor -- start 2023-10-16T14:35:27.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.212+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.212+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adbf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adc6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ad07e70 con 0x56400adb4e20 2023-10-16T14:35:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400aca4710 con 0x56400adb6620 2023-10-16T14:35:27.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400aca43b0 con 0x56400adb5780 2023-10-16T14:35:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2e5fa6b80 10 client.0 _ll_register_callbacks cb 0x56400ab01800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39496/0 (socket says 172.21.15.178:39496) 2023-10-16T14:35:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 -- 172.21.15.178:0/1396197926 learned_addr learned my addr 172.21.15.178:0/1396197926 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:27.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adbf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adc6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 -- 172.21.15.178:0/1396197926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 msgr2=0x56400adbf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adbf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 -- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 msgr2=0x56400adc6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adc6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 -- 172.21.15.178:0/1396197926 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc109c90 con 0x56400adb4e20 2023-10-16T14:35:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003b10 tx=0x7fc2bc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ebb0de67a3b566 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.213+0000 7fc2ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc005710 con 0x56400adb4e20 2023-10-16T14:35:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2bc00f460 con 0x56400adb4e20 2023-10-16T14:35:27.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56400ad0cef0 con 0x56400adb4e20 2023-10-16T14:35:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2bc00f620 con 0x56400adb4e20 2023-10-16T14:35:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2e5fa6b80 10 client.15708 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fc2c00b9040 con 0x56400adb4e20 2023-10-16T14:35:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56400ad0cef0 con 0x56400adb4e20 2023-10-16T14:35:27.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc2bc025770 con 0x56400adb4e20 2023-10-16T14:35:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2e5fa6b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.214+0000 7fc2e5fa6b80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2e5fa6b80 20 client.15708 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400adcaa50 0x56400adcce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2e5fa6b80 4 client.15708 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56400ad84c10 con 0x56400adcaa50 2023-10-16T14:35:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400adcaa50 0x56400adcce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2cb7fe700 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400adcaa50 0x56400adcce70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2a8a091ef2764 server_cookie=177c4398e0832ef8 in_seq=0 out_seq=0 2023-10-16T14:35:27.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.215+0000 7fc2ab7fe700 10 client.15708 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/1396197926 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56400ad84c10 con 0x56400adcaa50 2023-10-16T14:35:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2ab7fe700 10 client.15708 handle_command_reply: tid=0 2023-10-16T14:35:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 client.15708 shutdown 2023-10-16T14:35:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 msgr2=0x56400adbecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc003b10 tx=0x7fc2bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 shutdown_connections 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400adcaa50 0x56400adcce70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb6620 0x56400adc6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2ab7fe700 0 client.15708 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb4e20 0x56400adbecf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/1396197926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb5780 0x56400adbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.216+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 >> 172.21.15.178:0/1396197926 conn(0x56400ace2dd0 msgr2=0x56400adafbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.217+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 shutdown_connections 2023-10-16T14:35:27.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.217+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/1396197926 wait complete. 2023-10-16T14:35:27.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.217+0000 7fc2e5fa6b80 20 client.15708 trim_cache size 0 max 16384 2023-10-16T14:35:27.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 Processor -- start 2023-10-16T14:35:27.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 -- start start 2023-10-16T14:35:27.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400ab88920 con 0x56400adb6620 2023-10-16T14:35:27.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400adb64a0 con 0x56400adb5780 2023-10-16T14:35:27.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400adb1fb0 con 0x56400adb4e20 2023-10-16T14:35:27.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2d0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2cbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2d0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39504/0 (socket says 172.21.15.178:39504) 2023-10-16T14:35:27.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.230+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/3079555778 learned_addr learned my addr 172.21.15.178:0/3079555778 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:27.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/3079555778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 msgr2=0x56400adb5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adb5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/3079555778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 msgr2=0x56400adc6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/3079555778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56400ab32de0 con 0x56400adb6620 2023-10-16T14:35:27.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009e00 tx=0x7fc2c40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f517b235f927d31 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2b2ffd700 1 -- 172.21.15.178:0/3079555778 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4017640 con 0x56400adb6620 2023-10-16T14:35:27.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2b2ffd700 1 -- 172.21.15.178:0/3079555778 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c400f040 con 0x56400adb6620 2023-10-16T14:35:27.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.231+0000 7fc2b2ffd700 1 -- 172.21.15.178:0/3079555778 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4016940 con 0x56400adb6620 2023-10-16T14:35:27.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/3079555778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 msgr2=0x56400adbcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009e00 tx=0x7fc2c40077e0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/3079555778 shutdown_connections 2023-10-16T14:35:27.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adbcab0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adb5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/3079555778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/3079555778 >> 172.21.15.178:0/3079555778 conn(0x56400ace2dd0 msgr2=0x56400adae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/3079555778 shutdown_connections 2023-10-16T14:35:27.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/3079555778 wait complete. 2023-10-16T14:35:27.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 -- start start 2023-10-16T14:35:27.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.232+0000 7fc2e5fa6b80 1 Processor -- start 2023-10-16T14:35:27.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adc4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400adc5d90 con 0x56400adb6620 2023-10-16T14:35:27.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400adc5f00 con 0x56400adb5780 2023-10-16T14:35:27.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56400adc6070 con 0x56400adb4e20 2023-10-16T14:35:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2e5fa6b80 10 client.0 _ll_register_callbacks cb 0x56400ab01800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2cb7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39516/0 (socket says 172.21.15.178:39516) 2023-10-16T14:35:27.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/638260054 learned_addr learned my addr 172.21.15.178:0/638260054 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adc4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/638260054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 msgr2=0x56400adc4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adc4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 msgr2=0x56400adc4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.233+0000 7fc2d0b9c700 1 -- 172.21.15.178:0/638260054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c4009a80 con 0x56400adb6620 2023-10-16T14:35:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2d0b9c700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4016550 tx=0x7fc2c4016d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdca5aaf68e9dbb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2ab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4016f00 con 0x56400adb6620 2023-10-16T14:35:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2c4020450 con 0x56400adb6620 2023-10-16T14:35:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2c00a1ff0 con 0x56400adb6620 2023-10-16T14:35:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2c4020610 con 0x56400adb6620 2023-10-16T14:35:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2e5fa6b80 10 client.15714 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56400ab1bc60 con 0x56400adb6620 2023-10-16T14:35:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.234+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2c00a1ff0 con 0x56400adb6620 2023-10-16T14:35:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fc2c4014050 con 0x56400adb6620 2023-10-16T14:35:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 20 client.15714 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400ae1f9b0 0x56400ae21dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 4 client.15714 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56400ad00e50 con 0x56400ae1f9b0 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.235+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400ae1f9b0 0x56400ae21dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2ab7fe700 10 client.15714 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2cbfff700 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400ae1f9b0 0x56400ae21dd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1f2dd17376db3a server_cookie=9a39d52b2ccf4ad1 in_seq=0 out_seq=0 2023-10-16T14:35:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2ab7fe700 1 -- 172.21.15.178:0/638260054 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56400ad00e50 con 0x56400ae1f9b0 2023-10-16T14:35:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2ab7fe700 10 client.15714 handle_command_reply: tid=0 2023-10-16T14:35:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 client.15714 shutdown 2023-10-16T14:35:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 msgr2=0x56400adc5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4016550 tx=0x7fc2c4016d50 comp rx=0 tx=0).stop 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 shutdown_connections 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56400ae1f9b0 0x56400ae21dd0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56400adb6620 0x56400adc5850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56400adb4e20 0x56400adc4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2ab7fe700 0 client.15714 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:27.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 --2- 172.21.15.178:0/638260054 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56400adb5780 0x56400adc4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 >> 172.21.15.178:0/638260054 conn(0x56400ace2dd0 msgr2=0x56400adbd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 shutdown_connections 2023-10-16T14:35:27.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 1 -- 172.21.15.178:0/638260054 wait complete. 2023-10-16T14:35:27.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.236+0000 7fc2e5fa6b80 20 client.15714 trim_cache size 0 max 16384 2023-10-16T14:35:27.263 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:35:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 msgr2=0x7fc2b40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 0x7fc2b40a39c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b5c0 tx=0x7fc2bc00be00 comp rx=0 tx=0).stop 2023-10-16T14:35:27.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 msgr2=0x7fc2cc199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000d980 tx=0x7fc2c000dc90 comp rx=0 tx=0).stop 2023-10-16T14:35:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 shutdown_connections 2023-10-16T14:35:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc2b40a1500 0x7fc2b40a39c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc2cc102a40 0x7fc2cc1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2cc100ea0 0x7fc2cc198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 --2- 172.21.15.178:0/3406198817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2cc102080 0x7fc2cc199410 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 >> 172.21.15.178:0/3406198817 conn(0x7fc2cc0fc420 msgr2=0x7fc2cc107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:27.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 shutdown_connections 2023-10-16T14:35:27.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:27.237+0000 7fc2d25ff700 1 -- 172.21.15.178:0/3406198817 wait complete. 2023-10-16T14:35:27.528 INFO:tasks.workunit.client.0.smithi178.stdout:config.status: executing depfiles commands 2023-10-16T14:35:27.857 INFO:tasks.workunit.client.0.smithi178.stderr:+ make 2023-10-16T14:35:27.871 INFO:tasks.workunit.client.0.smithi178.stdout:make all-recursive 2023-10-16T14:35:27.873 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-16T14:35:27.880 INFO:tasks.workunit.client.0.smithi178.stdout:Making all in src 2023-10-16T14:35:27.891 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-16T14:35:27.891 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2023-10-16T14:35:27.891 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2023-10-16T14:35:28.035 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2023-10-16T14:35:28.036 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2023-10-16T14:35:28.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:27 smithi178 ceph-mon[98495]: pgmap v391: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 327 KiB/s wr, 48 op/s 2023-10-16T14:35:28.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:27 smithi186 ceph-mon[134892]: pgmap v391: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 327 KiB/s wr, 48 op/s 2023-10-16T14:35:28.165 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2023-10-16T14:35:28.165 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2023-10-16T14:35:28.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:27 smithi187 ceph-mon[135297]: pgmap v391: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 327 KiB/s wr, 48 op/s 2023-10-16T14:35:28.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:35:28.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:35:28.280 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2023-10-16T14:35:28.280 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2023-10-16T14:35:28.395 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2023-10-16T14:35:28.395 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2023-10-16T14:35:28.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 -- 172.21.15.178:0/2398673730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0ff7a0 msgr2=0x7f0eec0ffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/2398673730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec0ffbc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0098f0 tx=0x7f0edc009c00 comp rx=0 tx=0).stop 2023-10-16T14:35:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 -- 172.21.15.178:0/2398673730 shutdown_connections 2023-10-16T14:35:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/2398673730 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/2398673730 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec0ffbc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/2398673730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec100100 0x7f0eec100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 -- 172.21.15.178:0/2398673730 >> 172.21.15.178:0/2398673730 conn(0x7f0eec0fb320 msgr2=0x7f0eec0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 -- 172.21.15.178:0/2398673730 shutdown_connections 2023-10-16T14:35:28.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.474+0000 7f0ef0fff700 1 -- 172.21.15.178:0/2398673730 wait complete. 2023-10-16T14:35:28.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.476+0000 7f0ef0fff700 1 Processor -- start 2023-10-16T14:35:28.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.476+0000 7f0ef0fff700 1 -- start start 2023-10-16T14:35:28.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.476+0000 7f0ef0fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.476+0000 7f0ef0fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ef0fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec1a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ef0fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1a0990 con 0x7f0eec100100 2023-10-16T14:35:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ef0fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1a0ad0 con 0x7f0eec100ac0 2023-10-16T14:35:28.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ef0fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1a0c60 con 0x7f0eec0ff7a0 2023-10-16T14:35:28.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39534/0 (socket says 172.21.15.178:39534) 2023-10-16T14:35:28.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/185116304 learned_addr learned my addr 172.21.15.178:0/185116304 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0eead9d700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec1a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/185116304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec0ff7a0 msgr2=0x7f0eec198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/185116304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 msgr2=0x7f0eec1a03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec1a03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/185116304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc009520 con 0x7f0eec100100 2023-10-16T14:35:28.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.477+0000 7f0eead9d700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:28.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000e9a0 tx=0x7f0ee000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0edb7fe700 1 -- 172.21.15.178:0/185116304 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0009790 con 0x7f0eec100100 2023-10-16T14:35:28.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0edb7fe700 1 -- 172.21.15.178:0/185116304 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee0010430 con 0x7f0eec100100 2023-10-16T14:35:28.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec1a0f40 con 0x7f0eec100100 2023-10-16T14:35:28.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0edb7fe700 1 -- 172.21.15.178:0/185116304 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee0018680 con 0x7f0eec100100 2023-10-16T14:35:28.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.478+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1a1410 con 0x7f0eec100100 2023-10-16T14:35:28.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.482+0000 7f0edb7fe700 1 -- 172.21.15.178:0/185116304 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f0ee00105a0 con 0x7f0eec100100 2023-10-16T14:35:28.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.483+0000 7f0edb7fe700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 0x7f0ed40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.483+0000 7f0f049dcb80 1 Processor -- start 2023-10-16T14:35:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.483+0000 7f0edb7fe700 1 -- 172.21.15.178:0/185116304 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0ee0014070 con 0x7f0eec100100 2023-10-16T14:35:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.483+0000 7f0eea59c700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 0x7f0ed40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 -- start start 2023-10-16T14:35:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd3e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.490 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd3f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdcb2cf0 con 0x5563bdd3e800 2023-10-16T14:35:28.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc97ce0 con 0x5563bdd3f6a0 2023-10-16T14:35:28.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc95600 con 0x5563bdd3dea0 2023-10-16T14:35:28.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd3f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0eead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0eead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55708/0 (socket says 172.21.15.178:55708) 2023-10-16T14:35:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0eead9d700 1 -- 172.21.15.178:0/3355719301 learned_addr learned my addr 172.21.15.178:0/3355719301 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0eea59c700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 0x7f0ed40a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005e90 tx=0x7f0edc019040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.484+0000 7f0eea59c700 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd3e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 -- 172.21.15.178:0/3355719301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 msgr2=0x5563bdd3e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd3e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 -- 172.21.15.178:0/3355719301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 msgr2=0x5563bdd3f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd3f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 -- 172.21.15.178:0/3355719301 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bdc4e470 con 0x5563bdd3f6a0 2023-10-16T14:35:28.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0eead9d700 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003eb0 tx=0x7f0ee40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fa3c062f0d1a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0ed8ff9700 1 -- 172.21.15.178:0/3355719301 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40092d0 con 0x5563bdd3f6a0 2023-10-16T14:35:28.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0ed8ff9700 1 -- 172.21.15.178:0/3355719301 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee400f040 con 0x5563bdd3f6a0 2023-10-16T14:35:28.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.485+0000 7f0ed8ff9700 1 -- 172.21.15.178:0/3355719301 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400e750 con 0x5563bdd3f6a0 2023-10-16T14:35:28.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 -- 172.21.15.178:0/3355719301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 msgr2=0x5563bdd3fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003eb0 tx=0x7f0ee40099e0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 -- 172.21.15.178:0/3355719301 shutdown_connections 2023-10-16T14:35:28.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd3fb20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd3e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/3355719301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd3f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 -- 172.21.15.178:0/3355719301 >> 172.21.15.178:0/3355719301 conn(0x5563bdc6be50 msgr2=0x5563bdd39060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 -- 172.21.15.178:0/3355719301 shutdown_connections 2023-10-16T14:35:28.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.486+0000 7f0f049dcb80 1 -- 172.21.15.178:0/3355719301 wait complete. 2023-10-16T14:35:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 -- start start 2023-10-16T14:35:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 Processor -- start 2023-10-16T14:35:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd47ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd4f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc90ef0 con 0x5563bdd3dea0 2023-10-16T14:35:28.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc2d790 con 0x5563bdd3f6a0 2023-10-16T14:35:28.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.487+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc2d430 con 0x5563bdd3e800 2023-10-16T14:35:28.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0f049dcb80 10 client.0 _ll_register_callbacks cb 0x5563bda8a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:28.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43816/0 (socket says 172.21.15.178:43816) 2023-10-16T14:35:28.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/1156444715 learned_addr learned my addr 172.21.15.178:0/1156444715 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd47ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0eead9d700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd4f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 msgr2=0x5563bdd4f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd4f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/1156444715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 msgr2=0x5563bdd47ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd47ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 -- 172.21.15.178:0/1156444715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee4003b60 con 0x5563bdd3e800 2023-10-16T14:35:28.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ee009a0d0 tx=0x7f0ee009a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d91ad0517a84143 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.488+0000 7f0ed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:35:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee009e290 con 0x5563bdd3e800 2023-10-16T14:35:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0ee00a24d0 con 0x5563bdd3e800 2023-10-16T14:35:28.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563bdc95f70 con 0x5563bdd3e800 2023-10-16T14:35:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee00a2040 con 0x5563bdd3e800 2023-10-16T14:35:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 10 client.25759 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5563bdabbe60 con 0x5563bdd3e800 2023-10-16T14:35:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5563bdc95f70 con 0x5563bdd3e800 2023-10-16T14:35:28.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0ee00990a0 con 0x5563bdd3e800 2023-10-16T14:35:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 10 client.25759 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 10 client.25759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 20 client.25759 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bdd53a00 0x5563bdd55e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 4 client.25759 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:28.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.489+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563bdd0dc90 con 0x5563bdd53a00 2023-10-16T14:35:28.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.490+0000 7f0eea59c700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bdd53a00 0x5563bdd55e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.490+0000 7f0ed1ffb700 10 client.25759 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.490+0000 7f0eea59c700 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bdd53a00 0x5563bdd55e20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bb30b49408b621 server_cookie=f62f7ac97592ecc in_seq=0 out_seq=0 2023-10-16T14:35:28.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/1156444715 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5563bdd0dc90 con 0x5563bdd53a00 2023-10-16T14:35:28.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0ed1ffb700 10 client.25759 handle_command_reply: tid=0 2023-10-16T14:35:28.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 client.25759 shutdown 2023-10-16T14:35:28.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 msgr2=0x5563bdd481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0ee009a0d0 tx=0x7f0ee009a680 comp rx=0 tx=0).stop 2023-10-16T14:35:28.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 shutdown_connections 2023-10-16T14:35:28.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bdd53a00 0x5563bdd55e20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd4f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0ed1ffb700 0 client.25759 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd47ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/1156444715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd481e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 >> 172.21.15.178:0/1156444715 conn(0x5563bdc6be50 msgr2=0x5563bdd38c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 shutdown_connections 2023-10-16T14:35:28.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.491+0000 7f0f049dcb80 1 -- 172.21.15.178:0/1156444715 wait complete. 2023-10-16T14:35:28.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.492+0000 7f0f049dcb80 20 client.25759 trim_cache size 0 max 16384 2023-10-16T14:35:28.511 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2023-10-16T14:35:28.512 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2023-10-16T14:35:28.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 Processor -- start 2023-10-16T14:35:28.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 -- start start 2023-10-16T14:35:28.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd4f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdb119a0 con 0x5563bdd3dea0 2023-10-16T14:35:28.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdc95f70 con 0x5563bdd3f6a0 2023-10-16T14:35:28.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdd3f520 con 0x5563bdd3e800 2023-10-16T14:35:28.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39558/0 (socket says 172.21.15.178:39558) 2023-10-16T14:35:28.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0eea59c700 1 -- 172.21.15.178:0/406048613 learned_addr learned my addr 172.21.15.178:0/406048613 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0eead9d700 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.511+0000 7f0ee9d9b700 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd4f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 -- 172.21.15.178:0/406048613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 msgr2=0x5563bdd4f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd4f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 -- 172.21.15.178:0/406048613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 msgr2=0x5563bdd42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 -- 172.21.15.178:0/406048613 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563bdaa4ce0 con 0x5563bdd3dea0 2023-10-16T14:35:28.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0eea59c700 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009810 tx=0x7f0edc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8f477b3f01a8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0ed97fa700 1 -- 172.21.15.178:0/406048613 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc034c70 con 0x5563bdd3dea0 2023-10-16T14:35:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0ed97fa700 1 -- 172.21.15.178:0/406048613 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0edc02b030 con 0x5563bdd3dea0 2023-10-16T14:35:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.512+0000 7f0ed97fa700 1 -- 172.21.15.178:0/406048613 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc005680 con 0x5563bdd3dea0 2023-10-16T14:35:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- 172.21.15.178:0/406048613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 msgr2=0x5563bdd42640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009810 tx=0x7f0edc021000 comp rx=0 tx=0).stop 2023-10-16T14:35:28.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- 172.21.15.178:0/406048613 shutdown_connections 2023-10-16T14:35:28.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdd42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3dea0 0x5563bdd42640 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/406048613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3e800 0x5563bdd4f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- 172.21.15.178:0/406048613 >> 172.21.15.178:0/406048613 conn(0x5563bdc6be50 msgr2=0x5563bdd38b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- 172.21.15.178:0/406048613 shutdown_connections 2023-10-16T14:35:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- 172.21.15.178:0/406048613 wait complete. 2023-10-16T14:35:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.513+0000 7f0f049dcb80 1 -- start start 2023-10-16T14:35:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 Processor -- start 2023-10-16T14:35:28.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd4e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd4ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdda8430 con 0x5563bdd3e800 2023-10-16T14:35:28.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdda8570 con 0x5563bdd3f6a0 2023-10-16T14:35:28.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563bdda8730 con 0x5563bdd3dea0 2023-10-16T14:35:28.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0f049dcb80 10 client.0 _ll_register_callbacks cb 0x7f0ee0099ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:28.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.514+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd4ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55714/0 (socket says 172.21.15.178:55714) 2023-10-16T14:35:28.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0ee9d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd4ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:39560/0 (socket says 172.21.15.178:39560) 2023-10-16T14:35:28.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 -- 172.21.15.178:0/2785514014 learned_addr learned my addr 172.21.15.178:0/2785514014 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eea59c700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd4e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 -- 172.21.15.178:0/2785514014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 msgr2=0x5563bdd4e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd4e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 -- 172.21.15.178:0/2785514014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 msgr2=0x5563bdd4ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd4ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.527 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 -- 172.21.15.178:0/2785514014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec10be20 con 0x5563bdd3f6a0 2023-10-16T14:35:28.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0eead9d700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5563bdd3ee80 tx=0x7f0ee4004c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5957779396c4fb server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.515+0000 7f0ed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:35:28.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4024040 con 0x5563bdd3f6a0 2023-10-16T14:35:28.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0ee400e6f0 con 0x5563bdd3f6a0 2023-10-16T14:35:28.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ee400f040 con 0x5563bdd3f6a0 2023-10-16T14:35:28.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ee400e8f0 con 0x5563bdd3f6a0 2023-10-16T14:35:28.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 10 client.25778 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:28.531 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5563bdae22d0 con 0x5563bdd3f6a0 2023-10-16T14:35:28.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ee400f040 con 0x5563bdd3f6a0 2023-10-16T14:35:28.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f0ee4007b10 con 0x5563bdd3f6a0 2023-10-16T14:35:28.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 10 client.25778 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:28.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:28.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 20 client.25778 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:28.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bddaa0a0 0x5563bddac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 4 client.25778 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:35:28.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.516+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5563bdc89ed0 con 0x5563bddaa0a0 2023-10-16T14:35:28.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.517+0000 7f0eea59c700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bddaa0a0 0x5563bddac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.517+0000 7f0eea59c700 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bddaa0a0 0x5563bddac4c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa8c3d270ffac1f server_cookie=1977bab7a12c55c in_seq=0 out_seq=0 2023-10-16T14:35:28.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.517+0000 7f0ed1ffb700 10 client.25778 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.518+0000 7f0ed1ffb700 1 -- 172.21.15.178:0/2785514014 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5563bdc89ed0 con 0x5563bddaa0a0 2023-10-16T14:35:28.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.518+0000 7f0ed1ffb700 10 client.25778 handle_command_reply: tid=0 2023-10-16T14:35:28.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.518+0000 7f0f049dcb80 1 client.25778 shutdown 2023-10-16T14:35:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.518+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 msgr2=0x5563bdda7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.518+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5563bdd3ee80 tx=0x7f0ee4004c40 comp rx=0 tx=0).stop 2023-10-16T14:35:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 shutdown_connections 2023-10-16T14:35:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5563bddaa0a0 0x5563bddac4c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.535 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5563bdd3f6a0 0x5563bdda7ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0ed1ffb700 0 client.25778 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563bdd3dea0 0x5563bdd4e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 --2- 172.21.15.178:0/2785514014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5563bdd3e800 0x5563bdd4ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 >> 172.21.15.178:0/2785514014 conn(0x5563bdc6be50 msgr2=0x5563bdd38940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 shutdown_connections 2023-10-16T14:35:28.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 1 -- 172.21.15.178:0/2785514014 wait complete. 2023-10-16T14:35:28.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0f049dcb80 20 client.25778 trim_cache size 0 max 16384 2023-10-16T14:35:28.537 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:35:28.538 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:35:28.538 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "7a10d381-1324-40b4-93b4-cb91f2db4555", 2023-10-16T14:35:28.538 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:35:28.538 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:35:28.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 msgr2=0x7f0ed40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 0x7f0ed40a39c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005e90 tx=0x7f0edc019040 comp rx=0 tx=0).stop 2023-10-16T14:35:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 msgr2=0x7f0eec199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.519+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0ee000e9a0 tx=0x7f0ee000ecb0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 shutdown_connections 2023-10-16T14:35:28.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f0ed40a1500 0x7f0ed40a39c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f0eec100ac0 0x7f0eec1a03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0eec0ff7a0 0x7f0eec198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 --2- 172.21.15.178:0/185116304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec100100 0x7f0eec199350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 >> 172.21.15.178:0/185116304 conn(0x7f0eec0fb320 msgr2=0x7f0eec0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 shutdown_connections 2023-10-16T14:35:28.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.520+0000 7f0ef0fff700 1 -- 172.21.15.178:0/185116304 wait complete. 2023-10-16T14:35:28.543 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:35:28.676 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2023-10-16T14:35:28.678 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2023-10-16T14:35:28.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 -- 172.21.15.178:0/3326737369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c1042b0 msgr2=0x7fb26c1066a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 --2- 172.21.15.178:0/3326737369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c1042b0 0x7fb26c1066a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb260009990 tx=0x7fb260009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 -- 172.21.15.178:0/3326737369 shutdown_connections 2023-10-16T14:35:28.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 --2- 172.21.15.178:0/3326737369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c106be0 0x7fb26c109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 --2- 172.21.15.178:0/3326737369 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c101980 0x7fb26c103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 --2- 172.21.15.178:0/3326737369 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c1042b0 0x7fb26c1066a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 -- 172.21.15.178:0/3326737369 >> 172.21.15.178:0/3326737369 conn(0x7fb26c0fb370 msgr2=0x7fb26c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 -- 172.21.15.178:0/3326737369 shutdown_connections 2023-10-16T14:35:28.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.801+0000 7fb2725ff700 1 -- 172.21.15.178:0/3326737369 wait complete. 2023-10-16T14:35:28.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 Processor -- start 2023-10-16T14:35:28.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 -- start start 2023-10-16T14:35:28.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c101980 0x7fb26c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 0x7fb26c19e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19e730 con 0x7fb26c106be0 2023-10-16T14:35:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19e870 con 0x7fb26c101980 2023-10-16T14:35:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.802+0000 7fb2725ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19ea00 con 0x7fb26c1042b0 2023-10-16T14:35:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb270b9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 0x7fb26c19e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:43826/0 (socket says 172.21.15.178:43826) 2023-10-16T14:35:28.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 -- 172.21.15.178:0/536535056 learned_addr learned my addr 172.21.15.178:0/536535056 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26bfff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c101980 0x7fb26c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 -- 172.21.15.178:0/536535056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c101980 msgr2=0x7fb26c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c101980 0x7fb26c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 -- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 msgr2=0x7fb26c19e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 0x7fb26c19e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 -- 172.21.15.178:0/536535056 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600095f0 con 0x7fb26c1042b0 2023-10-16T14:35:28.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb270b9c700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 0x7fb26c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:28.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.803+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb260004740 tx=0x7fb260009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.804+0000 7fb2697fa700 1 -- 172.21.15.178:0/536535056 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26001d070 con 0x7fb26c1042b0 2023-10-16T14:35:28.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.804+0000 7fb2697fa700 1 -- 172.21.15.178:0/536535056 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb260004830 con 0x7fb26c1042b0 2023-10-16T14:35:28.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.804+0000 7fb2697fa700 1 -- 172.21.15.178:0/536535056 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2600053c0 con 0x7fb26c1042b0 2023-10-16T14:35:28.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.804+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c19ec80 con 0x7fb26c1042b0 2023-10-16T14:35:28.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.804+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c19f040 con 0x7fb26c1042b0 2023-10-16T14:35:28.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.807+0000 7fb2697fa700 1 -- 172.21.15.178:0/536535056 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb260005580 con 0x7fb26c1042b0 2023-10-16T14:35:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.808+0000 7fb2697fa700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 0x7fb2540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.808+0000 7fb285f7cb80 1 Processor -- start 2023-10-16T14:35:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb2697fa700 1 -- 172.21.15.178:0/536535056 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb2600ce1e0 con 0x7fb26c1042b0 2023-10-16T14:35:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 -- start start 2023-10-16T14:35:28.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb26bfff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 0x7fb2540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.818 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2023-10-16T14:35:28.818 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2023-10-16T14:35:28.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396732b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ba30 0x56396732beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396729ac70 con 0x56396732ba30 2023-10-16T14:35:28.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396727fc60 con 0x56396732a230 2023-10-16T14:35:28.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396727d580 con 0x56396732ab90 2023-10-16T14:35:28.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55728/0 (socket says 172.21.15.178:55728) 2023-10-16T14:35:28.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb26bfff700 1 -- 172.21.15.178:0/1013819784 learned_addr learned my addr 172.21.15.178:0/1013819784 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.809+0000 7fb270b9c700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ba30 0x56396732beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 0x7fb2540a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009cb0 tx=0x7fb25c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396732b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 -- 172.21.15.178:0/1013819784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 msgr2=0x56396732b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396732b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 -- 172.21.15.178:0/1013819784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ba30 msgr2=0x56396732beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ba30 0x56396732beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 -- 172.21.15.178:0/1013819784 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639672363f0 con 0x56396732a230 2023-10-16T14:35:28.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396732b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:28.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009fd0 tx=0x7fb25c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec02a747946a6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb2527fc700 1 -- 172.21.15.178:0/1013819784 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c01ce30 con 0x56396732a230 2023-10-16T14:35:28.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb2527fc700 1 -- 172.21.15.178:0/1013819784 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25c004cf0 con 0x56396732a230 2023-10-16T14:35:28.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.810+0000 7fb2527fc700 1 -- 172.21.15.178:0/1013819784 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c024680 con 0x56396732a230 2023-10-16T14:35:28.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1013819784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 msgr2=0x56396732a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009fd0 tx=0x7fb25c01c000 comp rx=0 tx=0).stop 2023-10-16T14:35:28.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1013819784 shutdown_connections 2023-10-16T14:35:28.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ba30 0x56396732beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x56396732a650 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1013819784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396732b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1013819784 >> 172.21.15.178:0/1013819784 conn(0x563967253dd0 msgr2=0x563967320fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1013819784 shutdown_connections 2023-10-16T14:35:28.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.811+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1013819784 wait complete. 2023-10-16T14:35:28.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 -- start start 2023-10-16T14:35:28.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 Processor -- start 2023-10-16T14:35:28.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x5639673345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ba30 0x56396733b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967278e70 con 0x56396732ab90 2023-10-16T14:35:28.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967215710 con 0x56396732a230 2023-10-16T14:35:28.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.812+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639672153b0 con 0x56396732ba30 2023-10-16T14:35:28.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb285f7cb80 10 client.0 _ll_register_callbacks cb 0x563967072800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:28.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:55738/0 (socket says 172.21.15.178:55738) 2023-10-16T14:35:28.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 -- 172.21.15.178:0/2106607961 learned_addr learned my addr 172.21.15.178:0/2106607961 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x5639673345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb270b9c700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ba30 0x56396733b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 -- 172.21.15.178:0/2106607961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ba30 msgr2=0x56396733b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ba30 0x56396733b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 -- 172.21.15.178:0/2106607961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 msgr2=0x5639673345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x5639673345e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.813+0000 7fb26bfff700 1 -- 172.21.15.178:0/2106607961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c017050 con 0x56396732a230 2023-10-16T14:35:28.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb26bfff700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005920 tx=0x7fb25c004ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2dd27493f0a6a59 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb24b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:35:28.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0052d0 con 0x56396732a230 2023-10-16T14:35:28.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56396727def0 con 0x56396732a230 2023-10-16T14:35:28.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb25c003680 con 0x56396732a230 2023-10-16T14:35:28.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25c003840 con 0x56396732a230 2023-10-16T14:35:28.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb285f7cb80 10 client.25790 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:28.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639670a3de0 con 0x56396732a230 2023-10-16T14:35:28.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.814+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56396727def0 con 0x56396732a230 2023-10-16T14:35:28.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb25c02db90 con 0x56396732a230 2023-10-16T14:35:28.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 10 client.25790 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:28.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:28.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 20 client.25790 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:28.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56396733fea0 0x5639673422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 4 client.25790 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:28.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639672f5c10 con 0x56396733fea0 2023-10-16T14:35:28.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56396733fea0 0x5639673422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56396733fea0 0x5639673422c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8573bd96cdb6a3e server_cookie=3519d94f93ad393f in_seq=0 out_seq=0 2023-10-16T14:35:28.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.815+0000 7fb24b7fe700 10 client.25790 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb24b7fe700 1 -- 172.21.15.178:0/2106607961 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5639672f5c10 con 0x56396733fea0 2023-10-16T14:35:28.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb24b7fe700 10 client.25790 handle_command_reply: tid=0 2023-10-16T14:35:28.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 client.25790 shutdown 2023-10-16T14:35:28.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 msgr2=0x5639673340a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005920 tx=0x7fb25c004ee0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 shutdown_connections 2023-10-16T14:35:28.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56396733fea0 0x5639673422c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ba30 0x56396733b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb24b7fe700 0 client.25790 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732a230 0x5639673340a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/2106607961 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x5639673345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.816+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 >> 172.21.15.178:0/2106607961 conn(0x563967253dd0 msgr2=0x563967320bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.817+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 shutdown_connections 2023-10-16T14:35:28.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.817+0000 7fb285f7cb80 1 -- 172.21.15.178:0/2106607961 wait complete. 2023-10-16T14:35:28.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.817+0000 7fb285f7cb80 20 client.25790 trim_cache size 0 max 16384 2023-10-16T14:35:28.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 Processor -- start 2023-10-16T14:35:28.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 -- start start 2023-10-16T14:35:28.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 0x563967331e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x56396733b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x56396732e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967333c70 con 0x56396732ab90 2023-10-16T14:35:28.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639670f9920 con 0x56396732ba30 2023-10-16T14:35:28.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967332bb0 con 0x56396732a230 2023-10-16T14:35:28.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb270b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x56396732e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 0x563967331e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb270b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x56396732e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:35794/0 (socket says 172.21.15.178:35794) 2023-10-16T14:35:28.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.836+0000 7fb270b9c700 1 -- 172.21.15.178:0/3861966839 learned_addr learned my addr 172.21.15.178:0/3861966839 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x56396733b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.852 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 -- 172.21.15.178:0/3861966839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 msgr2=0x56396732e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x56396732e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 -- 172.21.15.178:0/3861966839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 msgr2=0x56396733b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x56396733b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 -- 172.21.15.178:0/3861966839 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56396708cc60 con 0x56396732a230 2023-10-16T14:35:28.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb26bfff700 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 0x563967331e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb25c02def0 tx=0x7fb25c009aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb55f79b24b2078 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb252ffd700 1 -- 172.21.15.178:0/3861966839 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c024790 con 0x56396732a230 2023-10-16T14:35:28.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb252ffd700 1 -- 172.21.15.178:0/3861966839 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25c024e80 con 0x56396732a230 2023-10-16T14:35:28.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.837+0000 7fb252ffd700 1 -- 172.21.15.178:0/3861966839 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003680 con 0x56396732a230 2023-10-16T14:35:28.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 -- 172.21.15.178:0/3861966839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 msgr2=0x563967331e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 0x563967331e70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb25c02def0 tx=0x7fb25c009aa0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 -- 172.21.15.178:0/3861966839 shutdown_connections 2023-10-16T14:35:28.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x56396732e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732a230 0x563967331e70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/3861966839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732ab90 0x56396733b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 -- 172.21.15.178:0/3861966839 >> 172.21.15.178:0/3861966839 conn(0x563967253dd0 msgr2=0x563967320240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 -- 172.21.15.178:0/3861966839 shutdown_connections 2023-10-16T14:35:28.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.838+0000 7fb285f7cb80 1 -- 172.21.15.178:0/3861966839 wait complete. 2023-10-16T14:35:28.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 -- start start 2023-10-16T14:35:28.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 Processor -- start 2023-10-16T14:35:28.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396733b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x563967339610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967339cf0 con 0x56396732a230 2023-10-16T14:35:28.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563967339e60 con 0x56396732ba30 2023-10-16T14:35:28.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56396733a020 con 0x56396732ab90 2023-10-16T14:35:28.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb285f7cb80 10 client.0 _ll_register_callbacks cb 0x7fb264009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:28.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:51170/0 (socket says 172.21.15.178:51170) 2023-10-16T14:35:28.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb26bfff700 1 -- 172.21.15.178:0/1136662679 learned_addr learned my addr 172.21.15.178:0/1136662679 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:28.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb26b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396733b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.839+0000 7fb270b9c700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x563967339610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 -- 172.21.15.178:0/1136662679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 msgr2=0x56396733b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396733b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 -- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 msgr2=0x563967339610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:28.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x563967339610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 -- 172.21.15.178:0/1136662679 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639670a3de0 con 0x56396732a230 2023-10-16T14:35:28.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb270b9c700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x563967339610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:28.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb26bfff700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005a80 tx=0x7fb25c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08833d219e6228f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:28.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb24b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:28.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c003880 con 0x56396732a230 2023-10-16T14:35:28.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb25c005580 con 0x56396732a230 2023-10-16T14:35:28.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.840+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563967271e50 con 0x56396732a230 2023-10-16T14:35:28.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb25c0039e0 con 0x56396732a230 2023-10-16T14:35:28.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 10 client.15750 fetch_fsmap learned FSMap version 28 2023-10-16T14:35:28.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5639670ca250 con 0x56396732a230 2023-10-16T14:35:28.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563967271e50 con 0x56396732a230 2023-10-16T14:35:28.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fb25c024d10 con 0x56396732a230 2023-10-16T14:35:28.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2023-10-16T14:35:28.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:28.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 20 client.15750 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:28.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563967394bd0 0x563967396fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:28.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 4 client.15750 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:35:28.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563967271b70 con 0x563967394bd0 2023-10-16T14:35:28.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.841+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563967394bd0 0x563967396fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:28.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb26b7fe700 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563967394bd0 0x563967396fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3209e2cd877825 server_cookie=519660e0090bfb09 in_seq=0 out_seq=0 2023-10-16T14:35:28.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb24b7fe700 10 client.15750 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb24b7fe700 1 -- 172.21.15.178:0/1136662679 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563967271b70 con 0x563967394bd0 2023-10-16T14:35:28.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb24b7fe700 10 client.15750 handle_command_reply: tid=0 2023-10-16T14:35:28.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 client.15750 shutdown 2023-10-16T14:35:28.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 msgr2=0x56396733af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb25c005a80 tx=0x7fb25c011b30 comp rx=0 tx=0).stop 2023-10-16T14:35:28.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 shutdown_connections 2023-10-16T14:35:28.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563967394bd0 0x563967396fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56396732ba30 0x563967339610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56396732a230 0x56396733af20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 --2- 172.21.15.178:0/1136662679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56396732ab90 0x56396733b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb24b7fe700 0 client.15750 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:28.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 >> 172.21.15.178:0/1136662679 conn(0x563967253dd0 msgr2=0x56396732e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 shutdown_connections 2023-10-16T14:35:28.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.842+0000 7fb285f7cb80 1 -- 172.21.15.178:0/1136662679 wait complete. 2023-10-16T14:35:28.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb285f7cb80 20 client.15750 trim_cache size 0 max 16384 2023-10-16T14:35:28.869 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:35:28.869 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-16T14:35:28.869 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:35:28.869 INFO:teuthology.orchestra.run.smithi178.stdout: "7a10d381-1324-40b4-93b4-cb91f2db4555": { 2023-10-16T14:35:28.870 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:35:28.870 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "7a10d381-1324-40b4-93b4-cb91f2db4555", 2023-10-16T14:35:28.870 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:35:28.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:35:28.870 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:35:28.871 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:35:28.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 msgr2=0x7fb2540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 0x7fb2540a39c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009cb0 tx=0x7fb25c009260 comp rx=0 tx=0).stop 2023-10-16T14:35:28.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 msgr2=0x7fb26c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:28.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb260004740 tx=0x7fb260009580 comp rx=0 tx=0).stop 2023-10-16T14:35:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 shutdown_connections 2023-10-16T14:35:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb2540a1500 0x7fb2540a39c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb26c106be0 0x7fb26c19e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb26c101980 0x7fb26c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 --2- 172.21.15.178:0/536535056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb26c1042b0 0x7fb26c1970f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 >> 172.21.15.178:0/536535056 conn(0x7fb26c0fb370 msgr2=0x7fb26c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 shutdown_connections 2023-10-16T14:35:28.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:28.843+0000 7fb2725ff700 1 -- 172.21.15.178:0/536535056 wait complete. 2023-10-16T14:35:28.874 INFO:tasks.cephfs.filesystem:scrub status for tag:7a10d381-1324-40b4-93b4-cb91f2db4555 - {'path': '/', 'tag': '7a10d381-1324-40b4-93b4-cb91f2db4555', 'options': 'recursive,force'} 2023-10-16T14:35:28.918 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2023-10-16T14:35:28.918 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2023-10-16T14:35:29.009 INFO:tasks.workunit.client.0.smithi178.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -pthread -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2023-10-16T14:35:29.009 INFO:tasks.workunit.client.0.smithi178.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2023-10-16T14:35:29.095 INFO:tasks.workunit.client.0.smithi178.stdout:gcc -g -O2 -pthread -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2023-10-16T14:35:29.134 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2023-10-16T14:35:29.135 INFO:tasks.workunit.client.0.smithi178.stdout:Making all in man 2023-10-16T14:35:29.139 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-16T14:35:29.139 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Nothing to be done for 'all'. 2023-10-16T14:35:29.139 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2023-10-16T14:35:29.142 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-16T14:35:29.145 INFO:tasks.workunit.client.0.smithi178.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-16T14:35:29.145 INFO:tasks.workunit.client.0.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2023-10-16T14:35:29.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ cd src 2023-10-16T14:35:29.146 INFO:tasks.workunit.client.0.smithi178.stderr:+ mkdir blogtest_in 2023-10-16T14:35:29.152 INFO:tasks.workunit.client.0.smithi178.stdout:running blogbench 2023-10-16T14:35:29.152 INFO:tasks.workunit.client.0.smithi178.stderr:+ echo 'running blogbench' 2023-10-16T14:35:29.152 INFO:tasks.workunit.client.0.smithi178.stderr:+ ./blogbench -d blogtest_in 2023-10-16T14:35:29.153 INFO:tasks.workunit.client.0.smithi178.stdout: 2023-10-16T14:35:29.153 INFO:tasks.workunit.client.0.smithi178.stdout:Frequency = 10 secs 2023-10-16T14:35:29.154 INFO:tasks.workunit.client.0.smithi178.stdout:Scratch dir = [blogtest_in] 2023-10-16T14:35:30.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:35:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:35:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:35:30.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:29 smithi178 ceph-mon[98495]: pgmap v392: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-10-16T14:35:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:35:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:35:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:35:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:29 smithi186 ceph-mon[134892]: pgmap v392: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-10-16T14:35:30.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:35:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:35:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:35:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:35:30.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:29 smithi187 ceph-mon[135297]: pgmap v392: 129 pgs: 129 active+clean; 4.4 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 121 KiB/s wr, 31 op/s 2023-10-16T14:35:30.962 INFO:tasks.workunit.client.0.smithi178.stdout:Spawning 3 writers... 2023-10-16T14:35:30.963 INFO:tasks.workunit.client.0.smithi178.stdout:Spawning 1 rewriters... 2023-10-16T14:35:30.963 INFO:tasks.workunit.client.0.smithi178.stdout:Spawning 5 commenters... 2023-10-16T14:35:30.964 INFO:tasks.workunit.client.0.smithi178.stdout:Spawning 100 readers... 2023-10-16T14:35:30.971 INFO:tasks.workunit.client.0.smithi178.stdout:Benchmarking for 30 iterations. 2023-10-16T14:35:30.971 INFO:tasks.workunit.client.0.smithi178.stdout:The test will run during 5 minutes. 2023-10-16T14:35:30.971 INFO:tasks.workunit.client.0.smithi178.stdout: 2023-10-16T14:35:30.972 INFO:tasks.workunit.client.0.smithi178.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2023-10-16T14:35:31.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:31 smithi178 ceph-mon[98495]: pgmap v393: 129 pgs: 129 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 144 KiB/s wr, 36 op/s 2023-10-16T14:35:31.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:31 smithi186 ceph-mon[134892]: pgmap v393: 129 pgs: 129 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 144 KiB/s wr, 36 op/s 2023-10-16T14:35:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:31 smithi187 ceph-mon[135297]: pgmap v393: 129 pgs: 129 active+clean; 4.6 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 144 KiB/s wr, 36 op/s 2023-10-16T14:35:32.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:32 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:32.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:32 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:32.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:32 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:33.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:33 smithi178 ceph-mon[98495]: pgmap v394: 129 pgs: 129 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-10-16T14:35:33.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:33 smithi186 ceph-mon[134892]: pgmap v394: 129 pgs: 129 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-10-16T14:35:33.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:33 smithi187 ceph-mon[135297]: pgmap v394: 129 pgs: 129 active+clean; 4.8 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 136 KiB/s wr, 36 op/s 2023-10-16T14:35:36.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:35 smithi178 ceph-mon[98495]: pgmap v395: 129 pgs: 129 active+clean; 8.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 414 KiB/s wr, 40 op/s 2023-10-16T14:35:36.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:35 smithi186 ceph-mon[134892]: pgmap v395: 129 pgs: 129 active+clean; 8.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 414 KiB/s wr, 40 op/s 2023-10-16T14:35:36.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:35 smithi187 ceph-mon[135297]: pgmap v395: 129 pgs: 129 active+clean; 8.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 414 KiB/s wr, 40 op/s 2023-10-16T14:35:38.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:37 smithi178 ceph-mon[98495]: pgmap v396: 129 pgs: 129 active+clean; 14 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1008 KiB/s wr, 55 op/s 2023-10-16T14:35:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:37 smithi186 ceph-mon[134892]: pgmap v396: 129 pgs: 129 active+clean; 14 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1008 KiB/s wr, 55 op/s 2023-10-16T14:35:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:37 smithi187 ceph-mon[135297]: pgmap v396: 129 pgs: 129 active+clean; 14 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1008 KiB/s wr, 55 op/s 2023-10-16T14:35:40.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:39 smithi178 ceph-mon[98495]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2023-10-16T14:35:40.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:39 smithi186 ceph-mon[134892]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2023-10-16T14:35:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:39 smithi187 ceph-mon[135297]: pgmap v397: 129 pgs: 129 active+clean; 17 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 1.1 MiB/s wr, 50 op/s 2023-10-16T14:35:40.971 INFO:tasks.workunit.client.0.smithi178.stdout: 15 293456 926 200544 892 112325 1378 2023-10-16T14:35:41.560 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:41 smithi178 ceph-mon[98495]: pgmap v398: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 114 op/s 2023-10-16T14:35:41.560 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:41 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:35:41.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:41 smithi186 ceph-mon[134892]: pgmap v398: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 114 op/s 2023-10-16T14:35:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:41 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:35:41.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:41 smithi187 ceph-mon[135297]: pgmap v398: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 114 op/s 2023-10-16T14:35:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:41 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:35:42.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:42 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:35:42.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:42 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:35:42.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:42 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:35:43.559 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:43 smithi178 ceph-mon[98495]: pgmap v399: 129 pgs: 129 active+clean; 39 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 143 op/s 2023-10-16T14:35:43.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:43 smithi186 ceph-mon[134892]: pgmap v399: 129 pgs: 129 active+clean; 39 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 143 op/s 2023-10-16T14:35:43.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:43 smithi187 ceph-mon[135297]: pgmap v399: 129 pgs: 129 active+clean; 39 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 143 op/s 2023-10-16T14:35:46.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:45 smithi178 ceph-mon[98495]: pgmap v400: 129 pgs: 129 active+clean; 52 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 178 op/s 2023-10-16T14:35:46.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:45 smithi186 ceph-mon[134892]: pgmap v400: 129 pgs: 129 active+clean; 52 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 178 op/s 2023-10-16T14:35:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:45 smithi187 ceph-mon[135297]: pgmap v400: 129 pgs: 129 active+clean; 52 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 178 op/s 2023-10-16T14:35:48.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:47 smithi178 ceph-mon[98495]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 229 op/s 2023-10-16T14:35:48.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:47 smithi186 ceph-mon[134892]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 229 op/s 2023-10-16T14:35:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:47 smithi187 ceph-mon[135297]: pgmap v401: 129 pgs: 129 active+clean; 64 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 229 op/s 2023-10-16T14:35:50.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:49 smithi178 ceph-mon[98495]: pgmap v402: 129 pgs: 129 active+clean; 65 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-16T14:35:50.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:49 smithi186 ceph-mon[134892]: pgmap v402: 129 pgs: 129 active+clean; 65 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-16T14:35:50.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:49 smithi187 ceph-mon[135297]: pgmap v402: 129 pgs: 129 active+clean; 65 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 226 op/s 2023-10-16T14:35:50.973 INFO:tasks.workunit.client.0.smithi178.stdout: 27 349766 640 237818 762 147024 1134 2023-10-16T14:35:51.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:51 smithi178 ceph-mon[98495]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 238 op/s 2023-10-16T14:35:51.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:51 smithi186 ceph-mon[134892]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 238 op/s 2023-10-16T14:35:51.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:51 smithi187 ceph-mon[135297]: pgmap v403: 129 pgs: 129 active+clean; 70 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 238 op/s 2023-10-16T14:35:52.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:52 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:52 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:52.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:52 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:35:53.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:53 smithi178 ceph-mon[98495]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 179 op/s 2023-10-16T14:35:53.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:53 smithi186 ceph-mon[134892]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 179 op/s 2023-10-16T14:35:53.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:53 smithi187 ceph-mon[135297]: pgmap v404: 129 pgs: 129 active+clean; 70 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 179 op/s 2023-10-16T14:35:56.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:55 smithi178 ceph-mon[98495]: pgmap v405: 129 pgs: 129 active+clean; 76 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2023-10-16T14:35:56.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:55 smithi186 ceph-mon[134892]: pgmap v405: 129 pgs: 129 active+clean; 76 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2023-10-16T14:35:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:55 smithi187 ceph-mon[135297]: pgmap v405: 129 pgs: 129 active+clean; 76 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 159 op/s 2023-10-16T14:35:58.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:57 smithi178 ceph-mon[98495]: pgmap v406: 129 pgs: 129 active+clean; 77 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-10-16T14:35:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:57 smithi186 ceph-mon[134892]: pgmap v406: 129 pgs: 129 active+clean; 77 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-10-16T14:35:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:57 smithi187 ceph-mon[135297]: pgmap v406: 129 pgs: 129 active+clean; 77 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 138 op/s 2023-10-16T14:35:58.875 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:35:59.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 -- 172.21.15.178:0/1714663626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 msgr2=0x7f9a2c0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 --2- 172.21.15.178:0/1714663626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c0732e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009990 tx=0x7f9a28009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 -- 172.21.15.178:0/1714663626 shutdown_connections 2023-10-16T14:35:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 --2- 172.21.15.178:0/1714663626 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c108d90 0x7f9a2c10b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 --2- 172.21.15.178:0/1714663626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c0732e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 --2- 172.21.15.178:0/1714663626 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.137+0000 7f9a33bff700 1 -- 172.21.15.178:0/1714663626 >> 172.21.15.178:0/1714663626 conn(0x7f9a2c0fc3f0 msgr2=0x7f9a2c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.138+0000 7f9a33bff700 1 -- 172.21.15.178:0/1714663626 shutdown_connections 2023-10-16T14:35:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.138+0000 7f9a33bff700 1 -- 172.21.15.178:0/1714663626 wait complete. 2023-10-16T14:35:59.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 Processor -- start 2023-10-16T14:35:59.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 -- start start 2023-10-16T14:35:59.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 0x7f9a2c1a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a4d60 con 0x7f9a2c108d90 2023-10-16T14:35:59.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a4ea0 con 0x7f9a2c0738b0 2023-10-16T14:35:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a33bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2c1a5030 con 0x7f9a2c074e90 2023-10-16T14:35:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46516/0 (socket says 172.21.15.178:46516) 2023-10-16T14:35:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a3199b700 1 -- 172.21.15.178:0/2675236612 learned_addr learned my addr 172.21.15.178:0/2675236612 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.139+0000 7f9a3119a700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3219c700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 0x7f9a2c1a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 -- 172.21.15.178:0/2675236612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 msgr2=0x7f9a2c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 -- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 msgr2=0x7f9a2c1a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 0x7f9a2c1a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 -- 172.21.15.178:0/2675236612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a280095f0 con 0x7f9a2c0738b0 2023-10-16T14:35:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3219c700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 0x7f9a2c1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:35:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3119a700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a3199b700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a28004740 tx=0x7f9a28009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a22ffd700 1 -- 172.21.15.178:0/2675236612 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2801d070 con 0x7f9a2c0738b0 2023-10-16T14:35:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a22ffd700 1 -- 172.21.15.178:0/2675236612 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a28004830 con 0x7f9a2c0738b0 2023-10-16T14:35:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.140+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2c1a52b0 con 0x7f9a2c0738b0 2023-10-16T14:35:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.141+0000 7f9a22ffd700 1 -- 172.21.15.178:0/2675236612 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28005450 con 0x7f9a2c0738b0 2023-10-16T14:35:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.141+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c1a56d0 con 0x7f9a2c0738b0 2023-10-16T14:35:59.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.145+0000 7f9a22ffd700 1 -- 172.21.15.178:0/2675236612 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f9a2800fb00 con 0x7f9a2c0738b0 2023-10-16T14:35:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.146+0000 7f9a474a1b80 1 Processor -- start 2023-10-16T14:35:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.146+0000 7f9a474a1b80 1 -- start start 2023-10-16T14:35:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a22ffd700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 0x7f9a180a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7d7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d74ac70 con 0x55e98d7d6780 2023-10-16T14:35:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d72fc60 con 0x55e98d7d7620 2023-10-16T14:35:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d72d580 con 0x55e98d7d5e20 2023-10-16T14:35:59.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a22ffd700 1 -- 172.21.15.178:0/2675236612 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9a280ce230 con 0x7f9a2c0738b0 2023-10-16T14:35:59.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3219c700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 0x7f9a180a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3119a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52498/0 (socket says 172.21.15.178:52498) 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 -- 172.21.15.178:0/1194482030 learned_addr learned my addr 172.21.15.178:0/1194482030 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3219c700 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7d7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 -- 172.21.15.178:0/1194482030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 msgr2=0x55e98d7d7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7d7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 -- 172.21.15.178:0/1194482030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 msgr2=0x55e98d7d7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7d7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.147+0000 7f9a3199b700 1 -- 172.21.15.178:0/1194482030 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e98d6e63f0 con 0x55e98d7d5e20 2023-10-16T14:35:59.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a3219c700 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:59.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a28097770 tx=0x7f9a2809aee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3a13294e97eaa1c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a3219c700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 0x7f9a180a39c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009fd0 tx=0x7f9a24009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a17fff700 1 -- 172.21.15.178:0/1194482030 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2809b270 con 0x55e98d7d5e20 2023-10-16T14:35:59.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a17fff700 1 -- 172.21.15.178:0/1194482030 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2809f3c0 con 0x55e98d7d5e20 2023-10-16T14:35:59.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.148+0000 7f9a17fff700 1 -- 172.21.15.178:0/1194482030 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2809b730 con 0x55e98d7d5e20 2023-10-16T14:35:59.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1194482030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 msgr2=0x55e98d7d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a28097770 tx=0x7f9a2809aee0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1194482030 shutdown_connections 2023-10-16T14:35:59.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7d7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1194482030 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7d7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1194482030 >> 172.21.15.178:0/1194482030 conn(0x55e98d703dd0 msgr2=0x55e98d7d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1194482030 shutdown_connections 2023-10-16T14:35:59.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1194482030 wait complete. 2023-10-16T14:35:59.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.149+0000 7f9a474a1b80 1 -- start start 2023-10-16T14:35:59.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 Processor -- start 2023-10-16T14:35:59.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d728e70 con 0x55e98d7d6780 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d6c5710 con 0x55e98d7d7620 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d6c53b0 con 0x55e98d7d5e20 2023-10-16T14:35:59.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.150+0000 7f9a474a1b80 10 client.0 _ll_register_callbacks cb 0x55e98d522800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:59.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.151+0000 7f9a3219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.151+0000 7f9a3219c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46526/0 (socket says 172.21.15.178:46526) 2023-10-16T14:35:59.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.151+0000 7f9a3219c700 1 -- 172.21.15.178:0/3437467227 learned_addr learned my addr 172.21.15.178:0/3437467227 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.151+0000 7f9a3119a700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.152+0000 7f9a3119a700 1 -- 172.21.15.178:0/3437467227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 msgr2=0x55e98d7df960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:35:59.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.152+0000 7f9a3119a700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7df960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a3119a700 1 -- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 msgr2=0x55e98d7e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a3119a700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7e6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a3119a700 1 -- 172.21.15.178:0/3437467227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2c1a56d0 con 0x55e98d7d6780 2023-10-16T14:35:59.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a3119a700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7dfea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c012cd0 tx=0x7f9a1c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a05fed0172281a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a157fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:35:59.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c017070 con 0x55e98d7d6780 2023-10-16T14:35:59.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9a1c009e00 con 0x55e98d7d6780 2023-10-16T14:35:59.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e98d72def0 con 0x55e98d7d6780 2023-10-16T14:35:59.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1c00fc30 con 0x55e98d7d6780 2023-10-16T14:35:59.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a474a1b80 10 client.15759 fetch_fsmap learned FSMap version 30 2023-10-16T14:35:59.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.153+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e98d553de0 con 0x55e98d7d6780 2023-10-16T14:35:59.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e98d72def0 con 0x55e98d7d6780 2023-10-16T14:35:59.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9a1c003a80 con 0x55e98d7d6780 2023-10-16T14:35:59.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 10 client.15759 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 10 client.15759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 20 client.15759 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d7eb7b0 0x55e98d7edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 4 client.15759 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.154+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e98d7a5c10 con 0x55e98d7eb7b0 2023-10-16T14:35:59.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.155+0000 7f9a3199b700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d7eb7b0 0x55e98d7edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.155+0000 7f9a3199b700 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d7eb7b0 0x55e98d7edbd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18f07e46464a6d0 server_cookie=30c3d3654dc8c129 in_seq=0 out_seq=0 2023-10-16T14:35:59.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.155+0000 7f9a157fa700 10 client.15759 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:59.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a157fa700 1 -- 172.21.15.178:0/3437467227 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e98d7a5c10 con 0x55e98d7eb7b0 2023-10-16T14:35:59.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a157fa700 10 client.15759 handle_command_reply: tid=0 2023-10-16T14:35:59.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 client.15759 shutdown 2023-10-16T14:35:59.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 msgr2=0x55e98d7dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7dfea0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c012cd0 tx=0x7f9a1c00f6a0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 shutdown_connections 2023-10-16T14:35:59.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d7eb7b0 0x55e98d7edbd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d7620 0x55e98d7e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d5e20 0x55e98d7df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/3437467227 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d6780 0x55e98d7dfea0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 >> 172.21.15.178:0/3437467227 conn(0x55e98d703dd0 msgr2=0x55e98d7d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.158+0000 7f9a157fa700 0 client.15759 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:59.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.159+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 shutdown_connections 2023-10-16T14:35:59.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.159+0000 7f9a474a1b80 1 -- 172.21.15.178:0/3437467227 wait complete. 2023-10-16T14:35:59.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.159+0000 7f9a474a1b80 20 client.15759 trim_cache size 0 max 16384 2023-10-16T14:35:59.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 Processor -- start 2023-10-16T14:35:59.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 -- start start 2023-10-16T14:35:59.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d7620 0x55e98d7e77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d5a9920 con 0x55e98d7d5e20 2023-10-16T14:35:59.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d7d2f80 con 0x55e98d7d6780 2023-10-16T14:35:59.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d7dd7a0 con 0x55e98d7d7620 2023-10-16T14:35:59.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a3119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a3119a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46528/0 (socket says 172.21.15.178:46528) 2023-10-16T14:35:59.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a3119a700 1 -- 172.21.15.178:0/125779170 learned_addr learned my addr 172.21.15.178:0/125779170 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.179+0000 7f9a3219c700 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d7620 0x55e98d7e77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 -- 172.21.15.178:0/125779170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d7620 msgr2=0x55e98d7e77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d7620 0x55e98d7e77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 -- 172.21.15.178:0/125779170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 msgr2=0x55e98d7d6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 -- 172.21.15.178:0/125779170 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e98d53cc60 con 0x55e98d7d6780 2023-10-16T14:35:59.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3199b700 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:35:59.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a3119a700 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009340 tx=0x7f9a1c012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5dbf844c1a98f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a20bff700 1 -- 172.21.15.178:0/125779170 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c017040 con 0x55e98d7d6780 2023-10-16T14:35:59.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a20bff700 1 -- 172.21.15.178:0/125779170 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a1c010530 con 0x55e98d7d6780 2023-10-16T14:35:59.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.180+0000 7f9a20bff700 1 -- 172.21.15.178:0/125779170 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c010b50 con 0x55e98d7d6780 2023-10-16T14:35:59.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 -- 172.21.15.178:0/125779170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 msgr2=0x55e98d7e6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009340 tx=0x7f9a1c012320 comp rx=0 tx=0).stop 2023-10-16T14:35:59.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 -- 172.21.15.178:0/125779170 shutdown_connections 2023-10-16T14:35:59.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d7620 0x55e98d7e77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d5e20 0x55e98d7d6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/125779170 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d6780 0x55e98d7e6ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 -- 172.21.15.178:0/125779170 >> 172.21.15.178:0/125779170 conn(0x55e98d703dd0 msgr2=0x55e98d7de690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 -- 172.21.15.178:0/125779170 shutdown_connections 2023-10-16T14:35:59.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.181+0000 7f9a474a1b80 1 -- 172.21.15.178:0/125779170 wait complete. 2023-10-16T14:35:59.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.182+0000 7f9a474a1b80 1 -- start start 2023-10-16T14:35:59.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.182+0000 7f9a474a1b80 1 Processor -- start 2023-10-16T14:35:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 0x55e98d7e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 0x55e98d7e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d7e6500 con 0x55e98d7d7620 2023-10-16T14:35:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d7e6670 con 0x55e98d7d5e20 2023-10-16T14:35:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e98d7e6830 con 0x55e98d7d6780 2023-10-16T14:35:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a474a1b80 10 client.0 _ll_register_callbacks cb 0x7f9a1c0127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:35:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46536/0 (socket says 172.21.15.178:46536) 2023-10-16T14:35:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 -- 172.21.15.178:0/1588368951 learned_addr learned my addr 172.21.15.178:0/1588368951 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3119a700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 0x55e98d7e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3219c700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 0x55e98d7e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 -- 172.21.15.178:0/1588368951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 msgr2=0x55e98d7e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 0x55e98d7e2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 -- 172.21.15.178:0/1588368951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 msgr2=0x55e98d7e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:35:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 0x55e98d7e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3199b700 1 -- 172.21.15.178:0/1588368951 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e98d6e63f0 con 0x55e98d7d5e20 2023-10-16T14:35:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3219c700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 0x55e98d7e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.183+0000 7f9a3119a700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 0x55e98d7e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:35:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a3199b700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806b570 tx=0x7f9a2809b270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d9b43780272cc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a157fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:35:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280a35f0 con 0x55e98d7d5e20 2023-10-16T14:35:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9a2809f150 con 0x55e98d7d5e20 2023-10-16T14:35:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a2809f3c0 con 0x55e98d7d5e20 2023-10-16T14:35:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a280993b0 con 0x55e98d7d5e20 2023-10-16T14:35:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a474a1b80 10 client.25814 fetch_fsmap learned FSMap version 30 2023-10-16T14:35:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e98d553de0 con 0x55e98d7d5e20 2023-10-16T14:35:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.184+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a2809f3c0 con 0x55e98d7d5e20 2023-10-16T14:35:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f9a2809b560 con 0x55e98d7d5e20 2023-10-16T14:35:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:35:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 10 client.25814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:35:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 20 client.25814 populate_metadata read hostname 'smithi178' 2023-10-16T14:35:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d840000 0x55e98d842420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 4 client.25814 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:35:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e98d721e50 con 0x55e98d840000 2023-10-16T14:35:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.185+0000 7f9a3119a700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d840000 0x55e98d842420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.186+0000 7f9a157fa700 10 client.25814 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:59.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.186+0000 7f9a3119a700 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d840000 0x55e98d842420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bdad4b9085118b server_cookie=db25683afd29c355 in_seq=0 out_seq=0 2023-10-16T14:35:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a157fa700 1 -- 172.21.15.178:0/1588368951 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e98d721e50 con 0x55e98d840000 2023-10-16T14:35:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a157fa700 10 client.25814 handle_command_reply: tid=0 2023-10-16T14:35:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 client.25814 shutdown 2023-10-16T14:35:59.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 msgr2=0x55e98d7e2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9a2806b570 tx=0x7f9a2809b270 comp rx=0 tx=0).stop 2023-10-16T14:35:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 shutdown_connections 2023-10-16T14:35:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e98d840000 0x55e98d842420 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e98d7d7620 0x55e98d7e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e98d7d5e20 0x55e98d7e2a60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 --2- 172.21.15.178:0/1588368951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e98d7d6780 0x55e98d7e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a157fa700 0 client.25814 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:35:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 >> 172.21.15.178:0/1588368951 conn(0x55e98d703dd0 msgr2=0x55e98d7ed7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 shutdown_connections 2023-10-16T14:35:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.188+0000 7f9a474a1b80 1 -- 172.21.15.178:0/1588368951 wait complete. 2023-10-16T14:35:59.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a474a1b80 20 client.25814 trim_cache size 0 max 16384 2023-10-16T14:35:59.204 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:35:59.204 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:35:59.204 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:35:59.205 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:35:59.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 msgr2=0x7f9a180a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 0x7f9a180a39c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009fd0 tx=0x7f9a24009920 comp rx=0 tx=0).stop 2023-10-16T14:35:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 msgr2=0x7f9a2c19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9a28004740 tx=0x7f9a28009580 comp rx=0 tx=0).stop 2023-10-16T14:35:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 shutdown_connections 2023-10-16T14:35:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f9a180a1500 0x7f9a180a39c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a2c108d90 0x7f9a2c1a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9a2c0738b0 0x7f9a2c19d1e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 --2- 172.21.15.178:0/2675236612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9a2c074e90 0x7f9a2c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.189+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 >> 172.21.15.178:0/2675236612 conn(0x7f9a2c0fc3f0 msgr2=0x7f9a2c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.190+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 shutdown_connections 2023-10-16T14:35:59.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.190+0000 7f9a33bff700 1 -- 172.21.15.178:0/2675236612 wait complete. 2023-10-16T14:35:59.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:35:59.212 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:35:59.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 -- 172.21.15.178:0/1968750365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c100eb0 msgr2=0x7fb78c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 --2- 172.21.15.178:0/1968750365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c100eb0 0x7fb78c1012d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb788009990 tx=0x7fb788009ca0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 -- 172.21.15.178:0/1968750365 shutdown_connections 2023-10-16T14:35:59.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 --2- 172.21.15.178:0/1968750365 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 0x7fb78c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 --2- 172.21.15.178:0/1968750365 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c100eb0 0x7fb78c1012d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 --2- 172.21.15.178:0/1968750365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c102090 0x7fb78c102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 -- 172.21.15.178:0/1968750365 >> 172.21.15.178:0/1968750365 conn(0x7fb78c0fc430 msgr2=0x7fb78c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 -- 172.21.15.178:0/1968750365 shutdown_connections 2023-10-16T14:35:59.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.446+0000 7fb7937ff700 1 -- 172.21.15.178:0/1968750365 wait complete. 2023-10-16T14:35:59.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.447+0000 7fb7937ff700 1 Processor -- start 2023-10-16T14:35:59.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 -- start start 2023-10-16T14:35:59.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c100eb0 0x7fb78c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 0x7fb78c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a08e0 con 0x7fb78c102a50 2023-10-16T14:35:59.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a0a20 con 0x7fb78c102090 2023-10-16T14:35:59.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb7937ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c1a0bb0 con 0x7fb78c100eb0 2023-10-16T14:35:59.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb790d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb790d9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46550/0 (socket says 172.21.15.178:46550) 2023-10-16T14:35:59.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb790d9a700 1 -- 172.21.15.178:0/2864275755 learned_addr learned my addr 172.21.15.178:0/2864275755 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:35:59.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.448+0000 7fb791d9c700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 0x7fb78c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb79159b700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c100eb0 0x7fb78c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 -- 172.21.15.178:0/2864275755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c100eb0 msgr2=0x7fb78c198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c100eb0 0x7fb78c198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 -- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 msgr2=0x7fb78c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 0x7fb78c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 -- 172.21.15.178:0/2864275755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880095f0 con 0x7fb78c102090 2023-10-16T14:35:59.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb790d9a700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e970 tx=0x7fb77c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c009790 con 0x7fb78c102090 2023-10-16T14:35:59.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb77c00c7e0 con 0x7fb78c102090 2023-10-16T14:35:59.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c003c70 con 0x7fb78c102090 2023-10-16T14:35:59.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c1a0e90 con 0x7fb78c102090 2023-10-16T14:35:59.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.449+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c1a1310 con 0x7fb78c102090 2023-10-16T14:35:59.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.450+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c10c770 con 0x7fb78c102090 2023-10-16T14:35:59.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.453+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb77c00cc50 con 0x7fb78c102090 2023-10-16T14:35:59.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.454+0000 7fb7827fc700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 0x7fb7780a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:35:59.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.454+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb77c014070 con 0x7fb78c102090 2023-10-16T14:35:59.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.454+0000 7fb79159b700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 0x7fb7780a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:35:59.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.455+0000 7fb79159b700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 0x7fb7780a3a60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b2e0 tx=0x7fb788005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:35:59.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.457+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb77c096610 con 0x7fb78c102090 2023-10-16T14:35:59.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.883+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb78c10c980 con 0x7fb78c102090 2023-10-16T14:35:59.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.886+0000 7fb7827fc700 1 -- 172.21.15.178:0/2864275755 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fb77c003dd0 con 0x7fb78c102090 2023-10-16T14:35:59.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:35:59.893 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:35:50.922947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:35:59.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.915+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 msgr2=0x7fb7780a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.915+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 0x7fb7780a3a60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b2e0 tx=0x7fb788005fb0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.915+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 msgr2=0x7fb78c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:35:59.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.915+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00e970 tx=0x7fb77c00ec80 comp rx=0 tx=0).stop 2023-10-16T14:35:59.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 shutdown_connections 2023-10-16T14:35:59.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb7780a15a0 0x7fb7780a3a60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb78c102a50 0x7fb78c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb78c100eb0 0x7fb78c198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 --2- 172.21.15.178:0/2864275755 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb78c102090 0x7fb78c1992a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:35:59.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.941+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 >> 172.21.15.178:0/2864275755 conn(0x7fb78c0fc430 msgr2=0x7fb78c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:35:59.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.945+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 shutdown_connections 2023-10-16T14:35:59.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:35:59.945+0000 7fb7937ff700 1 -- 172.21.15.178:0/2864275755 wait complete. 2023-10-16T14:35:59.954 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 30 2023-10-16T14:36:00.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:35:59 smithi178 ceph-mon[98495]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 103 op/s 2023-10-16T14:36:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:35:59 smithi186 ceph-mon[134892]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 103 op/s 2023-10-16T14:36:00.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:36:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:35:59 smithi187 ceph-mon[135297]: pgmap v407: 129 pgs: 129 active+clean; 82 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 103 op/s 2023-10-16T14:36:00.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 -- 172.21.15.178:0/3159054994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102a20 msgr2=0x7ff77c109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3159054994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102a20 0x7ff77c109340 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff77400b0a0 tx=0x7ff77400b3b0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 -- 172.21.15.178:0/3159054994 shutdown_connections 2023-10-16T14:36:00.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3159054994 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102a20 0x7ff77c109340 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3159054994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c100e80 0x7ff77c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3159054994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c102060 0x7ff77c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 -- 172.21.15.178:0/3159054994 >> 172.21.15.178:0/3159054994 conn(0x7ff77c0fc400 msgr2=0x7ff77c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.544+0000 7ff7811ff700 1 -- 172.21.15.178:0/3159054994 shutdown_connections 2023-10-16T14:36:00.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.545+0000 7ff7811ff700 1 -- 172.21.15.178:0/3159054994 wait complete. 2023-10-16T14:36:00.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.545+0000 7ff7811ff700 1 Processor -- start 2023-10-16T14:36:00.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 -- start start 2023-10-16T14:36:00.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c100e80 0x7ff77c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 0x7ff77c1a0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1a0a50 con 0x7ff77c102a20 2023-10-16T14:36:00.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1a0b90 con 0x7ff77c102060 2023-10-16T14:36:00.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff7811ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1a0d20 con 0x7ff77c100e80 2023-10-16T14:36:00.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46558/0 (socket says 172.21.15.178:46558) 2023-10-16T14:36:00.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77a59c700 1 -- 172.21.15.178:0/3185821214 learned_addr learned my addr 172.21.15.178:0/3185821214 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77b59e700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 0x7ff77c1a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c100e80 0x7ff77c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77a59c700 1 -- 172.21.15.178:0/3185821214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c100e80 msgr2=0x7ff77c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.546+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c100e80 0x7ff77c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff77a59c700 1 -- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 msgr2=0x7ff77c1a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 0x7ff77c1a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff77a59c700 1 -- 172.21.15.178:0/3185821214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009d00 con 0x7ff77c102060 2023-10-16T14:36:00.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff77b59e700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 0x7ff77c1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:00.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff77000e970 tx=0x7ff77000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff763fff700 1 -- 172.21.15.178:0/3185821214 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770009790 con 0x7ff77c102060 2023-10-16T14:36:00.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff763fff700 1 -- 172.21.15.178:0/3185821214 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff77000c7e0 con 0x7ff77c102060 2023-10-16T14:36:00.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c1a0fa0 con 0x7ff77c102060 2023-10-16T14:36:00.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff763fff700 1 -- 172.21.15.178:0/3185821214 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770003c70 con 0x7ff77c102060 2023-10-16T14:36:00.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.547+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1a1480 con 0x7ff77c102060 2023-10-16T14:36:00.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.550+0000 7ff763fff700 1 -- 172.21.15.178:0/3185821214 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff77000cc50 con 0x7ff77c102060 2023-10-16T14:36:00.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.552+0000 7ff763fff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 0x7ff7640a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.552+0000 7ff794aa3b80 1 Processor -- start 2023-10-16T14:36:00.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.552+0000 7ff763fff700 1 -- 172.21.15.178:0/3185821214 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff770014070 con 0x7ff77c102060 2023-10-16T14:36:00.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.552+0000 7ff77b59e700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 0x7ff7640a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.552+0000 7ff794aa3b80 1 -- start start 2023-10-16T14:36:00.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57091fc70 con 0x55e5709ab780 2023-10-16T14:36:00.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e570904c60 con 0x55e5709aae20 2023-10-16T14:36:00.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e570902580 con 0x55e5709ac620 2023-10-16T14:36:00.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.563 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46570/0 (socket says 172.21.15.178:46570) 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 -- 172.21.15.178:0/2470939645 learned_addr learned my addr 172.21.15.178:0/2470939645 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77b59e700 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77b59e700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 0x7ff7640a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff774009fd0 tx=0x7ff77400b720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 -- 172.21.15.178:0/2470939645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 msgr2=0x55e5709acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 -- 172.21.15.178:0/2470939645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 msgr2=0x55e5709ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.553+0000 7ff77ad9d700 1 -- 172.21.15.178:0/2470939645 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5708bb3f0 con 0x55e5709aae20 2023-10-16T14:36:00.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.554+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d30 tx=0x7ff76c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b09f09ee7024ef1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.554+0000 7ff7617fa700 1 -- 172.21.15.178:0/2470939645 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004220 con 0x55e5709aae20 2023-10-16T14:36:00.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.554+0000 7ff7617fa700 1 -- 172.21.15.178:0/2470939645 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff76c004380 con 0x55e5709aae20 2023-10-16T14:36:00.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.554+0000 7ff7617fa700 1 -- 172.21.15.178:0/2470939645 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0198d0 con 0x55e5709aae20 2023-10-16T14:36:00.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- 172.21.15.178:0/2470939645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 msgr2=0x55e5709ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009d30 tx=0x7ff76c011720 comp rx=0 tx=0).stop 2023-10-16T14:36:00.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- 172.21.15.178:0/2470939645 shutdown_connections 2023-10-16T14:36:00.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab240 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/2470939645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- 172.21.15.178:0/2470939645 >> 172.21.15.178:0/2470939645 conn(0x55e5708d8dd0 msgr2=0x55e5709a5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- 172.21.15.178:0/2470939645 shutdown_connections 2023-10-16T14:36:00.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- 172.21.15.178:0/2470939645 wait complete. 2023-10-16T14:36:00.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.555+0000 7ff794aa3b80 1 -- start start 2023-10-16T14:36:00.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 Processor -- start 2023-10-16T14:36:00.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709aae20 0x55e5709bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5708fde70 con 0x55e5709aae20 2023-10-16T14:36:00.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57089a710 con 0x55e5709ab780 2023-10-16T14:36:00.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57089a3b0 con 0x55e5709ac620 2023-10-16T14:36:00.569 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff794aa3b80 10 client.0 _ll_register_callbacks cb 0x55e5706f7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:00.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709aae20 0x55e5709bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46576/0 (socket says 172.21.15.178:46576) 2023-10-16T14:36:00.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff77a59c700 1 -- 172.21.15.178:0/538215655 learned_addr learned my addr 172.21.15.178:0/538215655 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.570 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.556+0000 7ff77b59e700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 -- 172.21.15.178:0/538215655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 msgr2=0x55e5709c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 -- 172.21.15.178:0/538215655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709aae20 msgr2=0x55e5709bd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709aae20 0x55e5709bd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 -- 172.21.15.178:0/538215655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76c009960 con 0x55e5709ab780 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff77a59c700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff770005950 tx=0x7ff770099b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3796bba7b399f2e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff75eff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77009a610 con 0x55e5709ab780 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e570902ef0 con 0x55e5709ab780 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff77009a1b0 con 0x55e5709ab780 2023-10-16T14:36:00.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff77009e300 con 0x55e5709ab780 2023-10-16T14:36:00.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff794aa3b80 10 client.25832 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:00.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.557+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e570728de0 con 0x55e5709ab780 2023-10-16T14:36:00.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e570902ef0 con 0x55e5709ab780 2023-10-16T14:36:00.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff7700a20b0 con 0x55e5709ab780 2023-10-16T14:36:00.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:00.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 10 client.25832 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:36:00.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 20 client.25832 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:00.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e5709c9220 0x55e5709cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 4 client.25832 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:00.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e57097ac10 con 0x55e5709c9220 2023-10-16T14:36:00.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.558+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e5709c9220 0x55e5709cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.559+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e5709c9220 0x55e5709cb640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4124fd4c42506563 server_cookie=2a73cb5906880192 in_seq=0 out_seq=0 2023-10-16T14:36:00.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.559+0000 7ff75eff5700 10 client.25832 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:00.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff75eff5700 1 -- 172.21.15.178:0/538215655 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e57097ac10 con 0x55e5709c9220 2023-10-16T14:36:00.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff75eff5700 10 client.25832 handle_command_reply: tid=0 2023-10-16T14:36:00.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 client.25832 shutdown 2023-10-16T14:36:00.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 msgr2=0x55e5709bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff770005950 tx=0x7ff770099b60 comp rx=0 tx=0).stop 2023-10-16T14:36:00.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 shutdown_connections 2023-10-16T14:36:00.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e5709c9220 0x55e5709cb640 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff75eff5700 0 client.25832 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:00.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709aae20 0x55e5709bd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/538215655 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709ab780 0x55e5709bd990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 >> 172.21.15.178:0/538215655 conn(0x55e5708d8dd0 msgr2=0x55e5709a5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 shutdown_connections 2023-10-16T14:36:00.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.560+0000 7ff794aa3b80 1 -- 172.21.15.178:0/538215655 wait complete. 2023-10-16T14:36:00.579 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.561+0000 7ff794aa3b80 20 client.25832 trim_cache size 0 max 16384 2023-10-16T14:36:00.583 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 Processor -- start 2023-10-16T14:36:00.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 -- start start 2023-10-16T14:36:00.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ab780 0x55e5709c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ac620 0x55e5709af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e57077e920 con 0x55e5709ac620 2023-10-16T14:36:00.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5709ac4a0 con 0x55e5709aae20 2023-10-16T14:36:00.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5709b40f0 con 0x55e5709ab780 2023-10-16T14:36:00.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff77b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ac620 0x55e5709af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46588/0 (socket says 172.21.15.178:46588) 2023-10-16T14:36:00.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3974203308 learned_addr learned my addr 172.21.15.178:0/3974203308 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.576+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ab780 0x55e5709c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3974203308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ab780 msgr2=0x55e5709c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ab780 0x55e5709c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3974203308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ac620 msgr2=0x55e5709af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ac620 0x55e5709af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3974203308 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e570711c60 con 0x55e5709aae20 2023-10-16T14:36:00.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff76c011a70 tx=0x7ff76c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab1e5685fd385fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff761ffb700 1 -- 172.21.15.178:0/3974203308 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c029040 con 0x55e5709aae20 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff761ffb700 1 -- 172.21.15.178:0/3974203308 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff76c004030 con 0x55e5709aae20 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.577+0000 7ff761ffb700 1 -- 172.21.15.178:0/3974203308 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c005040 con 0x55e5709aae20 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3974203308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 msgr2=0x55e5709ab200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff76c011a70 tx=0x7ff76c011720 comp rx=0 tx=0).stop 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3974203308 shutdown_connections 2023-10-16T14:36:00.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ac620 0x55e5709af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709ab200 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3974203308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ab780 0x55e5709c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3974203308 >> 172.21.15.178:0/3974203308 conn(0x55e5708d8dd0 msgr2=0x55e5709a47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3974203308 shutdown_connections 2023-10-16T14:36:00.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3974203308 wait complete. 2023-10-16T14:36:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.578+0000 7ff794aa3b80 1 -- start start 2023-10-16T14:36:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 Processor -- start 2023-10-16T14:36:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5709c45b0 con 0x55e5709ab780 2023-10-16T14:36:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5709c4720 con 0x55e5709aae20 2023-10-16T14:36:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5709c48e0 con 0x55e5709ac620 2023-10-16T14:36:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff794aa3b80 10 client.0 _ll_register_callbacks cb 0x7ff770099cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:00.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff77ad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46604/0 (socket says 172.21.15.178:46604) 2023-10-16T14:36:00.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff77a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3646960525 learned_addr learned my addr 172.21.15.178:0/3646960525 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.579+0000 7ff77b59e700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3646960525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 msgr2=0x55e5709c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3646960525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 msgr2=0x55e5709c1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709c1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 -- 172.21.15.178:0/3646960525 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5708bb3f0 con 0x55e5709aae20 2023-10-16T14:36:00.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff77ad9d700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff76c014040 tx=0x7ff76c009cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc44b00491f14559 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff75eff5700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c029040 con 0x55e5709aae20 2023-10-16T14:36:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff76c004380 con 0x55e5709aae20 2023-10-16T14:36:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.580+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff76c039070 con 0x55e5709aae20 2023-10-16T14:36:00.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76c0041a0 con 0x55e5709aae20 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 10 client.25838 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e57074f250 con 0x55e5709aae20 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff76c004380 con 0x55e5709aae20 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff76c011440 con 0x55e5709aae20 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:00.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:36:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 20 client.25838 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e570a1dfa0 0x55e570a203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 4 client.25838 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e5708f6e50 con 0x55e570a1dfa0 2023-10-16T14:36:00.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.581+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e570a1dfa0 0x55e570a203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff77a59c700 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e570a1dfa0 0x55e570a203c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4fc348597c7a0ec9 server_cookie=46319aabac7dc6ac in_seq=0 out_seq=0 2023-10-16T14:36:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff75eff5700 10 client.25838 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:00.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff75eff5700 1 -- 172.21.15.178:0/3646960525 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e5708f6e50 con 0x55e570a1dfa0 2023-10-16T14:36:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff75eff5700 10 client.25838 handle_command_reply: tid=0 2023-10-16T14:36:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 client.25838 shutdown 2023-10-16T14:36:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 msgr2=0x55e5709c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff76c014040 tx=0x7ff76c009cc0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 shutdown_connections 2023-10-16T14:36:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55e570a1dfa0 0x55e570a203c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e5709ac620 0x55e5709c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff75eff5700 0 client.25838 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e5709aae20 0x55e5709c0b10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 --2- 172.21.15.178:0/3646960525 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e5709ab780 0x55e5709c1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 >> 172.21.15.178:0/3646960525 conn(0x55e5708d8dd0 msgr2=0x55e5709af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 shutdown_connections 2023-10-16T14:36:00.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 1 -- 172.21.15.178:0/3646960525 wait complete. 2023-10-16T14:36:00.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.582+0000 7ff794aa3b80 20 client.25838 trim_cache size 0 max 16384 2023-10-16T14:36:00.602 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:00.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 msgr2=0x7ff7640a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 0x7ff7640a3940 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff774009fd0 tx=0x7ff77400b720 comp rx=0 tx=0).stop 2023-10-16T14:36:00.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 msgr2=0x7ff77c199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff77000e970 tx=0x7ff77000ec80 comp rx=0 tx=0).stop 2023-10-16T14:36:00.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 shutdown_connections 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff7640a1480 0x7ff7640a3940 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c102a20 0x7ff77c1a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff77c100e80 0x7ff77c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 --2- 172.21.15.178:0/3185821214 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff77c102060 0x7ff77c199410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 >> 172.21.15.178:0/3185821214 conn(0x7ff77c0fc400 msgr2=0x7ff77c1079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 shutdown_connections 2023-10-16T14:36:00.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.583+0000 7ff7811ff700 1 -- 172.21.15.178:0/3185821214 wait complete. 2023-10-16T14:36:00.606 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:36:00.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 -- 172.21.15.178:0/1250078033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0101980 msgr2=0x7f96e0103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 --2- 172.21.15.178:0/1250078033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0101980 0x7f96e0103d70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009990 tx=0x7f96d0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 -- 172.21.15.178:0/1250078033 shutdown_connections 2023-10-16T14:36:00.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 --2- 172.21.15.178:0/1250078033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0106be0 0x7f96e0109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 --2- 172.21.15.178:0/1250078033 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0101980 0x7f96e0103d70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 --2- 172.21.15.178:0/1250078033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 0x7f96e01066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 -- 172.21.15.178:0/1250078033 >> 172.21.15.178:0/1250078033 conn(0x7f96e00fb370 msgr2=0x7f96e00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 -- 172.21.15.178:0/1250078033 shutdown_connections 2023-10-16T14:36:00.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.814+0000 7f96e65ff700 1 -- 172.21.15.178:0/1250078033 wait complete. 2023-10-16T14:36:00.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.815+0000 7f96e65ff700 1 Processor -- start 2023-10-16T14:36:00.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.815+0000 7f96e65ff700 1 -- start start 2023-10-16T14:36:00.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.815+0000 7f96e65ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 0x7f96e0198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e65ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 0x7f96e0199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e65ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 0x7f96e01a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e65ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0980 con 0x7f96e0101980 2023-10-16T14:36:00.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e65ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0ac0 con 0x7f96e0106be0 2023-10-16T14:36:00.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e65ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e01a0c50 con 0x7f96e01042b0 2023-10-16T14:36:00.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 0x7f96e0198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 0x7f96e01a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 0x7f96e0198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38796/0 (socket says 172.21.15.178:38796) 2023-10-16T14:36:00.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96dffff700 1 -- 172.21.15.178:0/2873000600 learned_addr learned my addr 172.21.15.178:0/2873000600 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96df7fe700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 0x7f96e0199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2873000600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 msgr2=0x7f96e0199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 0x7f96e0199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 msgr2=0x7f96e0198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 0x7f96e0198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.816+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2873000600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00095f0 con 0x7f96e0106be0 2023-10-16T14:36:00.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 0x7f96e01a03b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96d800efc0 tx=0x7f96d800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96dd7fa700 1 -- 172.21.15.178:0/2873000600 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8018070 con 0x7f96e0106be0 2023-10-16T14:36:00.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e0191d30 con 0x7f96e0106be0 2023-10-16T14:36:00.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96dd7fa700 1 -- 172.21.15.178:0/2873000600 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d800f040 con 0x7f96e0106be0 2023-10-16T14:36:00.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96dd7fa700 1 -- 172.21.15.178:0/2873000600 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800cd40 con 0x7f96e0106be0 2023-10-16T14:36:00.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.817+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e01921f0 con 0x7f96e0106be0 2023-10-16T14:36:00.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.821+0000 7f96dd7fa700 1 -- 172.21.15.178:0/2873000600 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f96d8003a00 con 0x7f96e0106be0 2023-10-16T14:36:00.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.822+0000 7f96dd7fa700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 0x7f96c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.822+0000 7f96dd7fa700 1 -- 172.21.15.178:0/2873000600 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f96d80ccc00 con 0x7f96e0106be0 2023-10-16T14:36:00.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.822+0000 7f96dffff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 0x7f96c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.823+0000 7f96dffff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 0x7f96c80a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f96d0006010 tx=0x7f96d0005960 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.823+0000 7f96f9eadb80 1 Processor -- start 2023-10-16T14:36:00.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 -- start start 2023-10-16T14:36:00.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f959240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f95a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f8cdc70 con 0x563d9f959780 2023-10-16T14:36:00.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f8b2c60 con 0x563d9f95a620 2023-10-16T14:36:00.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f8b0580 con 0x563d9f958e20 2023-10-16T14:36:00.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96df7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f95a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96e4b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.824+0000 7f96dffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f959240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46640/0 (socket says 172.21.15.178:46640) 2023-10-16T14:36:00.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2217793282 learned_addr learned my addr 172.21.15.178:0/2217793282 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2217793282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 msgr2=0x563d9f959240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f959240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2217793282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 msgr2=0x563d9f95a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f95a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 -- 172.21.15.178:0/2217793282 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9f8693f0 con 0x563d9f95a620 2023-10-16T14:36:00.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f96d80090a0 tx=0x7f96d8096020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a2f744d5a3db09a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96c67fc700 1 -- 172.21.15.178:0/2217793282 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8099540 con 0x563d9f95a620 2023-10-16T14:36:00.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96c67fc700 1 -- 172.21.15.178:0/2217793282 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d8004ad0 con 0x563d9f95a620 2023-10-16T14:36:00.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.825+0000 7f96c67fc700 1 -- 172.21.15.178:0/2217793282 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8099e40 con 0x563d9f95a620 2023-10-16T14:36:00.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.826+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2217793282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 msgr2=0x563d9f95aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.826+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f96d80090a0 tx=0x7f96d8096020 comp rx=0 tx=0).stop 2023-10-16T14:36:00.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2217793282 shutdown_connections 2023-10-16T14:36:00.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95aaa0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f959240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2217793282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f95a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2217793282 >> 172.21.15.178:0/2217793282 conn(0x563d9f886dd0 msgr2=0x563d9f953fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2217793282 shutdown_connections 2023-10-16T14:36:00.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2217793282 wait complete. 2023-10-16T14:36:00.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.827+0000 7f96f9eadb80 1 -- start start 2023-10-16T14:36:00.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 Processor -- start 2023-10-16T14:36:00.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f962c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f9631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f8abe70 con 0x563d9f958e20 2023-10-16T14:36:00.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f848710 con 0x563d9f95a620 2023-10-16T14:36:00.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.828+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f8483b0 con 0x563d9f959780 2023-10-16T14:36:00.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96f9eadb80 10 client.0 _ll_register_callbacks cb 0x563d9f6a5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:00.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96dffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f962c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46644/0 (socket says 172.21.15.178:46644) 2023-10-16T14:36:00.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 -- 172.21.15.178:0/1167515230 learned_addr learned my addr 172.21.15.178:0/1167515230 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96df7fe700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f9631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 -- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 msgr2=0x563d9f9631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f9631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 -- 172.21.15.178:0/1167515230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 msgr2=0x563d9f962c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f962c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 -- 172.21.15.178:0/1167515230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e01921f0 con 0x563d9f95a620 2023-10-16T14:36:00.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96d8004c60 tx=0x7f96d8004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c87c7a63a80980 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:00.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.829+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8099be0 con 0x563d9f95a620 2023-10-16T14:36:00.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f96d801b320 con 0x563d9f95a620 2023-10-16T14:36:00.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d9f8b0ef0 con 0x563d9f95a620 2023-10-16T14:36:00.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d809a2e0 con 0x563d9f95a620 2023-10-16T14:36:00.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 10 client.25850 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:00.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563d9f6d6de0 con 0x563d9f95a620 2023-10-16T14:36:00.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d9f8b0ef0 con 0x563d9f95a620 2023-10-16T14:36:00.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f96d80a1b70 con 0x563d9f95a620 2023-10-16T14:36:00.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 10 client.25850 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:00.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 10 client.25850 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:36:00.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 20 client.25850 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:00.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f96e460 0x563d9f970880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 4 client.25850 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:00.847 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.830+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d9f928c10 con 0x563d9f96e460 2023-10-16T14:36:00.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.831+0000 7f96dffff700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f96e460 0x563d9f970880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.831+0000 7f96dffff700 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f96e460 0x563d9f970880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=317e2358da8d78cb server_cookie=ce251be79404eb5c in_seq=0 out_seq=0 2023-10-16T14:36:00.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.831+0000 7f96bf7fe700 10 client.25850 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:00.848 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96bf7fe700 1 -- 172.21.15.178:0/1167515230 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d9f928c10 con 0x563d9f96e460 2023-10-16T14:36:00.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96bf7fe700 10 client.25850 handle_command_reply: tid=0 2023-10-16T14:36:00.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 client.25850 shutdown 2023-10-16T14:36:00.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 msgr2=0x563d9f96a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96d8004c60 tx=0x7f96d8004cc0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.849 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 shutdown_connections 2023-10-16T14:36:00.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f96e460 0x563d9f970880 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f96a200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f962c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/1167515230 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f9631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.850 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96bf7fe700 0 client.25850 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:00.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 >> 172.21.15.178:0/1167515230 conn(0x563d9f886dd0 msgr2=0x563d9f953bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 shutdown_connections 2023-10-16T14:36:00.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.832+0000 7f96f9eadb80 1 -- 172.21.15.178:0/1167515230 wait complete. 2023-10-16T14:36:00.851 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.833+0000 7f96f9eadb80 20 client.25850 trim_cache size 0 max 16384 2023-10-16T14:36:00.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 Processor -- start 2023-10-16T14:36:00.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 -- start start 2023-10-16T14:36:00.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f953b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f72c920 con 0x563d9f958e20 2023-10-16T14:36:00.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f963e70 con 0x563d9f95a620 2023-10-16T14:36:00.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.847+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f952e20 con 0x563d9f959780 2023-10-16T14:36:00.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52646/0 (socket says 172.21.15.178:52646) 2023-10-16T14:36:00.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 -- 172.21.15.178:0/2047161408 learned_addr learned my addr 172.21.15.178:0/2047161408 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96dffff700 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f953b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 -- 172.21.15.178:0/2047161408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 msgr2=0x563d9f95d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 -- 172.21.15.178:0/2047161408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 msgr2=0x563d9f953b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f953b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 -- 172.21.15.178:0/2047161408 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d9f6bfc60 con 0x563d9f959780 2023-10-16T14:36:00.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.848+0000 7f96df7fe700 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ed0 tx=0x7f96d4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5145b48da98bf35d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.849+0000 7f96c6ffd700 1 -- 172.21.15.178:0/2047161408 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4017050 con 0x563d9f959780 2023-10-16T14:36:00.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.849+0000 7f96c6ffd700 1 -- 172.21.15.178:0/2047161408 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d4018040 con 0x563d9f959780 2023-10-16T14:36:00.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.849+0000 7f96c6ffd700 1 -- 172.21.15.178:0/2047161408 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d401c400 con 0x563d9f959780 2023-10-16T14:36:00.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.849+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2047161408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 msgr2=0x563d9f96a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.849+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ed0 tx=0x7f96d4009260 comp rx=0 tx=0).stop 2023-10-16T14:36:00.860 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2047161408 shutdown_connections 2023-10-16T14:36:00.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f95d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f958e20 0x563d9f953b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/2047161408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f959780 0x563d9f96a1c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.861 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2047161408 >> 172.21.15.178:0/2047161408 conn(0x563d9f886dd0 msgr2=0x563d9f96aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2047161408 shutdown_connections 2023-10-16T14:36:00.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 -- 172.21.15.178:0/2047161408 wait complete. 2023-10-16T14:36:00.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 -- start start 2023-10-16T14:36:00.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.850+0000 7f96f9eadb80 1 Processor -- start 2023-10-16T14:36:00.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f96a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f9683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f968ac0 con 0x563d9f959780 2023-10-16T14:36:00.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f968c30 con 0x563d9f95a620 2023-10-16T14:36:00.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9f968df0 con 0x563d9f958e20 2023-10-16T14:36:00.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96f9eadb80 10 client.0 _ll_register_callbacks cb 0x7f96d4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:00.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96dffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96df7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f96a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.851+0000 7f96dffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52654/0 (socket says 172.21.15.178:52654) 2023-10-16T14:36:00.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 -- 172.21.15.178:0/3327084069 learned_addr learned my addr 172.21.15.178:0/3327084069 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:00.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 -- 172.21.15.178:0/3327084069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 msgr2=0x563d9f9683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f9683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f9683b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 -- 172.21.15.178:0/3327084069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 msgr2=0x563d9f96a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:00.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f96a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 -- 172.21.15.178:0/3327084069 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4009b80 con 0x563d9f958e20 2023-10-16T14:36:00.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96e4b9c700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f9683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:00.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96dffff700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f96d00059d0 tx=0x7f96d0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b98808ae3dd9d52 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:00.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.852+0000 7f96bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:00.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0033040 con 0x563d9f958e20 2023-10-16T14:36:00.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96d8004ad0 con 0x563d9f958e20 2023-10-16T14:36:00.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f96d00056a0 con 0x563d9f958e20 2023-10-16T14:36:00.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96d003eae0 con 0x563d9f958e20 2023-10-16T14:36:00.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 10 client.25840 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:00.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563d9f6fd250 con 0x563d9f958e20 2023-10-16T14:36:00.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d8004ad0 con 0x563d9f958e20 2023-10-16T14:36:00.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f96d0044020 con 0x563d9f958e20 2023-10-16T14:36:00.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 10 client.25840 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:00.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 10 client.25840 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:36:00.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.853+0000 7f96f9eadb80 20 client.25840 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:00.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f9c3630 0x563d9f9c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:00.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96f9eadb80 4 client.25840 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:00.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d9f8a4e50 con 0x563d9f9c3630 2023-10-16T14:36:00.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96df7fe700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f9c3630 0x563d9f9c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:00.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96bf7fe700 10 client.25840 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:00.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.854+0000 7f96df7fe700 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f9c3630 0x563d9f9c5a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d95fdcc2e3b108d4 server_cookie=de6862dfb4ca335 in_seq=0 out_seq=0 2023-10-16T14:36:00.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96bf7fe700 1 -- 172.21.15.178:0/3327084069 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d9f8a4e50 con 0x563d9f9c3630 2023-10-16T14:36:00.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96bf7fe700 10 client.25840 handle_command_reply: tid=0 2023-10-16T14:36:00.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 client.25840 shutdown 2023-10-16T14:36:00.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 msgr2=0x563d9f969cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f96d00059d0 tx=0x7f96d0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 shutdown_connections 2023-10-16T14:36:00.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563d9f9c3630 0x563d9f9c5a50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563d9f95a620 0x563d9f9683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d9f958e20 0x563d9f969cb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96bf7fe700 0 client.25840 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:00.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 --2- 172.21.15.178:0/3327084069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563d9f959780 0x563d9f96a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 >> 172.21.15.178:0/3327084069 conn(0x563d9f886dd0 msgr2=0x563d9f953af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 shutdown_connections 2023-10-16T14:36:00.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 1 -- 172.21.15.178:0/3327084069 wait complete. 2023-10-16T14:36:00.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.855+0000 7f96f9eadb80 20 client.25840 trim_cache size 0 max 16384 2023-10-16T14:36:00.875 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:00.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 msgr2=0x7f96c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 0x7f96c80a3940 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f96d0006010 tx=0x7f96d0005960 comp rx=0 tx=0).stop 2023-10-16T14:36:00.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 msgr2=0x7f96e01a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:00.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 0x7f96e01a03b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f96d800efc0 tx=0x7f96d800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 shutdown_connections 2023-10-16T14:36:00.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f96c80a1480 0x7f96c80a3940 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f96e0106be0 0x7f96e01a03b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96e0101980 0x7f96e0198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 --2- 172.21.15.178:0/2873000600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96e01042b0 0x7f96e0199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:00.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 >> 172.21.15.178:0/2873000600 conn(0x7f96e00fb370 msgr2=0x7f96e0100000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:00.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 shutdown_connections 2023-10-16T14:36:00.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:00.856+0000 7f96e65ff700 1 -- 172.21.15.178:0/2873000600 wait complete. 2023-10-16T14:36:00.879 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2864275755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:00.882 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:36:00.973 INFO:tasks.workunit.client.0.smithi178.stdout: 48 273951 1098 196310 917 113245 638 2023-10-16T14:36:01.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2864275755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:01.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 -- 172.21.15.178:0/272511731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102a40 msgr2=0x7f4cc0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/272511731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102a40 0x7f4cc0109360 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009f90 tx=0x7f4cb800b330 comp rx=0 tx=0).stop 2023-10-16T14:36:01.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 -- 172.21.15.178:0/272511731 shutdown_connections 2023-10-16T14:36:01.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/272511731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102a40 0x7f4cc0109360 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/272511731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0100ea0 0x7f4cc01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/272511731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0102080 0x7f4cc0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 -- 172.21.15.178:0/272511731 >> 172.21.15.178:0/272511731 conn(0x7f4cc00fc420 msgr2=0x7f4cc00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 -- 172.21.15.178:0/272511731 shutdown_connections 2023-10-16T14:36:01.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.140+0000 7f4cc59ff700 1 -- 172.21.15.178:0/272511731 wait complete. 2023-10-16T14:36:01.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.141+0000 7f4cc59ff700 1 Processor -- start 2023-10-16T14:36:01.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.141+0000 7f4cc59ff700 1 -- start start 2023-10-16T14:36:01.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0100ea0 0x7f4cc0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0102a40 0x7f4cc01a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01a0a20 con 0x7f4cc0102a40 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01a0b60 con 0x7f4cc0102080 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cc59ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc01a0cf0 con 0x7f4cc0100ea0 2023-10-16T14:36:01.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cbe7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0102a40 0x7f4cc01a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cbe7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46680/0 (socket says 172.21.15.178:46680) 2023-10-16T14:36:01.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cbe7fc700 1 -- 172.21.15.178:0/3229242731 learned_addr learned my addr 172.21.15.178:0/3229242731 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:01.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.142+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0100ea0 0x7f4cc0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 -- 172.21.15.178:0/3229242731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0100ea0 msgr2=0x7f4cc0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0100ea0 0x7f4cc0198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 -- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0102a40 msgr2=0x7f4cc01a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0102a40 0x7f4cc01a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 -- 172.21.15.178:0/3229242731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00095f0 con 0x7f4cc0102080 2023-10-16T14:36:01.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.143+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e970 tx=0x7f4cb400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.144+0000 7f4cc49fd700 1 -- 172.21.15.178:0/3229242731 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4009790 con 0x7f4cc0102080 2023-10-16T14:36:01.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.144+0000 7f4cc49fd700 1 -- 172.21.15.178:0/3229242731 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb400eec0 con 0x7f4cc0102080 2023-10-16T14:36:01.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.144+0000 7f4cc49fd700 1 -- 172.21.15.178:0/3229242731 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4003d60 con 0x7f4cc0102080 2023-10-16T14:36:01.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.146+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb8009c40 con 0x7f4cc0102080 2023-10-16T14:36:01.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.146+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc01a12d0 con 0x7f4cc0102080 2023-10-16T14:36:01.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.156+0000 7f4cc49fd700 1 -- 172.21.15.178:0/3229242731 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f4cb4003ec0 con 0x7f4cc0102080 2023-10-16T14:36:01.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.160+0000 7f4cd93c1b80 1 Processor -- start 2023-10-16T14:36:01.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.160+0000 7f4cc49fd700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 0x7f4ca80a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.160+0000 7f4cd93c1b80 1 -- start start 2023-10-16T14:36:01.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f229b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f229c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f220fc70 con 0x55f4f229b780 2023-10-16T14:36:01.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f21f4c60 con 0x55f4f229c620 2023-10-16T14:36:01.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f21f2580 con 0x55f4f229ae20 2023-10-16T14:36:01.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46684/0 (socket says 172.21.15.178:46684) 2023-10-16T14:36:01.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/1623106241 learned_addr learned my addr 172.21.15.178:0/1623106241 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:01.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.161+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f229b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/1623106241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 msgr2=0x55f4f229b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f229b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/1623106241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 msgr2=0x55f4f229c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:36:01.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f229c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f229c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/1623106241 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f21ab3f0 con 0x55f4f229c620 2023-10-16T14:36:01.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f229b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:01.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.162+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f229c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:01.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.163+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009380 tx=0x7f4cb80093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40714976998e445 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.163+0000 7f4ca5ffb700 1 -- 172.21.15.178:0/1623106241 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800e070 con 0x55f4f229c620 2023-10-16T14:36:01.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.163+0000 7f4ca5ffb700 1 -- 172.21.15.178:0/1623106241 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb8007420 con 0x55f4f229c620 2023-10-16T14:36:01.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.163+0000 7f4ca5ffb700 1 -- 172.21.15.178:0/1623106241 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800bc50 con 0x55f4f229c620 2023-10-16T14:36:01.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.164+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/1623106241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 msgr2=0x55f4f229caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.164+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009380 tx=0x7f4cb80093b0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.165+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 0x7f4ca80a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.165+0000 7f4cc49fd700 1 -- 172.21.15.178:0/3229242731 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4cb4014070 con 0x7f4cc0102080 2023-10-16T14:36:01.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 0x7f4ca80a3b10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8032000 tx=0x7f4cb8037040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/1623106241 shutdown_connections 2023-10-16T14:36:01.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f229caa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f229b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/1623106241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f229c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.166+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/1623106241 >> 172.21.15.178:0/1623106241 conn(0x55f4f21c8dd0 msgr2=0x55f4f2295fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.167+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/1623106241 shutdown_connections 2023-10-16T14:36:01.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.167+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/1623106241 wait complete. 2023-10-16T14:36:01.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.167+0000 7f4cd93c1b80 1 -- start start 2023-10-16T14:36:01.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2864275755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:01.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.168+0000 7f4cd93c1b80 1 Processor -- start 2023-10-16T14:36:01.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.168+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.168+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22a5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52678/0 (socket says 172.21.15.178:52678) 2023-10-16T14:36:01.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cbe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cbeffd700 1 -- 172.21.15.178:0/4248766502 learned_addr learned my addr 172.21.15.178:0/4248766502 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:01.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f21ede70 con 0x55f4f229b780 2023-10-16T14:36:01.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f218a710 con 0x55f4f229c620 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.169+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f218a3b0 con 0x55f4f229ae20 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbeffd700 1 -- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 msgr2=0x55f4f22ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbeffd700 1 -- 172.21.15.178:0/4248766502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 msgr2=0x55f4f22a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbeffd700 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc0109c90 con 0x55f4f229ae20 2023-10-16T14:36:01.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.170+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:01.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.171+0000 7f4cd93c1b80 10 client.0 _ll_register_callbacks cb 0x55f4f1fe7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:01.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.171+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4f21f2ef0 con 0x55f4f229ae20 2023-10-16T14:36:01.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.171+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0012810 tx=0x7f4cb0012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5def85193859ecd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.171+0000 7f4c9effd700 10 client.25852 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:01.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.172+0000 7f4c9effd700 1 -- 172.21.15.178:0/4248766502 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0017070 con 0x55f4f229ae20 2023-10-16T14:36:01.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.172+0000 7f4c9effd700 1 -- 172.21.15.178:0/4248766502 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4cb0004030 con 0x55f4f229ae20 2023-10-16T14:36:01.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.172+0000 7f4c9effd700 1 -- 172.21.15.178:0/4248766502 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb0004250 con 0x55f4f229ae20 2023-10-16T14:36:01.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.172+0000 7f4cd93c1b80 10 client.25852 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:01.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.172+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x55f4f2018de0 con 0x55f4f229ae20 2023-10-16T14:36:01.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4c9effd700 1 -- 172.21.15.178:0/4248766502 <== mon.1 v2:172.21.15.187:3300/0 4 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4cb00043b0 con 0x55f4f229ae20 2023-10-16T14:36:01.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 10 client.25852 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:01.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 20 client.25852 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f22b0770 0x55f4f22b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 4 client.25852 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.173+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4f226ac10 con 0x55f4f22b0770 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.174+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f22b0770 0x55f4f22b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.174+0000 7f4c9effd700 10 client.25852 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:01.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.174+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f22b0770 0x55f4f22b2c30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fed18cf6e3cb310 server_cookie=1016e3d95d367b87 in_seq=0 out_seq=0 2023-10-16T14:36:01.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4c9effd700 1 -- 172.21.15.178:0/4248766502 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f4f226ac10 con 0x55f4f22b0770 2023-10-16T14:36:01.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4c9effd700 10 client.25852 handle_command_reply: tid=0 2023-10-16T14:36:01.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 client.25852 shutdown 2023-10-16T14:36:01.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 msgr2=0x55f4f22a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0012810 tx=0x7f4cb0012b20 comp rx=0 tx=0).stop 2023-10-16T14:36:01.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 shutdown_connections 2023-10-16T14:36:01.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f22b0770 0x55f4f22b2c30 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22a4c20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/4248766502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 >> 172.21.15.178:0/4248766502 conn(0x55f4f21c8dd0 msgr2=0x55f4f2295bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.179+0000 7f4c9effd700 0 client.25852 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:01.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.180+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 shutdown_connections 2023-10-16T14:36:01.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.180+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/4248766502 wait complete. 2023-10-16T14:36:01.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.180+0000 7f4cd93c1b80 20 client.25852 trim_cache size 0 max 16384 2023-10-16T14:36:01.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.207+0000 7f4cd93c1b80 1 Processor -- start 2023-10-16T14:36:01.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.207+0000 7f4cd93c1b80 1 -- start start 2023-10-16T14:36:01.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22ac160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f206e920 con 0x55f4f229b780 2023-10-16T14:36:01.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f229bc40 con 0x55f4f229c620 2023-10-16T14:36:01.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f2290c00 con 0x55f4f229ae20 2023-10-16T14:36:01.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbe7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38856/0 (socket says 172.21.15.178:38856) 2023-10-16T14:36:01.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46700/0 (socket says 172.21.15.178:46700) 2023-10-16T14:36:01.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbe7fc700 1 -- 172.21.15.178:0/828793472 learned_addr learned my addr 172.21.15.178:0/828793472 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:01.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/828793472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 msgr2=0x55f4f22acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/828793472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 msgr2=0x55f4f22ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.208+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.209+0000 7f4cbf7fe700 1 -- 172.21.15.178:0/828793472 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f2001c60 con 0x55f4f229c620 2023-10-16T14:36:01.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.209+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800b6c0 tx=0x7f4cb803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe383602bfe6fcc5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.209+0000 7f4ca67fc700 1 -- 172.21.15.178:0/828793472 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb800e070 con 0x55f4f229c620 2023-10-16T14:36:01.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.209+0000 7f4ca67fc700 1 -- 172.21.15.178:0/828793472 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb800bc30 con 0x55f4f229c620 2023-10-16T14:36:01.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.209+0000 7f4ca67fc700 1 -- 172.21.15.178:0/828793472 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb803a040 con 0x55f4f229c620 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/828793472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 msgr2=0x55f4f22b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800b6c0 tx=0x7f4cb803f000 comp rx=0 tx=0).stop 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/828793472 shutdown_connections 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229c620 0x55f4f22b2780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229ae20 0x55f4f22acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/828793472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229b780 0x55f4f22ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/828793472 >> 172.21.15.178:0/828793472 conn(0x55f4f21c8dd0 msgr2=0x55f4f22947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/828793472 shutdown_connections 2023-10-16T14:36:01.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.210+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/828793472 wait complete. 2023-10-16T14:36:01.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.211+0000 7f4cd93c1b80 1 -- start start 2023-10-16T14:36:01.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.211+0000 7f4cd93c1b80 1 Processor -- start 2023-10-16T14:36:01.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229b780 0x55f4f22a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 0x55f4f22ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f22abef0 con 0x55f4f229c620 2023-10-16T14:36:01.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f22ac060 con 0x55f4f229ae20 2023-10-16T14:36:01.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f22ac220 con 0x55f4f229b780 2023-10-16T14:36:01.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cd93c1b80 10 client.0 _ll_register_callbacks cb 0x55f4f1fe7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:01.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46704/0 (socket says 172.21.15.178:46704) 2023-10-16T14:36:01.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 -- 172.21.15.178:0/3581519306 learned_addr learned my addr 172.21.15.178:0/3581519306 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:01.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 0x55f4f22ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 -- 172.21.15.178:0/3581519306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229b780 msgr2=0x55f4f22a8990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:36:01.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229b780 0x55f4f22a8990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 -- 172.21.15.178:0/3581519306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 msgr2=0x55f4f22ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:01.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 0x55f4f22ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbeffd700 1 -- 172.21.15.178:0/3581519306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4f21ab3f0 con 0x55f4f229ae20 2023-10-16T14:36:01.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.212+0000 7f4cbf7fe700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 0x55f4f22ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:01.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.213+0000 7f4cbeffd700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55f4f229ab70 tx=0x7f4cb0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9522a26ad5221873 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:01.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.213+0000 7f4c9effd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:01.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00036d0 con 0x55f4f229ae20 2023-10-16T14:36:01.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4cb0000e40 con 0x55f4f229ae20 2023-10-16T14:36:01.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4f21e6e50 con 0x55f4f229ae20 2023-10-16T14:36:01.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb0004830 con 0x55f4f229ae20 2023-10-16T14:36:01.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4cd93c1b80 10 client.25871 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:01.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f4f203f250 con 0x55f4f229ae20 2023-10-16T14:36:01.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4f21e6e50 con 0x55f4f229ae20 2023-10-16T14:36:01.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f4cb0010430 con 0x55f4f229ae20 2023-10-16T14:36:01.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4cd93c1b80 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:01.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.214+0000 7f4cd93c1b80 10 client.25871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:01.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cd93c1b80 20 client.25871 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:01.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f2305a00 0x55f4f2307e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:01.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cd93c1b80 4 client.25871 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:01.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f4f21e6b70 con 0x55f4f2305a00 2023-10-16T14:36:01.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f2305a00 0x55f4f2307e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:01.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4c9effd700 10 client.25871 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:01.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.215+0000 7f4cbe7fc700 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f2305a00 0x55f4f2307e20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c6efce963ab781 server_cookie=f17bdd1a2ff6015a in_seq=0 out_seq=0 2023-10-16T14:36:01.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4c9effd700 1 -- 172.21.15.178:0/3581519306 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f4f21e6b70 con 0x55f4f2305a00 2023-10-16T14:36:01.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4c9effd700 10 client.25871 handle_command_reply: tid=0 2023-10-16T14:36:01.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 client.25871 shutdown 2023-10-16T14:36:01.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 msgr2=0x55f4f22a8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55f4f229ab70 tx=0x7f4cb0004750 comp rx=0 tx=0).stop 2023-10-16T14:36:01.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 shutdown_connections 2023-10-16T14:36:01.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55f4f2305a00 0x55f4f2307e20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f4f229c620 0x55f4f22ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f4f229ae20 0x55f4f22a8450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4c9effd700 0 client.25871 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:01.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 --2- 172.21.15.178:0/3581519306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4f229b780 0x55f4f22a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 >> 172.21.15.178:0/3581519306 conn(0x55f4f21c8dd0 msgr2=0x55f4f22acaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 shutdown_connections 2023-10-16T14:36:01.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.221+0000 7f4cd93c1b80 1 -- 172.21.15.178:0/3581519306 wait complete. 2023-10-16T14:36:01.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cd93c1b80 20 client.25871 trim_cache size 0 max 16384 2023-10-16T14:36:01.233 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:01.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 msgr2=0x7f4ca80a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 0x7f4ca80a3b10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8032000 tx=0x7f4cb8037040 comp rx=0 tx=0).stop 2023-10-16T14:36:01.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 msgr2=0x7f4cc01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:01.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e970 tx=0x7f4cb400ec80 comp rx=0 tx=0).stop 2023-10-16T14:36:01.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 shutdown_connections 2023-10-16T14:36:01.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f4ca80a1650 0x7f4ca80a3b10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc0102a40 0x7f4cc01a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cc0100ea0 0x7f4cc0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 --2- 172.21.15.178:0/3229242731 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f4cc0102080 0x7f4cc01993e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:01.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.222+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 >> 172.21.15.178:0/3229242731 conn(0x7f4cc00fc420 msgr2=0x7f4cc00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:01.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.223+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 shutdown_connections 2023-10-16T14:36:01.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:01.223+0000 7f4cc59ff700 1 -- 172.21.15.178:0/3229242731 wait complete. 2023-10-16T14:36:02.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:01 smithi178 ceph-mon[98495]: pgmap v408: 129 pgs: 129 active+clean; 84 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 106 op/s 2023-10-16T14:36:02.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:01 smithi186 ceph-mon[134892]: pgmap v408: 129 pgs: 129 active+clean; 84 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 106 op/s 2023-10-16T14:36:02.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:01 smithi187 ceph-mon[135297]: pgmap v408: 129 pgs: 129 active+clean; 84 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 106 op/s 2023-10-16T14:36:02.245 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:36:02.246 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:36:03.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 -- 172.21.15.178:0/1052422849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a61e0 msgr2=0x7f16100a6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 --2- 172.21.15.178:0/1052422849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a61e0 0x7f16100a6640 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f160c007510 tx=0x7f160c00af90 comp rx=0 tx=0).stop 2023-10-16T14:36:03.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 -- 172.21.15.178:0/1052422849 shutdown_connections 2023-10-16T14:36:03.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 --2- 172.21.15.178:0/1052422849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a6b80 0x7f16100ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 --2- 172.21.15.178:0/1052422849 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a4ff0 0x7f16100a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 --2- 172.21.15.178:0/1052422849 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a61e0 0x7f16100a6640 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 -- 172.21.15.178:0/1052422849 >> 172.21.15.178:0/1052422849 conn(0x7f16100a0540 msgr2=0x7f16100a29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 -- 172.21.15.178:0/1052422849 shutdown_connections 2023-10-16T14:36:03.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.068+0000 7f161df9d700 1 -- 172.21.15.178:0/1052422849 wait complete. 2023-10-16T14:36:03.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.070+0000 7f161df9d700 1 Processor -- start 2023-10-16T14:36:03.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.070+0000 7f161df9d700 1 -- start start 2023-10-16T14:36:03.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.070+0000 7f161df9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46718/0 (socket says 172.21.15.178:46718) 2023-10-16T14:36:03.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161cf9b700 1 -- 172.21.15.178:0/318803583 learned_addr learned my addr 172.21.15.178:0/318803583 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a61e0 0x7f16100a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 0x7f16100a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100a82a0 con 0x7f16100a6b80 2023-10-16T14:36:03.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16100a8410 con 0x7f16100a4ff0 2023-10-16T14:36:03.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1610146b30 con 0x7f16100a61e0 2023-10-16T14:36:03.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.071+0000 7f1617fff700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a61e0 0x7f16100a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161d79c700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 0x7f16100a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161cf9b700 1 -- 172.21.15.178:0/318803583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a61e0 msgr2=0x7f16100a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161cf9b700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a61e0 0x7f16100a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161cf9b700 1 -- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 msgr2=0x7f16100a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161cf9b700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 0x7f16100a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161cf9b700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160c0071c0 con 0x7f16100a4ff0 2023-10-16T14:36:03.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.072+0000 7f161d79c700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 0x7f16100a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:03.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.073+0000 7f161cf9b700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1604014300 tx=0x7f1604014610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.073+0000 7f1615ffb700 1 -- 172.21.15.178:0/318803583 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604017070 con 0x7f16100a4ff0 2023-10-16T14:36:03.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.073+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161013c750 con 0x7f16100a4ff0 2023-10-16T14:36:03.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.073+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161013cc80 con 0x7f16100a4ff0 2023-10-16T14:36:03.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.073+0000 7f1615ffb700 1 -- 172.21.15.178:0/318803583 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1604014e70 con 0x7f16100a4ff0 2023-10-16T14:36:03.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.074+0000 7f1615ffb700 1 -- 172.21.15.178:0/318803583 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604017070 con 0x7f16100a4ff0 2023-10-16T14:36:03.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.080+0000 7f1632b1eb80 1 Processor -- start 2023-10-16T14:36:03.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.080+0000 7f1632b1eb80 1 -- start start 2023-10-16T14:36:03.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.081+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 0x5655362d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.081+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.081+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x565536248cf0 con 0x5655362d4030 2023-10-16T14:36:03.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553622dce0 con 0x5655362d5880 2023-10-16T14:36:03.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553622b600 con 0x5655362d4990 2023-10-16T14:36:03.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1615ffb700 1 -- 172.21.15.178:0/318803583 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f16040127b0 con 0x7f16100a4ff0 2023-10-16T14:36:03.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1617fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1617fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46730/0 (socket says 172.21.15.178:46730) 2023-10-16T14:36:03.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f1617fff700 1 -- 172.21.15.178:0/1262188784 learned_addr learned my addr 172.21.15.178:0/1262188784 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.082+0000 7f161d79c700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 0x5655362d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161cf9b700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161d79c700 1 -- 172.21.15.178:0/1262188784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 msgr2=0x5655362d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161d79c700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161d79c700 1 -- 172.21.15.178:0/1262188784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 msgr2=0x5655362d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161d79c700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f161d79c700 1 -- 172.21.15.178:0/1262188784 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5655361e4470 con 0x5655362d4030 2023-10-16T14:36:03.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.083+0000 7f1617fff700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:03.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.084+0000 7f161d79c700 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 0x5655362d4450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1618066c10 tx=0x7f161806bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279ecb7cb608336d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.084+0000 7f1602ffd700 1 -- 172.21.15.178:0/1262188784 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618069a50 con 0x5655362d4030 2023-10-16T14:36:03.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.084+0000 7f1602ffd700 1 -- 172.21.15.178:0/1262188784 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1618073050 con 0x5655362d4030 2023-10-16T14:36:03.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.085+0000 7f1602ffd700 1 -- 172.21.15.178:0/1262188784 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16180783d0 con 0x5655362d4030 2023-10-16T14:36:03.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.085+0000 7f1615ffb700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 0x7f16080a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.085+0000 7f1615ffb700 1 -- 172.21.15.178:0/318803583 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f16040d40c0 con 0x7f16100a4ff0 2023-10-16T14:36:03.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f161cf9b700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 0x7f16080a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 -- 172.21.15.178:0/1262188784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 msgr2=0x5655362d4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 0x5655362d4450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1618066c10 tx=0x7f161806bb50 comp rx=0 tx=0).stop 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 -- 172.21.15.178:0/1262188784 shutdown_connections 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d5880 0x5655362d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d4030 0x5655362d4450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/1262188784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.086+0000 7f1632b1eb80 1 -- 172.21.15.178:0/1262188784 >> 172.21.15.178:0/1262188784 conn(0x565536201e50 msgr2=0x5655362d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.090+0000 7f161cf9b700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 0x7f16080a39f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f16040a1120 tx=0x7f1604000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.090+0000 7f1632b1eb80 1 -- 172.21.15.178:0/1262188784 shutdown_connections 2023-10-16T14:36:03.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.090+0000 7f1632b1eb80 1 -- 172.21.15.178:0/1262188784 wait complete. 2023-10-16T14:36:03.094 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.091+0000 7f1632b1eb80 1 -- start start 2023-10-16T14:36:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.092+0000 7f1632b1eb80 1 Processor -- start 2023-10-16T14:36:03.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f161d79c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f161d79c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362ddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52718/0 (socket says 172.21.15.178:52718) 2023-10-16T14:36:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f161d79c700 1 -- 172.21.15.178:0/4179010617 learned_addr learned my addr 172.21.15.178:0/4179010617 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1617fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x565536226ef0 con 0x5655362d5880 2023-10-16T14:36:03.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.093+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655361c3790 con 0x5655362d4990 2023-10-16T14:36:03.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.094+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655361c3430 con 0x5655362d4030 2023-10-16T14:36:03.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.094+0000 7f1632b1eb80 10 client.0 _ll_register_callbacks cb 0x565536020880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:03.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f161d79c700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f1617fff700 1 -- 172.21.15.178:0/4179010617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 msgr2=0x5655362ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f1617fff700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362ddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f1617fff700 1 -- 172.21.15.178:0/4179010617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 msgr2=0x5655362e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f1617fff700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f1617fff700 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16100add80 con 0x5655362d4990 2023-10-16T14:36:03.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.095+0000 7f161d79c700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:03.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.096+0000 7f1617fff700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362de1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f160c016040 tx=0x7f160c00c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=187cf78086c6a76a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.096+0000 7f15f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:03.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.096+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c00f040 con 0x5655362d4990 2023-10-16T14:36:03.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.096+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56553622bf70 con 0x5655362d4990 2023-10-16T14:36:03.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.096+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f160c00cc50 con 0x5655362d4990 2023-10-16T14:36:03.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f160c00ce10 con 0x5655362d4990 2023-10-16T14:36:03.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 10 client.25883 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:03.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f161806b570 con 0x5655362d4990 2023-10-16T14:36:03.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56553622bf70 con 0x5655362d4990 2023-10-16T14:36:03.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f160c0040b0 con 0x5655362d4990 2023-10-16T14:36:03.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 10 client.25883 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:03.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:03.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 20 client.25883 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:03.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5655362e9b80 0x5655362ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 4 client.25883 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:03.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.097+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5655362a3c90 con 0x5655362e9b80 2023-10-16T14:36:03.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.098+0000 7f161d79c700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5655362e9b80 0x5655362ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.098+0000 7f15f7fff700 10 client.25883 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.098+0000 7f161d79c700 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5655362e9b80 0x5655362ebfa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efde65e62471eeaa server_cookie=428b84b8213fd00b in_seq=0 out_seq=0 2023-10-16T14:36:03.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.100+0000 7f15f7fff700 1 -- 172.21.15.178:0/4179010617 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5655362a3c90 con 0x5655362e9b80 2023-10-16T14:36:03.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.100+0000 7f15f7fff700 10 client.25883 handle_command_reply: tid=0 2023-10-16T14:36:03.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.100+0000 7f1632b1eb80 1 client.25883 shutdown 2023-10-16T14:36:03.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 msgr2=0x5655362de1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362de1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f160c016040 tx=0x7f160c00c730 comp rx=0 tx=0).stop 2023-10-16T14:36:03.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 shutdown_connections 2023-10-16T14:36:03.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5655362e9b80 0x5655362ebfa0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/4179010617 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362de1f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 >> 172.21.15.178:0/4179010617 conn(0x565536201e50 msgr2=0x5655362cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f15f7fff700 0 client.25883 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.101+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 shutdown_connections 2023-10-16T14:36:03.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.102+0000 7f1632b1eb80 1 -- 172.21.15.178:0/4179010617 wait complete. 2023-10-16T14:36:03.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.102+0000 7f1632b1eb80 20 client.25883 trim_cache size 0 max 16384 2023-10-16T14:36:03.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 Processor -- start 2023-10-16T14:36:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 -- start start 2023-10-16T14:36:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362d84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655360a79a0 con 0x5655362d5880 2023-10-16T14:36:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655362d11a0 con 0x5655362d4990 2023-10-16T14:36:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.129+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5655362c9bd0 con 0x5655362d4030 2023-10-16T14:36:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362d84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38922/0 (socket says 172.21.15.178:38922) 2023-10-16T14:36:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 -- 172.21.15.178:0/961930169 learned_addr learned my addr 172.21.15.178:0/961930169 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f1617fff700 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 -- 172.21.15.178:0/961930169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 msgr2=0x5655362d84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362d84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 -- 172.21.15.178:0/961930169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 msgr2=0x5655362d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 -- 172.21.15.178:0/961930169 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565536051e60 con 0x5655362d5880 2023-10-16T14:36:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.130+0000 7f161d79c700 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f16180644a0 tx=0x7f1618066d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4ad306039dc06a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.131+0000 7f16037fe700 1 -- 172.21.15.178:0/961930169 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618079930 con 0x5655362d5880 2023-10-16T14:36:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.131+0000 7f16037fe700 1 -- 172.21.15.178:0/961930169 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1618091070 con 0x5655362d5880 2023-10-16T14:36:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 -- 172.21.15.178:0/961930169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 msgr2=0x5655362e51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f16180644a0 tx=0x7f1618066d80 comp rx=0 tx=0).stop 2023-10-16T14:36:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 -- 172.21.15.178:0/961930169 shutdown_connections 2023-10-16T14:36:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e51f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4030 0x5655362d84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/961930169 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4990 0x5655362d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 -- 172.21.15.178:0/961930169 >> 172.21.15.178:0/961930169 conn(0x565536201e50 msgr2=0x5655362ddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 -- 172.21.15.178:0/961930169 shutdown_connections 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.132+0000 7f1632b1eb80 1 -- 172.21.15.178:0/961930169 wait complete. 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.133+0000 7f1632b1eb80 1 -- start start 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.133+0000 7f1632b1eb80 1 Processor -- start 2023-10-16T14:36:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.133+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362e1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553634e9b0 con 0x5655362d5880 2023-10-16T14:36:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553634eaf0 con 0x5655362d4030 2023-10-16T14:36:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56553634ec30 con 0x5655362d4990 2023-10-16T14:36:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f161d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f161d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46754/0 (socket says 172.21.15.178:46754) 2023-10-16T14:36:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f161d79c700 1 -- 172.21.15.178:0/3646198009 learned_addr learned my addr 172.21.15.178:0/3646198009 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1632b1eb80 10 client.0 _ll_register_callbacks cb 0x7f160c005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f161d79c700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.134+0000 7f1617fff700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362e1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 -- 172.21.15.178:0/3646198009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 msgr2=0x5655362e1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362e1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 -- 172.21.15.178:0/3646198009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 msgr2=0x5655362e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 -- 172.21.15.178:0/3646198009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5655361e4470 con 0x5655362d4030 2023-10-16T14:36:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f161d79c700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1618000eb0 tx=0x7f1618086410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4195756af04aba27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.135+0000 7f15f7fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16180698d0 con 0x5655362d4030 2023-10-16T14:36:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1618073050 con 0x5655362d4030 2023-10-16T14:36:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1618069a30 con 0x5655362d4030 2023-10-16T14:36:03.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1618069c30 con 0x5655362d4030 2023-10-16T14:36:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 10 client.25889 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56553603ace0 con 0x5655362d4030 2023-10-16T14:36:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f16180a6080 con 0x5655362d4030 2023-10-16T14:36:03.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f161806b870 con 0x5655362d4030 2023-10-16T14:36:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 10 client.25889 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.136+0000 7f1632b1eb80 20 client.25889 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x565536350120 0x565536352540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f1632b1eb80 4 client.25889 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:36:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56553621fed0 con 0x565536350120 2023-10-16T14:36:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f1617fff700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x565536350120 0x565536352540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f15f7fff700 10 client.25889 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.137+0000 7f1617fff700 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x565536350120 0x565536352540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db5e2c1699e59a0 server_cookie=b8ae9a843b8d426e in_seq=0 out_seq=0 2023-10-16T14:36:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.140+0000 7f15f7fff700 1 -- 172.21.15.178:0/3646198009 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56553621fed0 con 0x565536350120 2023-10-16T14:36:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.140+0000 7f15f7fff700 10 client.25889 handle_command_reply: tid=0 2023-10-16T14:36:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.140+0000 7f1632b1eb80 1 client.25889 shutdown 2023-10-16T14:36:03.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.140+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 msgr2=0x5655362e50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.140+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1618000eb0 tx=0x7f1618086410 comp rx=0 tx=0).stop 2023-10-16T14:36:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 shutdown_connections 2023-10-16T14:36:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x565536350120 0x565536352540 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f15f7fff700 0 client.25889 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5655362d5880 0x5655362e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5655362d4030 0x5655362e50c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 --2- 172.21.15.178:0/3646198009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5655362d4990 0x5655362e1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 >> 172.21.15.178:0/3646198009 conn(0x565536201e50 msgr2=0x5655362ddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.141+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 shutdown_connections 2023-10-16T14:36:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.142+0000 7f1632b1eb80 1 -- 172.21.15.178:0/3646198009 wait complete. 2023-10-16T14:36:03.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.142+0000 7f1632b1eb80 20 client.25889 trim_cache size 0 max 16384 2023-10-16T14:36:03.153 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:36:03.153 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:36:03.154 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "09e3c143-0c8e-4ec5-af09-344af47c71ac", 2023-10-16T14:36:03.154 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:36:03.154 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:36:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 msgr2=0x7f16080a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 0x7f16080a39f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f16040a1120 tx=0x7f1604000bc0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 msgr2=0x7f16100a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1604014300 tx=0x7f1604014610 comp rx=0 tx=0).stop 2023-10-16T14:36:03.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 shutdown_connections 2023-10-16T14:36:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f16080a1530 0x7f16080a39f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16100a6b80 0x7f16100a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f16100a4ff0 0x7f16100a8f80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 --2- 172.21.15.178:0/318803583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16100a61e0 0x7f16100a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.143+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 >> 172.21.15.178:0/318803583 conn(0x7f16100a0540 msgr2=0x7f16100ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.144+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 shutdown_connections 2023-10-16T14:36:03.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.144+0000 7f161df9d700 1 -- 172.21.15.178:0/318803583 wait complete. 2023-10-16T14:36:03.176 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:36:03.399 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.396+0000 7fe6da1ff700 1 -- 172.21.15.178:0/1675370712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d4102a10 msgr2=0x7fe6d4109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.399 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.396+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/1675370712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d4102a10 0x7fe6d4109330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00b0a0 tx=0x7fe6cc00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 -- 172.21.15.178:0/1675370712 shutdown_connections 2023-10-16T14:36:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/1675370712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d4102a10 0x7fe6d4109330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/1675370712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4100e70 0x7fe6d4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/1675370712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4102050 0x7fe6d41024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 -- 172.21.15.178:0/1675370712 >> 172.21.15.178:0/1675370712 conn(0x7fe6d40fc3f0 msgr2=0x7fe6d40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 -- 172.21.15.178:0/1675370712 shutdown_connections 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.397+0000 7fe6da1ff700 1 -- 172.21.15.178:0/1675370712 wait complete. 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.398+0000 7fe6da1ff700 1 Processor -- start 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.398+0000 7fe6da1ff700 1 -- start start 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4102a10 0x7fe6d41994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 0x7fe6d41a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d41a0ec0 con 0x7fe6d4102a10 2023-10-16T14:36:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d41a1000 con 0x7fe6d4100e70 2023-10-16T14:36:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6da1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d41a1190 con 0x7fe6d419e500 2023-10-16T14:36:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46770/0 (socket says 172.21.15.178:46770) 2023-10-16T14:36:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d37fe700 1 -- 172.21.15.178:0/125429611 learned_addr learned my addr 172.21.15.178:0/125429611 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 0x7fe6d41a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4102a10 0x7fe6d41994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 -- 172.21.15.178:0/125429611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 msgr2=0x7fe6d4198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 -- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4102a10 msgr2=0x7fe6d41994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4102a10 0x7fe6d41994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d3fff700 1 -- 172.21.15.178:0/125429611 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6cc009d00 con 0x7fe6d419e500 2023-10-16T14:36:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.399+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 0x7fe6d41a08f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009fd0 tx=0x7fe6cc00bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6d0ff9700 1 -- 172.21.15.178:0/125429611 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc00e040 con 0x7fe6d419e500 2023-10-16T14:36:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6d0ff9700 1 -- 172.21.15.178:0/125429611 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6cc004e60 con 0x7fe6d419e500 2023-10-16T14:36:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6d0ff9700 1 -- 172.21.15.178:0/125429611 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0052a0 con 0x7fe6d419e500 2023-10-16T14:36:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d41a1410 con 0x7fe6d419e500 2023-10-16T14:36:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.400+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d41a18c0 con 0x7fe6d419e500 2023-10-16T14:36:03.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.403+0000 7fe6d0ff9700 1 -- 172.21.15.178:0/125429611 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe6cc01d470 con 0x7fe6d419e500 2023-10-16T14:36:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.405+0000 7fe6edc16b80 1 Processor -- start 2023-10-16T14:36:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d0ff9700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 0x7fe6bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 -- start start 2023-10-16T14:36:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d0ff9700 1 -- 172.21.15.178:0/125429611 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe6cc0ce6f0 con 0x7fe6d419e500 2023-10-16T14:36:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679ebf90 0x5591679ec850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 0x7fe6bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559167957c70 con 0x5591679ece30 2023-10-16T14:36:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916793cc60 con 0x5591679eb630 2023-10-16T14:36:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55916793a580 con 0x5591679ebf90 2023-10-16T14:36:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d37fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46784/0 (socket says 172.21.15.178:46784) 2023-10-16T14:36:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d37fe700 1 -- 172.21.15.178:0/1851475510 learned_addr learned my addr 172.21.15.178:0/1851475510 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679ebf90 0x5591679ec850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.406+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 -- 172.21.15.178:0/1851475510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679ebf90 msgr2=0x5591679ec850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679ebf90 0x5591679ec850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 -- 172.21.15.178:0/1851475510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 msgr2=0x5591679ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 -- 172.21.15.178:0/1851475510 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591678f33f0 con 0x5591679eb630 2023-10-16T14:36:03.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011650 tx=0x7fe6c4011960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3957d58d85abe185 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6b9ffb700 1 -- 172.21.15.178:0/1851475510 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4004a30 con 0x5591679eb630 2023-10-16T14:36:03.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 0x7fe6bc0a39c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80093b0 tx=0x7fe6c8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6b9ffb700 1 -- 172.21.15.178:0/1851475510 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c40054e0 con 0x5591679eb630 2023-10-16T14:36:03.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.407+0000 7fe6b9ffb700 1 -- 172.21.15.178:0/1851475510 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400f940 con 0x5591679eb630 2023-10-16T14:36:03.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1851475510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 msgr2=0x5591679eba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4011650 tx=0x7fe6c4011960 comp rx=0 tx=0).stop 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1851475510 shutdown_connections 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679eb630 0x5591679eba50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1851475510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679ebf90 0x5591679ec850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1851475510 >> 172.21.15.178:0/1851475510 conn(0x559167910dd0 msgr2=0x5591679ddfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1851475510 shutdown_connections 2023-10-16T14:36:03.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.408+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1851475510 wait complete. 2023-10-16T14:36:03.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 -- start start 2023-10-16T14:36:03.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 Processor -- start 2023-10-16T14:36:03.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679fca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x559167935e70 con 0x5591679ece30 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591678d2710 con 0x5591679ebf90 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.409+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591678d23b0 con 0x5591679eb630 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6edc16b80 10 client.0 _ll_register_callbacks cb 0x55916772f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:03.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46796/0 (socket says 172.21.15.178:46796) 2023-10-16T14:36:03.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d37fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/3680164735 learned_addr learned my addr 172.21.15.178:0/3680164735 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/3680164735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 msgr2=0x5591679f5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/3680164735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 msgr2=0x5591679fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.410+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/3680164735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c4009e10 con 0x5591679ebf90 2023-10-16T14:36:03.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80186d0 tx=0x7fe6c8009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64a7fab90e1165f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:03.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8016600 con 0x5591679ebf90 2023-10-16T14:36:03.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe6c8016760 con 0x5591679ebf90 2023-10-16T14:36:03.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55916793aef0 con 0x5591679ebf90 2023-10-16T14:36:03.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c8016920 con 0x5591679ebf90 2023-10-16T14:36:03.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6edc16b80 10 client.25898 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:03.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559167760de0 con 0x5591679ebf90 2023-10-16T14:36:03.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.411+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55916793aef0 con 0x5591679ebf90 2023-10-16T14:36:03.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe6c8021070 con 0x5591679ebf90 2023-10-16T14:36:03.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 10 client.25898 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:03.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 10 client.25898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:03.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 20 client.25898 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:03.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a01440 0x559167a03860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 4 client.25898 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:03.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591679b2c10 con 0x559167a01440 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a01440 0x559167a03860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a01440 0x559167a03860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5f792ace9ac8a8 server_cookie=11a70a8614abf07e in_seq=0 out_seq=0 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.412+0000 7fe6b2ffd700 10 client.25898 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.414+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/3680164735 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5591679b2c10 con 0x559167a01440 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.414+0000 7fe6b2ffd700 10 client.25898 handle_command_reply: tid=0 2023-10-16T14:36:03.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 client.25898 shutdown 2023-10-16T14:36:03.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 msgr2=0x5591679f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80186d0 tx=0x7fe6c8009190 comp rx=0 tx=0).stop 2023-10-16T14:36:03.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 shutdown_connections 2023-10-16T14:36:03.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a01440 0x559167a03860 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6b2ffd700 0 client.25898 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/3680164735 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679f59d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 >> 172.21.15.178:0/3680164735 conn(0x559167910dd0 msgr2=0x5591679ddb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 shutdown_connections 2023-10-16T14:36:03.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 1 -- 172.21.15.178:0/3680164735 wait complete. 2023-10-16T14:36:03.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.415+0000 7fe6edc16b80 20 client.25898 trim_cache size 0 max 16384 2023-10-16T14:36:03.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 Processor -- start 2023-10-16T14:36:03.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 -- start start 2023-10-16T14:36:03.437 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591679f5060 con 0x5591679ece30 2023-10-16T14:36:03.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591677b6920 con 0x5591679ebf90 2023-10-16T14:36:03.438 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.434+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591679dec00 con 0x5591679eb630 2023-10-16T14:36:03.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679fc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:46804/0 (socket says 172.21.15.178:46804) 2023-10-16T14:36:03.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/2118265507 learned_addr learned my addr 172.21.15.178:0/2118265507 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.439 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d3fff700 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 -- 172.21.15.178:0/2118265507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 msgr2=0x5591679fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 -- 172.21.15.178:0/2118265507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 msgr2=0x5591679ddb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ddb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 -- 172.21.15.178:0/2118265507 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559167749c60 con 0x5591679eb630 2023-10-16T14:36:03.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.435+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f3210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4004690 tx=0x7fe6c40095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec064084ecd679af server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.436+0000 7fe6ba7fc700 1 -- 172.21.15.178:0/2118265507 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c402b070 con 0x5591679eb630 2023-10-16T14:36:03.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.436+0000 7fe6ba7fc700 1 -- 172.21.15.178:0/2118265507 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c402f070 con 0x5591679eb630 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.436+0000 7fe6ba7fc700 1 -- 172.21.15.178:0/2118265507 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c402ba00 con 0x5591679eb630 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 -- 172.21.15.178:0/2118265507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 msgr2=0x5591679f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f3210 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4004690 tx=0x7fe6c40095c0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 -- 172.21.15.178:0/2118265507 shutdown_connections 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ece30 0x5591679ddb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679f3210 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/2118265507 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ebf90 0x5591679fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 -- 172.21.15.178:0/2118265507 >> 172.21.15.178:0/2118265507 conn(0x559167910dd0 msgr2=0x5591679f4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 -- 172.21.15.178:0/2118265507 shutdown_connections 2023-10-16T14:36:03.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.437+0000 7fe6edc16b80 1 -- 172.21.15.178:0/2118265507 wait complete. 2023-10-16T14:36:03.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.438+0000 7fe6edc16b80 1 -- start start 2023-10-16T14:36:03.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.438+0000 7fe6edc16b80 1 Processor -- start 2023-10-16T14:36:03.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.438+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679fc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ece30 0x5591679fb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591679fcd40 con 0x5591679ebf90 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591679fb5b0 con 0x5591679ece30 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591679fb770 con 0x5591679eb630 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6edc16b80 10 client.0 _ll_register_callbacks cb 0x7fe6c8013c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38990/0 (socket says 172.21.15.178:38990) 2023-10-16T14:36:03.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/1442199114 learned_addr learned my addr 172.21.15.178:0/1442199114 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:03.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679fc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/1442199114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 msgr2=0x5591679fc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:03.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679fc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ece30 msgr2=0x5591679fb070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:36:03.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ece30 0x5591679fb070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.439+0000 7fe6d2ffd700 1 -- 172.21.15.178:0/1442199114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591678f33f0 con 0x5591679ebf90 2023-10-16T14:36:03.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6d2ffd700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8022600 tx=0x7fe6c8022630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e8406c421d92d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:03.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:36:03.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8016800 con 0x5591679ebf90 2023-10-16T14:36:03.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe6c8016960 con 0x5591679ebf90 2023-10-16T14:36:03.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe6c40054e0 con 0x5591679ebf90 2023-10-16T14:36:03.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6c8016b20 con 0x5591679ebf90 2023-10-16T14:36:03.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6edc16b80 10 client.15852 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:03.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559167787250 con 0x5591679ebf90 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6c40054e0 con 0x5591679ebf90 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe6c8018700 con 0x5591679ebf90 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.440+0000 7fe6edc16b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6edc16b80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6edc16b80 20 client.15852 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:03.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a568d0 0x559167a58cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:03.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6edc16b80 4 client.15852 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:36:03.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55916792ee50 con 0x559167a568d0 2023-10-16T14:36:03.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a568d0 0x559167a58cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:03.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6d37fe700 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a568d0 0x559167a58cc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c997f0df06311cbc server_cookie=f52666878394c744 in_seq=0 out_seq=0 2023-10-16T14:36:03.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.441+0000 7fe6b2ffd700 10 client.15852 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.443+0000 7fe6b2ffd700 1 -- 172.21.15.178:0/1442199114 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x55916792ee50 con 0x559167a568d0 2023-10-16T14:36:03.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.443+0000 7fe6b2ffd700 10 client.15852 handle_command_reply: tid=0 2023-10-16T14:36:03.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.443+0000 7fe6edc16b80 1 client.15852 shutdown 2023-10-16T14:36:03.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 msgr2=0x5591679fab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8022600 tx=0x7fe6c8022630 comp rx=0 tx=0).stop 2023-10-16T14:36:03.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 shutdown_connections 2023-10-16T14:36:03.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x559167a568d0 0x559167a58cc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5591679ece30 0x5591679fb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591679eb630 0x5591679fc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6b2ffd700 0 client.15852 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:03.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 --2- 172.21.15.178:0/1442199114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5591679ebf90 0x5591679fab30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 >> 172.21.15.178:0/1442199114 conn(0x559167910dd0 msgr2=0x5591679efda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 shutdown_connections 2023-10-16T14:36:03.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 1 -- 172.21.15.178:0/1442199114 wait complete. 2023-10-16T14:36:03.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.444+0000 7fe6edc16b80 20 client.15852 trim_cache size 0 max 16384 2023-10-16T14:36:03.458 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:36:03.458 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "scrub active (3111 inodes in the stack)", 2023-10-16T14:36:03.458 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:36:03.458 INFO:teuthology.orchestra.run.smithi178.stdout: "09e3c143-0c8e-4ec5-af09-344af47c71ac": { 2023-10-16T14:36:03.459 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:36:03.459 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "09e3c143-0c8e-4ec5-af09-344af47c71ac", 2023-10-16T14:36:03.459 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:36:03.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:36:03.459 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:36:03.460 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:36:03.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 msgr2=0x7fe6bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 0x7fe6bc0a39c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80093b0 tx=0x7fe6c8008040 comp rx=0 tx=0).stop 2023-10-16T14:36:03.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 msgr2=0x7fe6d41a08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:03.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 0x7fe6d41a08f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009fd0 tx=0x7fe6cc00bce0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 shutdown_connections 2023-10-16T14:36:03.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe6bc0a1500 0x7fe6bc0a39c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6d419e500 0x7fe6d41a08f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe6d4100e70 0x7fe6d4198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 --2- 172.21.15.178:0/125429611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6d4102a10 0x7fe6d41994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:03.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 >> 172.21.15.178:0/125429611 conn(0x7fe6d40fc3f0 msgr2=0x7fe6d40fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:03.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 shutdown_connections 2023-10-16T14:36:03.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:03.445+0000 7fe6da1ff700 1 -- 172.21.15.178:0/125429611 wait complete. 2023-10-16T14:36:03.466 INFO:tasks.cephfs.filesystem:scrub status for tag:09e3c143-0c8e-4ec5-af09-344af47c71ac - {'path': '/', 'tag': '09e3c143-0c8e-4ec5-af09-344af47c71ac', 'options': 'recursive,force'} 2023-10-16T14:36:04.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:03 smithi178 ceph-mon[98495]: pgmap v409: 129 pgs: 129 active+clean; 84 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 86 op/s 2023-10-16T14:36:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:03 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:03 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:03 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:04.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:03 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:03 smithi186 ceph-mon[134892]: pgmap v409: 129 pgs: 129 active+clean; 84 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 86 op/s 2023-10-16T14:36:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:03 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:03 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:03 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:04.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:03 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:03 smithi187 ceph-mon[135297]: pgmap v409: 129 pgs: 129 active+clean; 84 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 86 op/s 2023-10-16T14:36:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:03 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:03 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:03 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:04.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:03 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:36:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:05.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:36:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:36:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:05.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:05.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:06.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:05 smithi178 ceph-mon[98495]: pgmap v410: 129 pgs: 129 active+clean; 100 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2023-10-16T14:36:06.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:05 smithi186 ceph-mon[134892]: pgmap v410: 129 pgs: 129 active+clean; 100 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2023-10-16T14:36:06.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:05 smithi187 ceph-mon[135297]: pgmap v410: 129 pgs: 129 active+clean; 100 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 145 op/s 2023-10-16T14:36:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:07 smithi178 ceph-mon[98495]: pgmap v411: 129 pgs: 129 active+clean; 120 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2023-10-16T14:36:08.082 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:07 smithi186 ceph-mon[134892]: pgmap v411: 129 pgs: 129 active+clean; 120 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2023-10-16T14:36:08.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:07 smithi187 ceph-mon[135297]: pgmap v411: 129 pgs: 129 active+clean; 120 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 256 op/s 2023-10-16T14:36:10.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2023-10-16T14:36:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:36:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2023-10-16T14:36:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:36:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 349 op/s 2023-10-16T14:36:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:36:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:10.973 INFO:tasks.workunit.client.0.smithi178.stdout: 56 225303 570 161612 459 74190 303 2023-10-16T14:36:11.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:10 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:10 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:10 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:36:11.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:10 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:36:11.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:10 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:10 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:10 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:10 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:36:11.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:10 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:36:11.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:10 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:11.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:10 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:10 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:10 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:36:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:10 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:36:11.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:10 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:36:12.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:11 smithi178 ceph-mon[98495]: pgmap v413: 129 pgs: 129 active+clean; 199 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2023-10-16T14:36:12.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:11 smithi186 ceph-mon[134892]: pgmap v413: 129 pgs: 129 active+clean; 199 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2023-10-16T14:36:12.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:11 smithi187 ceph-mon[135297]: pgmap v413: 129 pgs: 129 active+clean; 199 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 605 op/s 2023-10-16T14:36:14.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: pgmap v414: 129 pgs: 129 active+clean; 219 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 685 op/s 2023-10-16T14:36:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.066 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.068 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.069 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.069 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.069 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.069 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.070 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.071 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.073 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.073 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.073 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.073 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.074 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.076 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.076 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.076 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.076 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.076 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.077 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.078 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.079 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.079 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.079 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.079 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.079 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.082 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:14.083 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:13 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.084 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: pgmap v414: 129 pgs: 129 active+clean; 219 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 685 op/s 2023-10-16T14:36:14.084 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.084 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.085 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.085 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.085 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.085 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.085 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.086 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.086 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.086 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.086 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.086 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.087 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.088 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.089 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.089 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.089 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.089 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.089 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.090 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.091 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.092 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.092 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.092 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.092 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.092 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.093 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.093 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.093 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.093 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.093 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.094 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.094 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.094 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.094 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.094 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.095 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.096 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.096 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.096 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.096 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.096 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.097 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.098 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.098 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.098 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.098 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.098 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.099 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.099 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.099 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.099 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.099 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.100 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.101 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.101 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.101 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.101 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.101 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.102 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.102 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.102 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.102 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.102 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.103 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.103 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.103 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.103 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.103 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.104 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.104 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.104 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.104 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.104 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.105 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.106 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.106 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.106 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.106 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.106 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.107 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.107 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.107 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.107 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.107 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.108 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.108 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.108 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.108 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.108 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.109 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.110 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.110 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.110 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.110 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.110 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.111 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.112 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.112 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.112 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.112 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.113 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.113 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.113 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:14.113 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:13 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: pgmap v414: 129 pgs: 129 active+clean; 219 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 685 op/s 2023-10-16T14:36:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.206 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.206 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:14.206 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:14.206 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:13 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:15 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:16.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:15 smithi178 ceph-mon[98495]: pgmap v415: 129 pgs: 129 active+clean; 224 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2023-10-16T14:36:16.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:15 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:15 smithi186 ceph-mon[134892]: pgmap v415: 129 pgs: 129 active+clean; 224 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2023-10-16T14:36:16.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:15 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:16.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:15 smithi187 ceph-mon[135297]: pgmap v415: 129 pgs: 129 active+clean; 224 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 703 op/s 2023-10-16T14:36:18.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:17 smithi178 ceph-mon[98495]: pgmap v416: 129 pgs: 129 active+clean; 228 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 661 op/s 2023-10-16T14:36:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:17 smithi186 ceph-mon[134892]: pgmap v416: 129 pgs: 129 active+clean; 228 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 661 op/s 2023-10-16T14:36:18.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:17 smithi187 ceph-mon[135297]: pgmap v416: 129 pgs: 129 active+clean; 228 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 661 op/s 2023-10-16T14:36:20.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:19 smithi178 ceph-mon[98495]: pgmap v417: 129 pgs: 129 active+clean; 229 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 556 op/s 2023-10-16T14:36:20.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:19 smithi186 ceph-mon[134892]: pgmap v417: 129 pgs: 129 active+clean; 229 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 556 op/s 2023-10-16T14:36:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:19 smithi187 ceph-mon[135297]: pgmap v417: 129 pgs: 129 active+clean; 229 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 556 op/s 2023-10-16T14:36:20.982 INFO:tasks.workunit.client.0.smithi178.stdout: 74 527824 1119 367120 906 149054 1318 2023-10-16T14:36:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:21 smithi178 ceph-mon[98495]: pgmap v418: 129 pgs: 129 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2023-10-16T14:36:21.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:21 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:21 smithi186 ceph-mon[134892]: pgmap v418: 129 pgs: 129 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2023-10-16T14:36:21.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:21 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:21.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:21 smithi187 ceph-mon[135297]: pgmap v418: 129 pgs: 129 active+clean; 234 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.7 MiB/s wr, 477 op/s 2023-10-16T14:36:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:21 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:22.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:22 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:22.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:22 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:22.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:22 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:22.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:22 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:22.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:22 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:22.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:22 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:23.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:23 smithi178 ceph-mon[98495]: pgmap v419: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2023-10-16T14:36:23.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:23 smithi186 ceph-mon[134892]: pgmap v419: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2023-10-16T14:36:23.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:23 smithi187 ceph-mon[135297]: pgmap v419: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 220 op/s 2023-10-16T14:36:26.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:25 smithi178 ceph-mon[98495]: pgmap v420: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-10-16T14:36:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:25 smithi186 ceph-mon[134892]: pgmap v420: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-10-16T14:36:26.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:25 smithi187 ceph-mon[135297]: pgmap v420: 129 pgs: 129 active+clean; 236 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2023-10-16T14:36:28.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:27 smithi178 ceph-mon[98495]: pgmap v421: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-10-16T14:36:28.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:27 smithi186 ceph-mon[134892]: pgmap v421: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-10-16T14:36:28.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:27 smithi187 ceph-mon[135297]: pgmap v421: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 162 op/s 2023-10-16T14:36:30.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:29 smithi178 ceph-mon[98495]: pgmap v422: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2023-10-16T14:36:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:29 smithi186 ceph-mon[134892]: pgmap v422: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2023-10-16T14:36:30.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:29 smithi187 ceph-mon[135297]: pgmap v422: 129 pgs: 129 active+clean; 244 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2023-10-16T14:36:30.977 INFO:tasks.workunit.client.0.smithi178.stdout: 87 243161 821 166603 638 82038 690 2023-10-16T14:36:31.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:31 smithi178 ceph-mon[98495]: pgmap v423: 129 pgs: 129 active+clean; 291 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 343 op/s 2023-10-16T14:36:31.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:31 smithi186 ceph-mon[134892]: pgmap v423: 129 pgs: 129 active+clean; 291 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 343 op/s 2023-10-16T14:36:31.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:31 smithi187 ceph-mon[135297]: pgmap v423: 129 pgs: 129 active+clean; 291 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.8 MiB/s wr, 343 op/s 2023-10-16T14:36:33.467 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:36:34.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:33 smithi178 ceph-mon[98495]: pgmap v424: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 405 op/s 2023-10-16T14:36:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:33 smithi186 ceph-mon[134892]: pgmap v424: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 405 op/s 2023-10-16T14:36:34.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:33 smithi187 ceph-mon[135297]: pgmap v424: 129 pgs: 129 active+clean; 308 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 405 op/s 2023-10-16T14:36:36.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:35 smithi178 ceph-mon[98495]: pgmap v425: 129 pgs: 129 active+clean; 321 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2023-10-16T14:36:36.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:35 smithi186 ceph-mon[134892]: pgmap v425: 129 pgs: 129 active+clean; 321 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2023-10-16T14:36:36.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.084+0000 7f70d3fff700 1 --2- 172.21.15.178:0/4140916515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d4104c90 0x7f70d410b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.084+0000 7f70d9fff700 1 -- 172.21.15.178:0/4140916515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4103100 msgr2=0x7f70d4103520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.084+0000 7f70d9fff700 1 --2- 172.21.15.178:0/4140916515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4103100 0x7f70d4103520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f70c80099c0 tx=0x7f70c8009cd0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.086+0000 7f70d9fff700 1 -- 172.21.15.178:0/4140916515 shutdown_connections 2023-10-16T14:36:36.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.086+0000 7f70d9fff700 1 --2- 172.21.15.178:0/4140916515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d4104c90 0x7f70d410b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.086+0000 7f70d9fff700 1 --2- 172.21.15.178:0/4140916515 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4103100 0x7f70d4103520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.086+0000 7f70d9fff700 1 --2- 172.21.15.178:0/4140916515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d41042f0 0x7f70d4104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.086+0000 7f70d9fff700 1 -- 172.21.15.178:0/4140916515 >> 172.21.15.178:0/4140916515 conn(0x7f70d40fe630 msgr2=0x7f70d4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.087+0000 7f70d9fff700 1 -- 172.21.15.178:0/4140916515 shutdown_connections 2023-10-16T14:36:36.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.087+0000 7f70d9fff700 1 -- 172.21.15.178:0/4140916515 wait complete. 2023-10-16T14:36:36.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.092+0000 7f70d9fff700 1 Processor -- start 2023-10-16T14:36:36.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.094+0000 7f70d9fff700 1 -- start start 2023-10-16T14:36:36.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.094+0000 7f70d9fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 0x7f70d41948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.095+0000 7f70d9fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 0x7f70d4194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.095+0000 7f70d9fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 0x7f70d419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.096+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 0x7f70d419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.096+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 0x7f70d419bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56254/0 (socket says 172.21.15.178:56254) 2023-10-16T14:36:36.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.096+0000 7f70d3fff700 1 -- 172.21.15.178:0/451125949 learned_addr learned my addr 172.21.15.178:0/451125949 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.096+0000 7f70d37fe700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 0x7f70d41948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.096+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 0x7f70d4194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.097+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419c4d0 con 0x7f70d41042f0 2023-10-16T14:36:36.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.097+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419c640 con 0x7f70d4104c90 2023-10-16T14:36:36.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.097+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d419c7b0 con 0x7f70d4103100 2023-10-16T14:36:36.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d37fe700 1 -- 172.21.15.178:0/451125949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 msgr2=0x7f70d419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.102 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d37fe700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 0x7f70d419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d37fe700 1 -- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 msgr2=0x7f70d4194e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d37fe700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 0x7f70d4194e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d37fe700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80095f0 con 0x7f70d4103100 2023-10-16T14:36:36.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.098+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 0x7f70d4194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:36.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.099+0000 7f70d37fe700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 0x7f70d41948e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f70c80098e0 tx=0x7f70c8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.099+0000 7f70d0ff9700 1 -- 172.21.15.178:0/451125949 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c801d070 con 0x7f70d4103100 2023-10-16T14:36:36.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.099+0000 7f70d0ff9700 1 -- 172.21.15.178:0/451125949 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70c80040b0 con 0x7f70d4103100 2023-10-16T14:36:36.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.100+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d419ca30 con 0x7f70d4103100 2023-10-16T14:36:36.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.100+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d419cf00 con 0x7f70d4103100 2023-10-16T14:36:36.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.100+0000 7f70d0ff9700 1 -- 172.21.15.178:0/451125949 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8022e10 con 0x7f70d4103100 2023-10-16T14:36:36.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.108+0000 7f70d0ff9700 1 -- 172.21.15.178:0/451125949 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f70c8003bf0 con 0x7f70d4103100 2023-10-16T14:36:36.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.110+0000 7f70ed852b80 1 Processor -- start 2023-10-16T14:36:36.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.111+0000 7f70ed852b80 1 -- start start 2023-10-16T14:36:36.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.112+0000 7f70d0ff9700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 0x7f70bc0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.113+0000 7f70d3fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 0x7f70bc0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.114+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b784e20 0x55891b785240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.114+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.115+0000 7f70d0ff9700 1 -- 172.21.15.178:0/451125949 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f70c80ce430 con 0x7f70d4103100 2023-10-16T14:36:36.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.115+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b786aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.116+0000 7f70ed852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b6f9c70 con 0x55891b786620 2023-10-16T14:36:36.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.116+0000 7f70ed852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b6dec60 con 0x55891b785780 2023-10-16T14:36:36.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.116+0000 7f70ed852b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b6dc580 con 0x55891b784e20 2023-10-16T14:36:36.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.117+0000 7f70d3fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 0x7f70bc0a3a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e350 tx=0x7f70cc009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.117+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b784e20 0x55891b785240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.116+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.117+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56260/0 (socket says 172.21.15.178:56260) 2023-10-16T14:36:36.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.117+0000 7f70d2ffd700 1 -- 172.21.15.178:0/2630446127 learned_addr learned my addr 172.21.15.178:0/2630446127 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.117+0000 7f70d3fff700 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b786aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.118+0000 7f70d3fff700 1 -- 172.21.15.178:0/2630446127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b784e20 msgr2=0x55891b785240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.118+0000 7f70d3fff700 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b784e20 0x55891b785240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.118+0000 7f70d3fff700 1 -- 172.21.15.178:0/2630446127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 msgr2=0x55891b786040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.118+0000 7f70d3fff700 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.118+0000 7f70d3fff700 1 -- 172.21.15.178:0/2630446127 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891b6953f0 con 0x55891b786620 2023-10-16T14:36:36.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.119+0000 7f70d3fff700 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b786aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007ae0 tx=0x7f70cc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4765f08bcd903aa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.119+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:36.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.119+0000 7f70b9ffb700 1 -- 172.21.15.178:0/2630446127 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00ee60 con 0x55891b786620 2023-10-16T14:36:36.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.120+0000 7f70b9ffb700 1 -- 172.21.15.178:0/2630446127 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70cc015d80 con 0x55891b786620 2023-10-16T14:36:36.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.120+0000 7f70b9ffb700 1 -- 172.21.15.178:0/2630446127 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc020e80 con 0x55891b786620 2023-10-16T14:36:36.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.121+0000 7f70ed852b80 1 -- 172.21.15.178:0/2630446127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 msgr2=0x55891b786aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.121+0000 7f70ed852b80 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b786aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007ae0 tx=0x7f70cc011000 comp rx=0 tx=0).stop 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.125+0000 7f70ed852b80 1 -- 172.21.15.178:0/2630446127 shutdown_connections 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.125+0000 7f70ed852b80 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b786aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.125+0000 7f70ed852b80 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b784e20 0x55891b785240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.125+0000 7f70ed852b80 1 --2- 172.21.15.178:0/2630446127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b786040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.125+0000 7f70ed852b80 1 -- 172.21.15.178:0/2630446127 >> 172.21.15.178:0/2630446127 conn(0x55891b6b2dd0 msgr2=0x55891b77ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.126+0000 7f70ed852b80 1 -- 172.21.15.178:0/2630446127 shutdown_connections 2023-10-16T14:36:36.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.126+0000 7f70ed852b80 1 -- 172.21.15.178:0/2630446127 wait complete. 2023-10-16T14:36:36.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.128+0000 7f70ed852b80 1 -- start start 2023-10-16T14:36:36.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.130+0000 7f70ed852b80 1 Processor -- start 2023-10-16T14:36:36.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.135+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b78ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.135+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.136+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b796230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.136+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.136+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50480/0 (socket says 172.21.15.178:50480) 2023-10-16T14:36:36.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b796230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 -- 172.21.15.178:0/3715227846 learned_addr learned my addr 172.21.15.178:0/3715227846 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b6d7e70 con 0x55891b786620 2023-10-16T14:36:36.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b674710 con 0x55891b784e20 2023-10-16T14:36:36.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b6743b0 con 0x55891b785780 2023-10-16T14:36:36.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d37fe700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b78ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70ed852b80 10 client.0 _ll_register_callbacks cb 0x55891b4d1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:36.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 -- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 msgr2=0x55891b78ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b78ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 -- 172.21.15.178:0/3715227846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 msgr2=0x55891b796230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b796230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d2ffd700 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c400ad00 con 0x55891b785780 2023-10-16T14:36:36.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55891b6dcef0 con 0x55891b785780 2023-10-16T14:36:36.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.137+0000 7f70d3fff700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b796230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:36.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.138+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70c4010f10 tx=0x7f70c400f700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c68e609c40bb6d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.138+0000 7f70b2ffd700 10 client.25900 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:36.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.139+0000 7f70b2ffd700 1 -- 172.21.15.178:0/3715227846 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4015070 con 0x55891b785780 2023-10-16T14:36:36.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.139+0000 7f70b2ffd700 1 -- 172.21.15.178:0/3715227846 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f70c4004830 con 0x55891b785780 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.139+0000 7f70b2ffd700 1 -- 172.21.15.178:0/3715227846 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c4004a30 con 0x55891b785780 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.139+0000 7f70ed852b80 10 client.25900 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.139+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x7f70cc00f040 con 0x55891b785780 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70b2ffd700 1 -- 172.21.15.178:0/3715227846 <== mon.1 v2:172.21.15.187:3300/0 4 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f70c4020050 con 0x55891b785780 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:36.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:36.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 20 client.25900 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:36.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b79a930 0x55891b79cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 4 client.25900 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:36.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.140+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55891b754c10 con 0x55891b79a930 2023-10-16T14:36:36.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.143+0000 7f70d37fe700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b79a930 0x55891b79cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.143+0000 7f70d37fe700 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b79a930 0x55891b79cd50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a5c34856d8f1a6 server_cookie=c27b3e5bc88c30ed in_seq=0 out_seq=0 2023-10-16T14:36:36.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.143+0000 7f70b2ffd700 10 client.25900 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:36.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.145+0000 7f70b2ffd700 1 -- 172.21.15.178:0/3715227846 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55891b754c10 con 0x55891b79a930 2023-10-16T14:36:36.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.145+0000 7f70b2ffd700 10 client.25900 handle_command_reply: tid=0 2023-10-16T14:36:36.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.145+0000 7f70ed852b80 1 client.25900 shutdown 2023-10-16T14:36:36.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.146+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 msgr2=0x55891b78f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.146+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f70c4010f10 tx=0x7f70c400f700 comp rx=0 tx=0).stop 2023-10-16T14:36:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 shutdown_connections 2023-10-16T14:36:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b79a930 0x55891b79cd50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b796230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b78ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 --2- 172.21.15.178:0/3715227846 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b78f1f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.148+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 >> 172.21.15.178:0/3715227846 conn(0x55891b6b2dd0 msgr2=0x55891b77fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.149+0000 7f70b2ffd700 0 client.25900 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.149+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 shutdown_connections 2023-10-16T14:36:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.149+0000 7f70ed852b80 1 -- 172.21.15.178:0/3715227846 wait complete. 2023-10-16T14:36:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.149+0000 7f70ed852b80 20 client.25900 trim_cache size 0 max 16384 2023-10-16T14:36:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:35 smithi187 ceph-mon[135297]: pgmap v425: 129 pgs: 129 active+clean; 321 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 455 op/s 2023-10-16T14:36:36.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.183+0000 7f70ed852b80 1 Processor -- start 2023-10-16T14:36:36.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.183+0000 7f70ed852b80 1 -- start start 2023-10-16T14:36:36.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 0x55891b796ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b7961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b786620 0x55891b789590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b558920 con 0x55891b784e20 2023-10-16T14:36:36.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b78e160 con 0x55891b785780 2023-10-16T14:36:36.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.184+0000 7f70ed852b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b786ec0 con 0x55891b786620 2023-10-16T14:36:36.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 0x55891b796ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b786620 0x55891b789590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d37fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 0x55891b796ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37410/0 (socket says 172.21.15.178:37410) 2023-10-16T14:36:36.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d37fe700 1 -- 172.21.15.178:0/1250625757 learned_addr learned my addr 172.21.15.178:0/1250625757 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b7961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 -- 172.21.15.178:0/1250625757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b786620 msgr2=0x55891b789590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b786620 0x55891b789590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 -- 172.21.15.178:0/1250625757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 msgr2=0x55891b796ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 0x55891b796ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.185+0000 7f70d2ffd700 1 -- 172.21.15.178:0/1250625757 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55891b502de0 con 0x55891b785780 2023-10-16T14:36:36.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.186+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b7961f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f70c400aac0 tx=0x7f70c4012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e74360a2559b8d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.186+0000 7f70ba7fc700 1 -- 172.21.15.178:0/1250625757 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4018570 con 0x55891b785780 2023-10-16T14:36:36.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.186+0000 7f70ba7fc700 1 -- 172.21.15.178:0/1250625757 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70c4027070 con 0x55891b785780 2023-10-16T14:36:36.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.186+0000 7f70ba7fc700 1 -- 172.21.15.178:0/1250625757 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4018b60 con 0x55891b785780 2023-10-16T14:36:36.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 -- 172.21.15.178:0/1250625757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 msgr2=0x55891b7961f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b7961f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f70c400aac0 tx=0x7f70c4012000 comp rx=0 tx=0).stop 2023-10-16T14:36:36.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 -- 172.21.15.178:0/1250625757 shutdown_connections 2023-10-16T14:36:36.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b786620 0x55891b789590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b784e20 0x55891b796ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 --2- 172.21.15.178:0/1250625757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b785780 0x55891b7961f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.187+0000 7f70ed852b80 1 -- 172.21.15.178:0/1250625757 >> 172.21.15.178:0/1250625757 conn(0x55891b6b2dd0 msgr2=0x55891b77e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.189+0000 7f70ed852b80 1 -- 172.21.15.178:0/1250625757 shutdown_connections 2023-10-16T14:36:36.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.189+0000 7f70ed852b80 1 -- 172.21.15.178:0/1250625757 wait complete. 2023-10-16T14:36:36.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.189+0000 7f70ed852b80 1 -- start start 2023-10-16T14:36:36.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.190+0000 7f70ed852b80 1 Processor -- start 2023-10-16T14:36:36.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.190+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b795b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.190+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70ed852b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b794180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70ed852b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b794850 con 0x55891b786620 2023-10-16T14:36:36.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70ed852b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b7949c0 con 0x55891b784e20 2023-10-16T14:36:36.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70ed852b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55891b794b80 con 0x55891b785780 2023-10-16T14:36:36.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d2ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50496/0 (socket says 172.21.15.178:50496) 2023-10-16T14:36:36.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d2ffd700 1 -- 172.21.15.178:0/660967579 learned_addr learned my addr 172.21.15.178:0/660967579 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70ed852b80 10 client.0 _ll_register_callbacks cb 0x7f70c40107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:36.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d37fe700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b795b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d2ffd700 1 -- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 msgr2=0x55891b795b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.191+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b795b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70d2ffd700 1 -- 172.21.15.178:0/660967579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 msgr2=0x55891b794180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:36.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b794180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70d2ffd700 1 -- 172.21.15.178:0/660967579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc00f040 con 0x55891b785780 2023-10-16T14:36:36.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70d2ffd700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70c4000c00 tx=0x7f70c4005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85f9ccb90dfa2d72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70b2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:36.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4015070 con 0x55891b785780 2023-10-16T14:36:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.192+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70cc015d80 con 0x55891b785780 2023-10-16T14:36:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f70c4000e40 con 0x55891b785780 2023-10-16T14:36:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70c4032070 con 0x55891b785780 2023-10-16T14:36:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 10 client.25906 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:36.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55891b4ebc60 con 0x55891b785780 2023-10-16T14:36:36.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f70cc015d80 con 0x55891b785780 2023-10-16T14:36:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f70c400f420 con 0x55891b785780 2023-10-16T14:36:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 10 client.25906 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:36.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 20 client.25906 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b7ef820 0x55891b7f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.193+0000 7f70ed852b80 4 client.25906 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:36:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.194+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55891b6d0e50 con 0x55891b7ef820 2023-10-16T14:36:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.194+0000 7f70d37fe700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b7ef820 0x55891b7f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.195+0000 7f70b2ffd700 10 client.25906 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.195+0000 7f70d37fe700 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b7ef820 0x55891b7f1c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562a0cd5ecba96d7 server_cookie=24fb76875e58015 in_seq=0 out_seq=0 2023-10-16T14:36:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70b2ffd700 1 -- 172.21.15.178:0/660967579 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55891b6d0e50 con 0x55891b7ef820 2023-10-16T14:36:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70b2ffd700 10 client.25906 handle_command_reply: tid=0 2023-10-16T14:36:36.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70ed852b80 1 client.25906 shutdown 2023-10-16T14:36:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 msgr2=0x55891b7960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70c4000c00 tx=0x7f70c4005f20 comp rx=0 tx=0).stop 2023-10-16T14:36:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 shutdown_connections 2023-10-16T14:36:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.197+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55891b7ef820 0x55891b7f1c40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.198+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55891b786620 0x55891b794180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.198+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55891b784e20 0x55891b795b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.198+0000 7f70ed852b80 1 --2- 172.21.15.178:0/660967579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55891b785780 0x55891b7960b0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f70c4000c00 tx=0x7f70c4005f20 comp rx=0 tx=0).stop 2023-10-16T14:36:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.198+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 >> 172.21.15.178:0/660967579 conn(0x55891b6b2dd0 msgr2=0x55891b796ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.198+0000 7f70b2ffd700 0 client.25906 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:36.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.200+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 shutdown_connections 2023-10-16T14:36:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.202+0000 7f70ed852b80 1 -- 172.21.15.178:0/660967579 wait complete. 2023-10-16T14:36:36.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.202+0000 7f70ed852b80 20 client.25906 trim_cache size 0 max 16384 2023-10-16T14:36:36.215 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:36:36.215 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:36:36.215 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:36:36.215 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:36:36.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 msgr2=0x7f70bc0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 0x7f70bc0a3a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e350 tx=0x7f70cc009040 comp rx=0 tx=0).stop 2023-10-16T14:36:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 msgr2=0x7f70d41948e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 0x7f70d41948e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f70c80098e0 tx=0x7f70c8005740 comp rx=0 tx=0).stop 2023-10-16T14:36:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 shutdown_connections 2023-10-16T14:36:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f70bc0a1580 0x7f70bc0a3a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f70d4104c90 0x7f70d419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70d4103100 0x7f70d41948e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 --2- 172.21.15.178:0/451125949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70d41042f0 0x7f70d4194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.204+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 >> 172.21.15.178:0/451125949 conn(0x7f70d40fe630 msgr2=0x7f70d4100790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.205+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 shutdown_connections 2023-10-16T14:36:36.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.205+0000 7f70d9fff700 1 -- 172.21.15.178:0/451125949 wait complete. 2023-10-16T14:36:36.238 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:36:36.238 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:36:36.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 -- 172.21.15.178:0/2593540602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc106c00 msgr2=0x7f40bc109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 --2- 172.21.15.178:0/2593540602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc106c00 0x7f40bc109070 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009f90 tx=0x7f40b400b330 comp rx=0 tx=0).stop 2023-10-16T14:36:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 -- 172.21.15.178:0/2593540602 shutdown_connections 2023-10-16T14:36:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 --2- 172.21.15.178:0/2593540602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc106c00 0x7f40bc109070 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 --2- 172.21.15.178:0/2593540602 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc1019a0 0x7f40bc103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 --2- 172.21.15.178:0/2593540602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1066c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 -- 172.21.15.178:0/2593540602 >> 172.21.15.178:0/2593540602 conn(0x7f40bc0fb3b0 msgr2=0x7f40bc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.504+0000 7f40c1bff700 1 -- 172.21.15.178:0/2593540602 shutdown_connections 2023-10-16T14:36:36.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.505+0000 7f40c1bff700 1 -- 172.21.15.178:0/2593540602 wait complete. 2023-10-16T14:36:36.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 Processor -- start 2023-10-16T14:36:36.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 -- start start 2023-10-16T14:36:36.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 0x7f40bc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1a07e0 con 0x7f40bc1042d0 2023-10-16T14:36:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1a0950 con 0x7f40bc106c00 2023-10-16T14:36:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.506+0000 7f40c1bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc1a0b10 con 0x7f40bc1019a0 2023-10-16T14:36:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56310/0 (socket says 172.21.15.178:56310) 2023-10-16T14:36:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 -- 172.21.15.178:0/3626884702 learned_addr learned my addr 172.21.15.178:0/3626884702 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bb7fe700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 0x7f40bc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40baffd700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 -- 172.21.15.178:0/3626884702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 msgr2=0x7f40bc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 0x7f40bc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 -- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 msgr2=0x7f40bc1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 -- 172.21.15.178:0/3626884702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4009c40 con 0x7f40bc106c00 2023-10-16T14:36:36.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40baffd700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bb7fe700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 0x7f40bc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.507+0000 7f40bbfff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40b4004ca0 tx=0x7f40b4004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.508+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400e040 con 0x7f40bc106c00 2023-10-16T14:36:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.508+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b400be00 con 0x7f40bc106c00 2023-10-16T14:36:36.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.508+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b401d470 con 0x7f40bc106c00 2023-10-16T14:36:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.509+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc1a0d90 con 0x7f40bc106c00 2023-10-16T14:36:36.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.509+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc1a11c0 con 0x7f40bc106c00 2023-10-16T14:36:36.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.510+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0fcf30 con 0x7f40bc106c00 2023-10-16T14:36:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.512+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f40b40040c0 con 0x7f40bc106c00 2023-10-16T14:36:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.513+0000 7f40b8ff9700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 0x7f40a40a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.513+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f40b40ce190 con 0x7f40bc106c00 2023-10-16T14:36:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.514+0000 7f40bb7fe700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 0x7f40a40a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:36.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.514+0000 7f40bb7fe700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 0x7f40a40a39e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f40bc199b40 tx=0x7f40ac009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:36.520 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:36.515+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f40b40974b0 con 0x7f40bc106c00 2023-10-16T14:36:37.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.365+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40bc10e9d0 con 0x7f40bc106c00 2023-10-16T14:36:37.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.366+0000 7f40b8ff9700 1 -- 172.21.15.178:0/3626884702 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f40b4025e80 con 0x7f40bc106c00 2023-10-16T14:36:37.369 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:36:37.369 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:35:50.922947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:36:37.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 msgr2=0x7f40a40a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 0x7f40a40a39e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f40bc199b40 tx=0x7f40ac009310 comp rx=0 tx=0).stop 2023-10-16T14:36:37.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 msgr2=0x7f40bc1a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f40b4004ca0 tx=0x7f40b4004d80 comp rx=0 tx=0).stop 2023-10-16T14:36:37.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 shutdown_connections 2023-10-16T14:36:37.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f40a40a1520 0x7f40a40a39e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f40bc106c00 0x7f40bc1a0210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40bc1019a0 0x7f40bc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 --2- 172.21.15.178:0/3626884702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40bc1042d0 0x7f40bc1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.369+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 >> 172.21.15.178:0/3626884702 conn(0x7f40bc0fb3b0 msgr2=0x7f40bc102610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.370+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 shutdown_connections 2023-10-16T14:36:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.370+0000 7f40c1bff700 1 -- 172.21.15.178:0/3626884702 wait complete. 2023-10-16T14:36:37.375 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 30 2023-10-16T14:36:37.394 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:36:37.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.627+0000 7fe8b0bff700 1 -- 172.21.15.178:0/2334809254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac100ea0 msgr2=0x7fe8ac1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.627+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/2334809254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac1012c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009990 tx=0x7fe89c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 -- 172.21.15.178:0/2334809254 shutdown_connections 2023-10-16T14:36:37.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/2334809254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac102a40 0x7fe8ac109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/2334809254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac1012c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/2334809254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102080 0x7fe8ac102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 -- 172.21.15.178:0/2334809254 >> 172.21.15.178:0/2334809254 conn(0x7fe8ac0fc420 msgr2=0x7fe8ac0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 -- 172.21.15.178:0/2334809254 shutdown_connections 2023-10-16T14:36:37.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.628+0000 7fe8b0bff700 1 -- 172.21.15.178:0/2334809254 wait complete. 2023-10-16T14:36:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.629+0000 7fe8b0bff700 1 Processor -- start 2023-10-16T14:36:37.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.629+0000 7fe8b0bff700 1 -- start start 2023-10-16T14:36:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 0x7fe8ac1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1a0a00 con 0x7fe8ac100ea0 2023-10-16T14:36:37.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1a0b40 con 0x7fe8ac102a40 2023-10-16T14:36:37.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8b0bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac1a0cd0 con 0x7fe8ac102080 2023-10-16T14:36:37.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 0x7fe8ac1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56320/0 (socket says 172.21.15.178:56320) 2023-10-16T14:36:37.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 -- 172.21.15.178:0/877412527 learned_addr learned my addr 172.21.15.178:0/877412527 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 -- 172.21.15.178:0/877412527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 msgr2=0x7fe8ac1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 0x7fe8ac1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 -- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac100ea0 msgr2=0x7fe8ac198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aad9d700 1 -- 172.21.15.178:0/877412527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0095f0 con 0x7fe8ac102a40 2023-10-16T14:36:37.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.630+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 0x7fe8ac1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:37.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400ef30 tx=0x7fe8a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8a3fff700 1 -- 172.21.15.178:0/877412527 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4018070 con 0x7fe8ac102a40 2023-10-16T14:36:37.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8a3fff700 1 -- 172.21.15.178:0/877412527 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8a400f040 con 0x7fe8ac102a40 2023-10-16T14:36:37.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac1a0f50 con 0x7fe8ac102a40 2023-10-16T14:36:37.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8a3fff700 1 -- 172.21.15.178:0/877412527 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400cd00 con 0x7fe8ac102a40 2023-10-16T14:36:37.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.631+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1a13d0 con 0x7fe8ac102a40 2023-10-16T14:36:37.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.634+0000 7fe8a3fff700 1 -- 172.21.15.178:0/877412527 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe8a4003a00 con 0x7fe8ac102a40 2023-10-16T14:36:37.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.635+0000 7fe8c45c3b80 1 Processor -- start 2023-10-16T14:36:37.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8a3fff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 0x7fe8980a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 -- start start 2023-10-16T14:36:37.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8a3fff700 1 -- 172.21.15.178:0/877412527 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe8a40ccbc0 con 0x7fe8ac102a40 2023-10-16T14:36:37.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9218040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9218620 0x55f1a9218aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 0x7fe8980a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a918bc70 con 0x55f1a9218620 2023-10-16T14:36:37.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a9170c60 con 0x55f1a9217780 2023-10-16T14:36:37.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a916e580 con 0x55f1a9216e20 2023-10-16T14:36:37.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9218620 0x55f1a9218aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50528/0 (socket says 172.21.15.178:50528) 2023-10-16T14:36:37.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.636+0000 7fe8a259c700 1 -- 172.21.15.178:0/661657153 learned_addr learned my addr 172.21.15.178:0/661657153 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9218040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 -- 172.21.15.178:0/661657153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 msgr2=0x55f1a9218040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9218040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 -- 172.21.15.178:0/661657153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9218620 msgr2=0x55f1a9218aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9218620 0x55f1a9218aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 -- 172.21.15.178:0/661657153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1a91273f0 con 0x55f1a9216e20 2023-10-16T14:36:37.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 0x7fe8980a39c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac100bc0 tx=0x7fe894013c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a259c700 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac101bf0 tx=0x7fe89c005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35884ff63a52faf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a0d99700 1 -- 172.21.15.178:0/661657153 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c01d070 con 0x55f1a9216e20 2023-10-16T14:36:37.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a0d99700 1 -- 172.21.15.178:0/661657153 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89c01e040 con 0x55f1a9216e20 2023-10-16T14:36:37.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.637+0000 7fe8a0d99700 1 -- 172.21.15.178:0/661657153 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c004890 con 0x55f1a9216e20 2023-10-16T14:36:37.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/661657153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 msgr2=0x55f1a9217240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac101bf0 tx=0x7fe89c005ea0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/661657153 shutdown_connections 2023-10-16T14:36:37.654 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9218620 0x55f1a9218aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9216e20 0x55f1a9217240 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/661657153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9218040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/661657153 >> 172.21.15.178:0/661657153 conn(0x55f1a9144dd0 msgr2=0x55f1a9211fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/661657153 shutdown_connections 2023-10-16T14:36:37.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.638+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/661657153 wait complete. 2023-10-16T14:36:37.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.639+0000 7fe8c45c3b80 1 -- start start 2023-10-16T14:36:37.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.639+0000 7fe8c45c3b80 1 Processor -- start 2023-10-16T14:36:37.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.639+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9220bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9221100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a9169e70 con 0x55f1a9216e20 2023-10-16T14:36:37.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a9106710 con 0x55f1a9217780 2023-10-16T14:36:37.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a91063b0 con 0x55f1a9218620 2023-10-16T14:36:37.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8c45c3b80 10 client.0 _ll_register_callbacks cb 0x55f1a8f63800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:37.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9221100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50538/0 (socket says 172.21.15.178:50538) 2023-10-16T14:36:37.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 -- 172.21.15.178:0/1544384715 learned_addr learned my addr 172.21.15.178:0/1544384715 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8a259c700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9220bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 -- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 msgr2=0x55f1a9221100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9221100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 -- 172.21.15.178:0/1544384715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 msgr2=0x55f1a9220bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9220bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 -- 172.21.15.178:0/1544384715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac109c90 con 0x55f1a9218620 2023-10-16T14:36:37.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9221100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:37.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40ca2f0 tx=0x7fe8a4099580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96ff73545e155e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.640+0000 7fe891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:37.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a401b320 con 0x55f1a9218620 2023-10-16T14:36:37.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe8a409a2a0 con 0x55f1a9218620 2023-10-16T14:36:37.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1a916eef0 con 0x55f1a9218620 2023-10-16T14:36:37.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8a400ce60 con 0x55f1a9218620 2023-10-16T14:36:37.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 10 client.25921 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f1a8f94de0 con 0x55f1a9218620 2023-10-16T14:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1a916eef0 con 0x55f1a9218620 2023-10-16T14:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe8a40b2290 con 0x55f1a9218620 2023-10-16T14:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 10 client.25921 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:37.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 10 client.25921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 20 client.25921 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.641+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a922ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.642+0000 7fe8c45c3b80 4 client.25921 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.642+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1a91e6c10 con 0x55f1a922c860 2023-10-16T14:36:37.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.642+0000 7fe8a259c700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a922ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.642+0000 7fe8a259c700 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a922ec80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=241cebbcbcfe4254 server_cookie=5e9e34e6cecee76d in_seq=0 out_seq=0 2023-10-16T14:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.642+0000 7fe891ffb700 10 client.25921 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe891ffb700 1 -- 172.21.15.178:0/1544384715 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f1a91e6c10 con 0x55f1a922c860 2023-10-16T14:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe891ffb700 10 client.25921 handle_command_reply: tid=0 2023-10-16T14:36:37.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 client.25921 shutdown 2023-10-16T14:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 msgr2=0x55f1a9228140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40ca2f0 tx=0x7fe8a4099580 comp rx=0 tx=0).stop 2023-10-16T14:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 shutdown_connections 2023-10-16T14:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a922ec80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9228140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9220bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe891ffb700 0 client.25921 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/1544384715 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9217780 0x55f1a9221100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 >> 172.21.15.178:0/1544384715 conn(0x55f1a9144dd0 msgr2=0x55f1a9211bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 shutdown_connections 2023-10-16T14:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.643+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/1544384715 wait complete. 2023-10-16T14:36:37.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.644+0000 7fe8c45c3b80 20 client.25921 trim_cache size 0 max 16384 2023-10-16T14:36:37.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 Processor -- start 2023-10-16T14:36:37.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 -- start start 2023-10-16T14:36:37.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9217200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9218620 0x55f1a921e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a8fea920 con 0x55f1a9216e20 2023-10-16T14:36:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a9217f00 con 0x55f1a9218620 2023-10-16T14:36:37.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a920cc00 con 0x55f1a9217780 2023-10-16T14:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9217200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8aad9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9218620 0x55f1a921e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50552/0 (socket says 172.21.15.178:50552) 2023-10-16T14:36:37.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9217200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37484/0 (socket says 172.21.15.178:37484) 2023-10-16T14:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.658+0000 7fe8aa59c700 1 -- 172.21.15.178:0/3563639431 learned_addr learned my addr 172.21.15.178:0/3563639431 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 -- 172.21.15.178:0/3563639431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9218620 msgr2=0x55f1a921e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9218620 0x55f1a921e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 -- 172.21.15.178:0/3563639431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 msgr2=0x55f1a9217200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9217200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 -- 172.21.15.178:0/3563639431 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1a8f7dc60 con 0x55f1a9217780 2023-10-16T14:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fb70 tx=0x7fe894009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98236ed11bf56303 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8a159a700 1 -- 172.21.15.178:0/3563639431 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894010840 con 0x55f1a9217780 2023-10-16T14:36:37.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8a159a700 1 -- 172.21.15.178:0/3563639431 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe894025070 con 0x55f1a9217780 2023-10-16T14:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.659+0000 7fe8a159a700 1 -- 172.21.15.178:0/3563639431 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894005280 con 0x55f1a9217780 2023-10-16T14:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/3563639431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 msgr2=0x55f1a9228100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe89400fb70 tx=0x7fe894009260 comp rx=0 tx=0).stop 2023-10-16T14:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/3563639431 shutdown_connections 2023-10-16T14:36:37.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9218620 0x55f1a921e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9216e20 0x55f1a9217200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/3563639431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9217780 0x55f1a9228100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/3563639431 >> 172.21.15.178:0/3563639431 conn(0x55f1a9144dd0 msgr2=0x55f1a92107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/3563639431 shutdown_connections 2023-10-16T14:36:37.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.660+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/3563639431 wait complete. 2023-10-16T14:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.661+0000 7fe8c45c3b80 1 -- start start 2023-10-16T14:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.661+0000 7fe8c45c3b80 1 Processor -- start 2023-10-16T14:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.661+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9217780 0x55f1a9227ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9225fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a92265c0 con 0x55f1a9217780 2023-10-16T14:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a9226700 con 0x55f1a9216e20 2023-10-16T14:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1a92268c0 con 0x55f1a9218620 2023-10-16T14:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8c45c3b80 10 client.0 _ll_register_callbacks cb 0x7fe8a40999b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:37.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56340/0 (socket says 172.21.15.178:56340) 2023-10-16T14:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8aa59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9217780 0x55f1a9227ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 -- 172.21.15.178:0/292103597 learned_addr learned my addr 172.21.15.178:0/292103597 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8aad9d700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9225fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 -- 172.21.15.178:0/292103597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 msgr2=0x55f1a9225fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9225fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 -- 172.21.15.178:0/292103597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9217780 msgr2=0x55f1a9227ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9217780 0x55f1a9227ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.682 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 -- 172.21.15.178:0/292103597 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89400f8b0 con 0x55f1a9216e20 2023-10-16T14:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe8a259c700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009350 tx=0x7fe89c00be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=813df4158fd1ded4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.662+0000 7fe891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c032070 con 0x55f1a9216e20 2023-10-16T14:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe89c0344e0 con 0x55f1a9216e20 2023-10-16T14:36:37.683 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe89c01e040 con 0x55f1a9216e20 2023-10-16T14:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe89c01d040 con 0x55f1a9216e20 2023-10-16T14:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 10 client.25940 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55f1a8fbb250 con 0x55f1a9216e20 2023-10-16T14:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe89c01e040 con 0x55f1a9216e20 2023-10-16T14:36:37.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fe89c03d070 con 0x55f1a9216e20 2023-10-16T14:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 10 client.25940 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 20 client.25940 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.663+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a9283620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe8c45c3b80 4 client.25940 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:37.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1a9162e50 con 0x55f1a922c860 2023-10-16T14:36:37.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a9283620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe8aa59c700 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a9283620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=405a70f3e1f2897e server_cookie=39b4d3233650f141 in_seq=0 out_seq=0 2023-10-16T14:36:37.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe891ffb700 10 client.25940 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:37.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe891ffb700 1 -- 172.21.15.178:0/292103597 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1a9162e50 con 0x55f1a922c860 2023-10-16T14:36:37.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.664+0000 7fe891ffb700 10 client.25940 handle_command_reply: tid=0 2023-10-16T14:36:37.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 client.25940 shutdown 2023-10-16T14:36:37.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 msgr2=0x55f1a9227990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.687 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009350 tx=0x7fe89c00be00 comp rx=0 tx=0).stop 2023-10-16T14:36:37.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 shutdown_connections 2023-10-16T14:36:37.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55f1a922c860 0x55f1a9283620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f1a9218620 0x55f1a9225fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe891ffb700 0 client.25940 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:36:37.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f1a9216e20 0x55f1a9227990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 --2- 172.21.15.178:0/292103597 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f1a9217780 0x55f1a9227ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 >> 172.21.15.178:0/292103597 conn(0x55f1a9144dd0 msgr2=0x55f1a921e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.665+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 shutdown_connections 2023-10-16T14:36:37.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.666+0000 7fe8c45c3b80 1 -- 172.21.15.178:0/292103597 wait complete. 2023-10-16T14:36:37.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.666+0000 7fe8c45c3b80 20 client.25940 trim_cache size 0 max 16384 2023-10-16T14:36:37.690 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:37.691 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 msgr2=0x7fe8980a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 0x7fe8980a39c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac100bc0 tx=0x7fe894013c70 comp rx=0 tx=0).stop 2023-10-16T14:36:37.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 msgr2=0x7fe8ac1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe8a400ef30 tx=0x7fe8a400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 shutdown_connections 2023-10-16T14:36:37.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe8980a1500 0x7fe8980a39c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe8ac102a40 0x7fe8ac1a0430 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe8ac100ea0 0x7fe8ac198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 --2- 172.21.15.178:0/877412527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8ac102080 0x7fe8ac1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 >> 172.21.15.178:0/877412527 conn(0x7fe8ac0fc420 msgr2=0x7fe8ac107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 shutdown_connections 2023-10-16T14:36:37.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.667+0000 7fe8b0bff700 1 -- 172.21.15.178:0/877412527 wait complete. 2023-10-16T14:36:37.694 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:36:37.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/651141679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 msgr2=0x7f8d64109000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/651141679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d64109000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00b0a0 tx=0x7f8d5c00b3b0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/651141679 shutdown_connections 2023-10-16T14:36:37.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/651141679 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d64109000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/651141679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d64101990 0x7f8d64103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/651141679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d641042c0 0x7f8d641066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/651141679 >> 172.21.15.178:0/651141679 conn(0x7f8d640fb320 msgr2=0x7f8d640fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/651141679 shutdown_connections 2023-10-16T14:36:37.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.925+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/651141679 wait complete. 2023-10-16T14:36:37.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.926+0000 7f8d6a3ff700 1 Processor -- start 2023-10-16T14:36:37.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 -- start start 2023-10-16T14:36:37.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042c0 0x7f8d64199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d641a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a0a70 con 0x7f8d641042c0 2023-10-16T14:36:37.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a0bb0 con 0x7f8d64106bf0 2023-10-16T14:36:37.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6a3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a0d40 con 0x7f8d64101990 2023-10-16T14:36:37.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50562/0 (socket says 172.21.15.178:50562) 2023-10-16T14:36:37.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d6899c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d641a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042c0 0x7f8d64199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.927+0000 7f8d63fff700 1 -- 172.21.15.178:0/3959425891 learned_addr learned my addr 172.21.15.178:0/3959425891 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 -- 172.21.15.178:0/3959425891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 msgr2=0x7f8d64198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 -- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042c0 msgr2=0x7f8d64199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042c0 0x7f8d64199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 -- 172.21.15.178:0/3959425891 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d58009520 con 0x7f8d64106bf0 2023-10-16T14:36:37.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d63fff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:37.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6899c700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d641a04a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009fd0 tx=0x7f8d5c007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d617fa700 1 -- 172.21.15.178:0/3959425891 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00e040 con 0x7f8d64106bf0 2023-10-16T14:36:37.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d617fa700 1 -- 172.21.15.178:0/3959425891 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c004c50 con 0x7f8d64106bf0 2023-10-16T14:36:37.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c009d00 con 0x7f8d64106bf0 2023-10-16T14:36:37.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d617fa700 1 -- 172.21.15.178:0/3959425891 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c01b6c0 con 0x7f8d64106bf0 2023-10-16T14:36:37.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.928+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d641a1250 con 0x7f8d64106bf0 2023-10-16T14:36:37.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.931+0000 7f8d617fa700 1 -- 172.21.15.178:0/3959425891 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d5c01d470 con 0x7f8d64106bf0 2023-10-16T14:36:37.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 Processor -- start 2023-10-16T14:36:37.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d617fa700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 0x7f8d4c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 -- start start 2023-10-16T14:36:37.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d617fa700 1 -- 172.21.15.178:0/3959425891 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d5c0cdfe0 con 0x7f8d64106bf0 2023-10-16T14:36:37.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 0x7f8d4c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 0x55d4448d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d44483cc70 con 0x55d4448d1e30 2023-10-16T14:36:37.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d444821c60 con 0x55d4448d0f90 2023-10-16T14:36:37.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.933+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d44481f580 con 0x55d4448d0630 2023-10-16T14:36:37.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 0x55d4448d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 0x55d4448d1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56374/0 (socket says 172.21.15.178:56374) 2023-10-16T14:36:37.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d637fe700 1 -- 172.21.15.178:0/826810178 learned_addr learned my addr 172.21.15.178:0/826810178 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d63fff700 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 -- 172.21.15.178:0/826810178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 msgr2=0x55d4448d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 -- 172.21.15.178:0/826810178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 msgr2=0x55d4448d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 0x55d4448d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 -- 172.21.15.178:0/826810178 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4447d83f0 con 0x55d4448d1e30 2023-10-16T14:36:37.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d63fff700 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:37.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 0x7f8d4c0a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009fd0 tx=0x7f8d58011b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d6899c700 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448d22b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00b780 tx=0x7f8d5c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da12bf78a180454 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/826810178 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c09b6c0 con 0x55d4448d1e30 2023-10-16T14:36:37.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/826810178 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c09afc0 con 0x55d4448d1e30 2023-10-16T14:36:37.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.934+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/826810178 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c01a3c0 con 0x55d4448d1e30 2023-10-16T14:36:37.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 -- 172.21.15.178:0/826810178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 msgr2=0x55d4448d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448d22b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00b780 tx=0x7f8d5c0097e0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 -- 172.21.15.178:0/826810178 shutdown_connections 2023-10-16T14:36:37.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448d22b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/826810178 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d0f90 0x55d4448d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 -- 172.21.15.178:0/826810178 >> 172.21.15.178:0/826810178 conn(0x55d4447f5dd0 msgr2=0x55d4448c2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 -- 172.21.15.178:0/826810178 shutdown_connections 2023-10-16T14:36:37.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.935+0000 7f8d7de19b80 1 -- 172.21.15.178:0/826810178 wait complete. 2023-10-16T14:36:37.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.936+0000 7f8d7de19b80 1 -- start start 2023-10-16T14:36:37.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.936+0000 7f8d7de19b80 1 Processor -- start 2023-10-16T14:36:37.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d44481ae70 con 0x55d4448d0630 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4447b7710 con 0x55d4448d1e30 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4447b73b0 con 0x55d4448d0f90 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50588/0 (socket says 172.21.15.178:50588) 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 -- 172.21.15.178:0/2547382585 learned_addr learned my addr 172.21.15.178:0/2547382585 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d63fff700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 -- 172.21.15.178:0/2547382585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 msgr2=0x55d4448e1890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:36:37.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 10 client.0 _ll_register_callbacks cb 0x55d444614800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:37.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 -- 172.21.15.178:0/2547382585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 msgr2=0x55d4448da310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d637fe700 1 -- 172.21.15.178:0/2547382585 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d54009d60 con 0x55d4448d0f90 2023-10-16T14:36:37.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d63fff700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:37.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.937+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d44481fef0 con 0x55d4448d0f90 2023-10-16T14:36:37.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d437fe700 10 client.25939 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:37.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d637fe700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d58004290 tx=0x7f8d580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdc3b7a4818f12e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d437fe700 1 -- 172.21.15.178:0/2547382585 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58003ea0 con 0x55d4448d0f90 2023-10-16T14:36:37.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d437fe700 1 -- 172.21.15.178:0/2547382585 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d5800f630 con 0x55d4448d0f90 2023-10-16T14:36:37.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d437fe700 1 -- 172.21.15.178:0/2547382585 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d5800f810 con 0x55d4448d0f90 2023-10-16T14:36:37.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d7de19b80 10 client.25939 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:37.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.938+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=30}) v3 -- 0x7f8d641a1250 con 0x55d4448d0f90 2023-10-16T14:36:37.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d437fe700 1 -- 172.21.15.178:0/2547382585 <== mon.1 v2:172.21.15.187:3300/0 4 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8d5800f970 con 0x55d4448d0f90 2023-10-16T14:36:37.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 10 client.25939 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:37.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 10 client.25939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:36:37.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 20 client.25939 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:37.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d4448e5fb0 0x55d4448e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 4 client.25939 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:37.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d444897c10 con 0x55d4448e5fb0 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.939+0000 7f8d63fff700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d4448e5fb0 0x55d4448e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.940+0000 7f8d63fff700 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d4448e5fb0 0x55d4448e83f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9650942caf5c852 server_cookie=904d5261c78455e4 in_seq=0 out_seq=0 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.940+0000 7f8d437fe700 10 client.25939 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d437fe700 1 -- 172.21.15.178:0/2547382585 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d444897c10 con 0x55d4448e5fb0 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d437fe700 10 client.25939 handle_command_reply: tid=0 2023-10-16T14:36:37.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 client.25939 shutdown 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 msgr2=0x55d4448da850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d58004290 tx=0x7f8d580047c0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 shutdown_connections 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d4448e5fb0 0x55d4448e83f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/2547382585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0f90 0x55d4448da850 secure :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8d58004290 tx=0x7f8d580047c0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 >> 172.21.15.178:0/2547382585 conn(0x55d4447f5dd0 msgr2=0x55d4448c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d437fe700 0 client.25939 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:37.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 shutdown_connections 2023-10-16T14:36:37.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.941+0000 7f8d7de19b80 1 -- 172.21.15.178:0/2547382585 wait complete. 2023-10-16T14:36:37.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.942+0000 7f8d7de19b80 20 client.25939 trim_cache size 0 max 16384 2023-10-16T14:36:37.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 Processor -- start 2023-10-16T14:36:37.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 -- start start 2023-10-16T14:36:37.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d44469b920 con 0x55d4448d0630 2023-10-16T14:36:37.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4448daf90 con 0x55d4448d1e30 2023-10-16T14:36:37.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4448e2830 con 0x55d4448ecd90 2023-10-16T14:36:37.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d6899c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56400/0 (socket says 172.21.15.178:56400) 2023-10-16T14:36:37.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d6899c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50600/0 (socket says 172.21.15.178:50600) 2023-10-16T14:36:37.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.957+0000 7f8d637fe700 1 -- 172.21.15.178:0/1531047777 learned_addr learned my addr 172.21.15.178:0/1531047777 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 -- 172.21.15.178:0/1531047777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 msgr2=0x55d4448e1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 -- 172.21.15.178:0/1531047777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 msgr2=0x55d4448da2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 -- 172.21.15.178:0/1531047777 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d444645de0 con 0x55d4448ecd90 2023-10-16T14:36:37.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d6899c700 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55d4448d8b40 tx=0x7f8d5c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b98484de70b97c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d4affd700 1 -- 172.21.15.178:0/1531047777 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c09b6c0 con 0x55d4448ecd90 2023-10-16T14:36:37.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d4affd700 1 -- 172.21.15.178:0/1531047777 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c09f110 con 0x55d4448ecd90 2023-10-16T14:36:37.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.958+0000 7f8d4affd700 1 -- 172.21.15.178:0/1531047777 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c09b220 con 0x55d4448ecd90 2023-10-16T14:36:37.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1531047777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 msgr2=0x55d4448da810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55d4448d8b40 tx=0x7f8d5c009c30 comp rx=0 tx=0).stop 2023-10-16T14:36:37.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1531047777 shutdown_connections 2023-10-16T14:36:37.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448ecd90 0x55d4448da810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d0630 0x55d4448da2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1531047777 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448d1e30 0x55d4448e1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1531047777 >> 172.21.15.178:0/1531047777 conn(0x55d4447f5dd0 msgr2=0x55d4448e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1531047777 shutdown_connections 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.959+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1531047777 wait complete. 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 -- start start 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 Processor -- start 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448e1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448ecd90 0x55d4448df740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4448dfd90 con 0x55d4448d1e30 2023-10-16T14:36:37.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4448dff00 con 0x55d4448ecd90 2023-10-16T14:36:37.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.960+0000 7f8d7de19b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4448e00c0 con 0x55d4448d0630 2023-10-16T14:36:37.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d7de19b80 10 client.0 _ll_register_callbacks cb 0x55d444614800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:37.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50610/0 (socket says 172.21.15.178:50610) 2023-10-16T14:36:37.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 -- 172.21.15.178:0/1222473153 learned_addr learned my addr 172.21.15.178:0/1222473153 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:37.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d6899c700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448ecd90 0x55d4448df740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d637fe700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448e1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 -- 172.21.15.178:0/1222473153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448ecd90 msgr2=0x55d4448df740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448ecd90 0x55d4448df740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 -- 172.21.15.178:0/1222473153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 msgr2=0x55d4448e1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:37.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448e1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 -- 172.21.15.178:0/1222473153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d4447d83f0 con 0x55d4448d0630 2023-10-16T14:36:37.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d63fff700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400a010 tx=0x7f8d54010cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6660f199b3a91421 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:37.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:37.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.961+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54004250 con 0x55d4448d0630 2023-10-16T14:36:37.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d540043b0 con 0x55d4448d0630 2023-10-16T14:36:37.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d5c09afc0 con 0x55d4448d0630 2023-10-16T14:36:37.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d54004570 con 0x55d4448d0630 2023-10-16T14:36:37.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 10 client.25945 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:37.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55d44462ec60 con 0x55d4448d0630 2023-10-16T14:36:37.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c09afc0 con 0x55d4448d0630 2023-10-16T14:36:37.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f8d540055d0 con 0x55d4448d0630 2023-10-16T14:36:37.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 10 client.25945 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:37.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 10 client.25945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:36:37.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 20 client.25945 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:37.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d44493b2d0 0x55d44493d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:37.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 4 client.25945 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:37.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.962+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d444813e50 con 0x55d44493b2d0 2023-10-16T14:36:37.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.963+0000 7f8d637fe700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d44493b2d0 0x55d44493d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:37.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.963+0000 7f8d637fe700 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d44493b2d0 0x55d44493d6f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49ca879cf7f1dcac server_cookie=e87e99377d037129 in_seq=0 out_seq=0 2023-10-16T14:36:37.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.963+0000 7f8d437fe700 10 client.25945 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:37.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.964+0000 7f8d437fe700 1 -- 172.21.15.178:0/1222473153 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d444813e50 con 0x55d44493b2d0 2023-10-16T14:36:37.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.964+0000 7f8d437fe700 10 client.25945 handle_command_reply: tid=0 2023-10-16T14:36:37.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.964+0000 7f8d7de19b80 1 client.25945 shutdown 2023-10-16T14:36:37.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 msgr2=0x55d4448e1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400a010 tx=0x7f8d54010cf0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 shutdown_connections 2023-10-16T14:36:37.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55d44493b2d0 0x55d44493d6f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d4448ecd90 0x55d4448df740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d437fe700 0 client.25945 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:36:37.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d4448d0630 0x55d4448e1130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 --2- 172.21.15.178:0/1222473153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d4448d1e30 0x55d4448e1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 >> 172.21.15.178:0/1222473153 conn(0x55d4447f5dd0 msgr2=0x55d4448da2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 shutdown_connections 2023-10-16T14:36:37.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 1 -- 172.21.15.178:0/1222473153 wait complete. 2023-10-16T14:36:37.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.965+0000 7f8d7de19b80 20 client.25945 trim_cache size 0 max 16384 2023-10-16T14:36:37.981 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:37.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 msgr2=0x7f8d4c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 0x7f8d4c0a39c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009fd0 tx=0x7f8d58011b40 comp rx=0 tx=0).stop 2023-10-16T14:36:37.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 msgr2=0x7f8d641a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:37.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d641a04a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c009fd0 tx=0x7f8d5c007930 comp rx=0 tx=0).stop 2023-10-16T14:36:37.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 shutdown_connections 2023-10-16T14:36:37.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1500 0x7f8d4c0a39c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64106bf0 0x7f8d641a04a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64101990 0x7f8d64198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 --2- 172.21.15.178:0/3959425891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042c0 0x7f8d64199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:37.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 >> 172.21.15.178:0/3959425891 conn(0x7f8d640fb320 msgr2=0x7f8d64102620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:37.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 shutdown_connections 2023-10-16T14:36:37.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:37.966+0000 7f8d6a3ff700 1 -- 172.21.15.178:0/3959425891 wait complete. 2023-10-16T14:36:37.988 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:37 smithi178 ceph-mon[98495]: pgmap v426: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 576 op/s 2023-10-16T14:36:37.989 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:37 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3626884702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:37.991 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:36:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:37 smithi186 ceph-mon[134892]: pgmap v426: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 576 op/s 2023-10-16T14:36:38.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:37 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3626884702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:37 smithi187 ceph-mon[135297]: pgmap v426: 129 pgs: 129 active+clean; 347 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 576 op/s 2023-10-16T14:36:38.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:37 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3626884702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:36:38.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 -- 172.21.15.178:0/3417476790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278102a40 msgr2=0x7ff278109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/3417476790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278102a40 0x7ff278109360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b0a0 tx=0x7ff27000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 -- 172.21.15.178:0/3417476790 shutdown_connections 2023-10-16T14:36:38.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/3417476790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278102a40 0x7ff278109360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/3417476790 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278100ea0 0x7ff2781012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/3417476790 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102080 0x7ff278102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 -- 172.21.15.178:0/3417476790 >> 172.21.15.178:0/3417476790 conn(0x7ff2780fc460 msgr2=0x7ff2780fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 -- 172.21.15.178:0/3417476790 shutdown_connections 2023-10-16T14:36:38.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.210+0000 7ff27f1ff700 1 -- 172.21.15.178:0/3417476790 wait complete. 2023-10-16T14:36:38.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.211+0000 7ff27f1ff700 1 Processor -- start 2023-10-16T14:36:38.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.211+0000 7ff27f1ff700 1 -- start start 2023-10-16T14:36:38.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278102080 0x7ff2780732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 0x7ff2780737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278073f80 con 0x7ff278102080 2023-10-16T14:36:38.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2780740f0 con 0x7ff278102a40 2023-10-16T14:36:38.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27f1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2781a0d20 con 0x7ff278100ea0 2023-10-16T14:36:38.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 0x7ff2780737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50622/0 (socket says 172.21.15.178:50622) 2023-10-16T14:36:38.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 -- 172.21.15.178:0/4106660375 learned_addr learned my addr 172.21.15.178:0/4106660375 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:38.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff277fff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278102080 0x7ff2780732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 -- 172.21.15.178:0/4106660375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 msgr2=0x7ff2780737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 0x7ff2780737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 -- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278102080 msgr2=0x7ff2780732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278102080 0x7ff2780732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27cf9b700 1 -- 172.21.15.178:0/4106660375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270009d00 con 0x7ff278100ea0 2023-10-16T14:36:38.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.212+0000 7ff27d79c700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 0x7ff2780737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:38.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e970 tx=0x7ff26c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff275ffb700 1 -- 172.21.15.178:0/4106660375 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c009790 con 0x7ff278100ea0 2023-10-16T14:36:38.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff275ffb700 1 -- 172.21.15.178:0/4106660375 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26c00c7e0 con 0x7ff278100ea0 2023-10-16T14:36:38.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781a0fa0 con 0x7ff278100ea0 2023-10-16T14:36:38.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff275ffb700 1 -- 172.21.15.178:0/4106660375 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003c70 con 0x7ff278100ea0 2023-10-16T14:36:38.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.213+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781a1420 con 0x7ff278100ea0 2023-10-16T14:36:38.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.216+0000 7ff275ffb700 1 -- 172.21.15.178:0/4106660375 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff26c00cc50 con 0x7ff278100ea0 2023-10-16T14:36:38.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.218+0000 7ff292ab7b80 1 Processor -- start 2023-10-16T14:36:38.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 -- start start 2023-10-16T14:36:38.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff275ffb700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 0x7ff2680a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3adc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff275ffb700 1 -- 172.21.15.178:0/4106660375 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff26c014070 con 0x7ff278100ea0 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ad37c70 con 0x564d3adc3780 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ad1cc60 con 0x564d3adc2e20 2023-10-16T14:36:38.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ad1a580 con 0x564d3adc4620 2023-10-16T14:36:38.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27d79c700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 0x7ff2680a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56424/0 (socket says 172.21.15.178:56424) 2023-10-16T14:36:38.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 -- 172.21.15.178:0/1324181245 learned_addr learned my addr 172.21.15.178:0/1324181245 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:38.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 -- 172.21.15.178:0/1324181245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 msgr2=0x564d3adc4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27d79c700 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.219+0000 7ff27cf9b700 1 -- 172.21.15.178:0/1324181245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 msgr2=0x564d3adc4040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:36:38.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3adc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff27cf9b700 1 -- 172.21.15.178:0/1324181245 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3acd33f0 con 0x564d3adc2e20 2023-10-16T14:36:38.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff27d79c700 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:38.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09a0d0 tx=0x7ff26c09a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efdca8293fd95f06 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff262ffd700 1 -- 172.21.15.178:0/1324181245 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09e290 con 0x564d3adc2e20 2023-10-16T14:36:38.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff262ffd700 1 -- 172.21.15.178:0/1324181245 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26c0a24d0 con 0x564d3adc2e20 2023-10-16T14:36:38.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.220+0000 7ff262ffd700 1 -- 172.21.15.178:0/1324181245 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09a600 con 0x564d3adc2e20 2023-10-16T14:36:38.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 -- 172.21.15.178:0/1324181245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 msgr2=0x564d3adc3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09a0d0 tx=0x7ff26c09a040 comp rx=0 tx=0).stop 2023-10-16T14:36:38.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff27d79c700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 0x7ff2680a39c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b780 tx=0x7ff270006230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 -- 172.21.15.178:0/1324181245 shutdown_connections 2023-10-16T14:36:38.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/1324181245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3adc4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 -- 172.21.15.178:0/1324181245 >> 172.21.15.178:0/1324181245 conn(0x564d3acf0dd0 msgr2=0x564d3adbdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 -- 172.21.15.178:0/1324181245 shutdown_connections 2023-10-16T14:36:38.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.221+0000 7ff292ab7b80 1 -- 172.21.15.178:0/1324181245 wait complete. 2023-10-16T14:36:38.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 -- start start 2023-10-16T14:36:38.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 Processor -- start 2023-10-16T14:36:38.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc2e20 0x564d3adccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 0x564d3add41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ad15e70 con 0x564d3adc2e20 2023-10-16T14:36:38.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3acb2710 con 0x564d3adc4620 2023-10-16T14:36:38.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3acb23b0 con 0x564d3adc3780 2023-10-16T14:36:38.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.222+0000 7ff292ab7b80 10 client.0 _ll_register_callbacks cb 0x564d3ab0f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:38.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff27d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 0x564d3add41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc2e20 0x564d3adccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff27d79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 0x564d3add41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56432/0 (socket says 172.21.15.178:56432) 2023-10-16T14:36:38.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:50644/0 (socket says 172.21.15.178:50644) 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff27d79c700 1 -- 172.21.15.178:0/3877674139 learned_addr learned my addr 172.21.15.178:0/3877674139 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 -- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 msgr2=0x564d3add41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 0x564d3add41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 -- 172.21.15.178:0/3877674139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc2e20 msgr2=0x564d3adccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc2e20 0x564d3adccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 -- 172.21.15.178:0/3877674139 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781a1420 con 0x564d3adc3780 2023-10-16T14:36:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff277fff700 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x564d3adbef70 tx=0x7ff2640119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935e926c7e463b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff257fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.223+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264004230 con 0x564d3adc3780 2023-10-16T14:36:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d3ad1aef0 con 0x564d3adc3780 2023-10-16T14:36:38.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff264004390 con 0x564d3adc3780 2023-10-16T14:36:38.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff264004550 con 0x564d3adc3780 2023-10-16T14:36:38.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 10 client.25954 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:38.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564d3ab40de0 con 0x564d3adc3780 2023-10-16T14:36:38.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d3ad1aef0 con 0x564d3adc3780 2023-10-16T14:36:38.239 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff26400ea00 con 0x564d3adc3780 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 10 client.25954 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 20 client.25954 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3add8ae0 0x564d3addaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 4 client.25954 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:38.240 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.224+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3ad92c10 con 0x564d3add8ae0 2023-10-16T14:36:38.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.225+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3add8ae0 0x564d3addaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.225+0000 7ff257fff700 10 client.25954 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:38.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.225+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3add8ae0 0x564d3addaf00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47371a6ec8757f16 server_cookie=523f4837a840a181 in_seq=0 out_seq=0 2023-10-16T14:36:38.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff257fff700 1 -- 172.21.15.178:0/3877674139 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564d3ad92c10 con 0x564d3add8ae0 2023-10-16T14:36:38.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff257fff700 10 client.25954 handle_command_reply: tid=0 2023-10-16T14:36:38.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 client.25954 shutdown 2023-10-16T14:36:38.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 msgr2=0x564d3adcd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x564d3adbef70 tx=0x7ff2640119a0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 shutdown_connections 2023-10-16T14:36:38.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3add8ae0 0x564d3addaf00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc4620 0x564d3add41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc2e20 0x564d3adccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/3877674139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc3780 0x564d3adcd1b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff257fff700 0 client.25954 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 >> 172.21.15.178:0/3877674139 conn(0x564d3acf0dd0 msgr2=0x564d3adbdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 shutdown_connections 2023-10-16T14:36:38.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.229+0000 7ff292ab7b80 1 -- 172.21.15.178:0/3877674139 wait complete. 2023-10-16T14:36:38.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.230+0000 7ff292ab7b80 20 client.25954 trim_cache size 0 max 16384 2023-10-16T14:36:38.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.243+0000 7ff292ab7b80 1 Processor -- start 2023-10-16T14:36:38.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.243+0000 7ff292ab7b80 1 -- start start 2023-10-16T14:36:38.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ab96920 con 0x564d3adc3780 2023-10-16T14:36:38.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3add5220 con 0x564d3adc2e20 2023-10-16T14:36:38.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3adbe5b0 con 0x564d3adc4620 2023-10-16T14:36:38.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56440/0 (socket says 172.21.15.178:56440) 2023-10-16T14:36:38.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 -- 172.21.15.178:0/2456768046 learned_addr learned my addr 172.21.15.178:0/2456768046 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:38.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27d79c700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff277fff700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 -- 172.21.15.178:0/2456768046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 msgr2=0x564d3adc7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 -- 172.21.15.178:0/2456768046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 msgr2=0x564d3add41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27cf9b700 1 -- 172.21.15.178:0/2456768046 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3ab29c60 con 0x564d3adc2e20 2023-10-16T14:36:38.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff277fff700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:38.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.244+0000 7ff27d79c700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff26c099c10 tx=0x7ff26c099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bcb52f6743c79b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff2637fe700 1 -- 172.21.15.178:0/2456768046 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c098330 con 0x564d3adc2e20 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff2637fe700 1 -- 172.21.15.178:0/2456768046 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26c0b8310 con 0x564d3adc2e20 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff2637fe700 1 -- 172.21.15.178:0/2456768046 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c09a600 con 0x564d3adc2e20 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff292ab7b80 1 -- 172.21.15.178:0/2456768046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 msgr2=0x564d3adc3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.245+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff26c099c10 tx=0x7ff26c099fa0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 -- 172.21.15.178:0/2456768046 shutdown_connections 2023-10-16T14:36:38.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3adc7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3adc3200 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/2456768046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 -- 172.21.15.178:0/2456768046 >> 172.21.15.178:0/2456768046 conn(0x564d3acf0dd0 msgr2=0x564d3add4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 -- 172.21.15.178:0/2456768046 shutdown_connections 2023-10-16T14:36:38.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 -- 172.21.15.178:0/2456768046 wait complete. 2023-10-16T14:36:38.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 -- start start 2023-10-16T14:36:38.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.246+0000 7ff292ab7b80 1 Processor -- start 2023-10-16T14:36:38.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3add3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3add3a70 con 0x564d3adc3780 2023-10-16T14:36:38.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3add3be0 con 0x564d3adc2e20 2023-10-16T14:36:38.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3add3da0 con 0x564d3adc4620 2023-10-16T14:36:38.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff292ab7b80 10 client.0 _ll_register_callbacks cb 0x564d3ab0f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:38.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff277fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff27cf9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56448/0 (socket says 172.21.15.178:56448) 2023-10-16T14:36:38.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff27cf9b700 1 -- 172.21.15.178:0/955805394 learned_addr learned my addr 172.21.15.178:0/955805394 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:38.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.247+0000 7ff27d79c700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3add3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 -- 172.21.15.178:0/955805394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 msgr2=0x564d3add3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3add3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 -- 172.21.15.178:0/955805394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 msgr2=0x564d3add0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:38.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 -- 172.21.15.178:0/955805394 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d3acd33f0 con 0x564d3adc2e20 2023-10-16T14:36:38.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff27cf9b700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09a0d0 tx=0x7ff26c005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a46058b6d14596d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:38.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff257fff700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:36:38.261 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c098330 con 0x564d3adc2e20 2023-10-16T14:36:38.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff26c09a3a0 con 0x564d3adc2e20 2023-10-16T14:36:38.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26c0a24d0 con 0x564d3adc2e20 2023-10-16T14:36:38.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c09a560 con 0x564d3adc2e20 2023-10-16T14:36:38.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff292ab7b80 10 client.25973 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:38.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.248+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564d3ab67250 con 0x564d3adc2e20 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26c0a24d0 con 0x564d3adc2e20 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7ff26c095f30 con 0x564d3adc2e20 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 10 client.25973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 20 client.25973 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:38.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3ae2d4d0 0x564d3ae2f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:38.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 4 client.25973 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:36:38.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564d3ad0ee50 con 0x564d3ae2d4d0 2023-10-16T14:36:38.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff277fff700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3ae2d4d0 0x564d3ae2f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:38.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff277fff700 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3ae2d4d0 0x564d3ae2f8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=397a5bd93a5f97a1 server_cookie=2b6ad671b89ec124 in_seq=0 out_seq=0 2023-10-16T14:36:38.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.249+0000 7ff257fff700 10 client.25973 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:38.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.253+0000 7ff257fff700 1 -- 172.21.15.178:0/955805394 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564d3ad0ee50 con 0x564d3ae2d4d0 2023-10-16T14:36:38.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff257fff700 10 client.25973 handle_command_reply: tid=0 2023-10-16T14:36:38.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 client.25973 shutdown 2023-10-16T14:36:38.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 msgr2=0x564d3add03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff26c09a0d0 tx=0x7ff26c005b40 comp rx=0 tx=0).stop 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 shutdown_connections 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x564d3ae2d4d0 0x564d3ae2f8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3adc4620 0x564d3add3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x564d3adc2e20 0x564d3add03e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff257fff700 0 client.25973 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:38.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 --2- 172.21.15.178:0/955805394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x564d3adc3780 0x564d3add0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 >> 172.21.15.178:0/955805394 conn(0x564d3acf0dd0 msgr2=0x564d3adc7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 shutdown_connections 2023-10-16T14:36:38.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 1 -- 172.21.15.178:0/955805394 wait complete. 2023-10-16T14:36:38.267 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.254+0000 7ff292ab7b80 20 client.25973 trim_cache size 0 max 16384 2023-10-16T14:36:38.268 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:36:38.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 msgr2=0x7ff2680a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.268 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 0x7ff2680a39c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff27000b780 tx=0x7ff270006230 comp rx=0 tx=0).stop 2023-10-16T14:36:38.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 msgr2=0x7ff278074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:38.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00e970 tx=0x7ff26c00ec80 comp rx=0 tx=0).stop 2023-10-16T14:36:38.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 shutdown_connections 2023-10-16T14:36:38.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff2680a1500 0x7ff2680a39c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.269 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff278102a40 0x7ff2780737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff278100ea0 0x7ff278074c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 --2- 172.21.15.178:0/4106660375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff278102080 0x7ff2780732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:38.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 >> 172.21.15.178:0/4106660375 conn(0x7ff2780fc460 msgr2=0x7ff2780fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:38.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 shutdown_connections 2023-10-16T14:36:38.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:38.255+0000 7ff27f1ff700 1 -- 172.21.15.178:0/4106660375 wait complete. 2023-10-16T14:36:39.278 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:36:39.278 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:36:39.757 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:39 smithi178 ceph-mon[98495]: pgmap v427: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-16T14:36:39.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.964+0000 7eff6619d700 1 -- 172.21.15.178:0/2144654523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010a6a0 msgr2=0x7eff6010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:39.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.964+0000 7eff6619d700 1 --2- 172.21.15.178:0/2144654523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010a6a0 0x7eff6010ab00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b0d0 tx=0x7eff5800b3e0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.969+0000 7eff6619d700 1 -- 172.21.15.178:0/2144654523 shutdown_connections 2023-10-16T14:36:39.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.969+0000 7eff6619d700 1 --2- 172.21.15.178:0/2144654523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff6010b040 0x7eff601118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.969+0000 7eff6619d700 1 --2- 172.21.15.178:0/2144654523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff601098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.969+0000 7eff6619d700 1 --2- 172.21.15.178:0/2144654523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010a6a0 0x7eff6010ab00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.969+0000 7eff6619d700 1 -- 172.21.15.178:0/2144654523 >> 172.21.15.178:0/2144654523 conn(0x7eff6006fc70 msgr2=0x7eff60072090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:39.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.976+0000 7eff6619d700 1 -- 172.21.15.178:0/2144654523 shutdown_connections 2023-10-16T14:36:39.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.976+0000 7eff6619d700 1 -- 172.21.15.178:0/2144654523 wait complete. 2023-10-16T14:36:39.984 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.982+0000 7eff6619d700 1 Processor -- start 2023-10-16T14:36:39.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 -- start start 2023-10-16T14:36:39.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff6019d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:39.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff6010a6a0 0x7eff6019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:39.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:39.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff601a0650 con 0x7eff6010a6a0 2023-10-16T14:36:39.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff601a07c0 con 0x7eff6010b040 2023-10-16T14:36:39.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.984+0000 7eff6619d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff601b5f70 con 0x7eff601094b0 2023-10-16T14:36:39.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.985+0000 7eff6599c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:39.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.985+0000 7eff6599c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52718/0 (socket says 172.21.15.178:52718) 2023-10-16T14:36:39.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.985+0000 7eff6599c700 1 -- 172.21.15.178:0/2871700300 learned_addr learned my addr 172.21.15.178:0/2871700300 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.985+0000 7eff6519b700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff6019d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.986+0000 7eff6599c700 1 -- 172.21.15.178:0/2871700300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 msgr2=0x7eff6019d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.986+0000 7eff6599c700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff6019d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.986+0000 7eff6599c700 1 -- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff6010a6a0 msgr2=0x7eff6019d570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:36:39.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.986+0000 7eff6599c700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff6010a6a0 0x7eff6019d570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:39.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.986+0000 7eff6599c700 1 -- 172.21.15.178:0/2871700300 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009d00 con 0x7eff6010b040 2023-10-16T14:36:39.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.987+0000 7eff6519b700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff6019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:39.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.987+0000 7eff6599c700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d980 tx=0x7eff5c00dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:39.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.990+0000 7eff567fc700 1 -- 172.21.15.178:0/2871700300 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004220 con 0x7eff6010b040 2023-10-16T14:36:39.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.990+0000 7eff567fc700 1 -- 172.21.15.178:0/2871700300 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5c007e20 con 0x7eff6010b040 2023-10-16T14:36:39.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.990+0000 7eff567fc700 1 -- 172.21.15.178:0/2871700300 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004dd0 con 0x7eff6010b040 2023-10-16T14:36:39.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.991+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff601b61d0 con 0x7eff6010b040 2023-10-16T14:36:39.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:39.991+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff601b6660 con 0x7eff6010b040 2023-10-16T14:36:40.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.009+0000 7eff567fc700 1 -- 172.21.15.178:0/2871700300 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7eff5c00b900 con 0x7eff6010b040 2023-10-16T14:36:40.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.011+0000 7eff567fc700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 0x7eff4c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.011+0000 7eff567fc700 1 -- 172.21.15.178:0/2871700300 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7eff5c0ccd20 con 0x7eff6010b040 2023-10-16T14:36:40.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.013+0000 7eff6519b700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 0x7eff4c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.014+0000 7eff6519b700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 0x7eff4c0a3af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff50006fd0 tx=0x7eff50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.026+0000 7eff7ac44b80 1 Processor -- start 2023-10-16T14:36:40.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 -- start start 2023-10-16T14:36:40.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e50450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445dc4cf0 con 0x55d445e51880 2023-10-16T14:36:40.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445da9ce0 con 0x55d445e50030 2023-10-16T14:36:40.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.032+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445da7600 con 0x55d445e50990 2023-10-16T14:36:40.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.033+0000 7eff6599c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.033+0000 7eff6599c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33512/0 (socket says 172.21.15.178:33512) 2023-10-16T14:36:40.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.033+0000 7eff6599c700 1 -- 172.21.15.178:0/152929308 learned_addr learned my addr 172.21.15.178:0/152929308 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.038+0000 7eff6499a700 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 -- 172.21.15.178:0/152929308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 msgr2=0x55d445e50450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:36:40.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e50450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 -- 172.21.15.178:0/152929308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 msgr2=0x55d445e51d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 -- 172.21.15.178:0/152929308 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d445d60470 con 0x55d445e50990 2023-10-16T14:36:40.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6599c700 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:40.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.039+0000 7eff6499a700 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e512a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b820 tx=0x7eff58009c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac49561b54565f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.040+0000 7eff4b7fe700 1 -- 172.21.15.178:0/152929308 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800e040 con 0x55d445e50990 2023-10-16T14:36:40.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.040+0000 7eff4b7fe700 1 -- 172.21.15.178:0/152929308 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff580092d0 con 0x55d445e50990 2023-10-16T14:36:40.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.040+0000 7eff4b7fe700 1 -- 172.21.15.178:0/152929308 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5801b690 con 0x55d445e50990 2023-10-16T14:36:40.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.041+0000 7eff7ac44b80 1 -- 172.21.15.178:0/152929308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 msgr2=0x55d445e512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.041+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e512a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff5800b820 tx=0x7eff58009c80 comp rx=0 tx=0).stop 2023-10-16T14:36:40.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.046+0000 7eff7ac44b80 1 -- 172.21.15.178:0/152929308 shutdown_connections 2023-10-16T14:36:40.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.046+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e51880 0x55d445e51d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.046+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e50450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.046+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/152929308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e512a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.046+0000 7eff7ac44b80 1 -- 172.21.15.178:0/152929308 >> 172.21.15.178:0/152929308 conn(0x55d445d7de50 msgr2=0x55d445e4c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.049+0000 7eff7ac44b80 1 -- 172.21.15.178:0/152929308 shutdown_connections 2023-10-16T14:36:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.050+0000 7eff7ac44b80 1 -- 172.21.15.178:0/152929308 wait complete. 2023-10-16T14:36:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.055+0000 7eff7ac44b80 1 -- start start 2023-10-16T14:36:40.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.055+0000 7eff7ac44b80 1 Processor -- start 2023-10-16T14:36:40.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e5a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 0x55d445e61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445da2ef0 con 0x55d445e50030 2023-10-16T14:36:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445d3f790 con 0x55d445e51880 2023-10-16T14:36:40.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445d3f430 con 0x55d445e50990 2023-10-16T14:36:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.065+0000 7eff7ac44b80 10 client.0 _ll_register_callbacks cb 0x55d445b9c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.066+0000 7eff6519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.066+0000 7eff6519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33524/0 (socket says 172.21.15.178:33524) 2023-10-16T14:36:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.066+0000 7eff6519b700 1 -- 172.21.15.178:0/2719523611 learned_addr learned my addr 172.21.15.178:0/2719523611 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6499a700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e5a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6599c700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 0x55d445e61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6519b700 1 -- 172.21.15.178:0/2719523611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 msgr2=0x55d445e5a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6519b700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e5a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6519b700 1 -- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 msgr2=0x55d445e61420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6519b700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 0x55d445e61420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6519b700 1 -- 172.21.15.178:0/2719523611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff601b6660 con 0x55d445e50030 2023-10-16T14:36:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.067+0000 7eff6599c700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 0x55d445e61420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:40.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.068+0000 7eff6519b700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7eff5000cd20 tx=0x7eff500047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8368c00a168ef33a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.069+0000 7eff48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:36:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.069+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ae80 con 0x55d445e50030 2023-10-16T14:36:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.069+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7eff5000ce00 con 0x55d445e50030 2023-10-16T14:36:40.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.069+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d445da7f70 con 0x55d445e50030 2023-10-16T14:36:40.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:39 smithi186 ceph-mon[134892]: pgmap v427: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-16T14:36:40.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.076+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff50013ba0 con 0x55d445e50030 2023-10-16T14:36:40.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.076+0000 7eff7ac44b80 10 client.15924 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:40.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.076+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7eff60112240 con 0x55d445e50030 2023-10-16T14:36:40.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.080+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d445da7f70 con 0x55d445e50030 2023-10-16T14:36:40.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.080+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7eff5001d410 con 0x55d445e50030 2023-10-16T14:36:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 20 client.15924 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:40.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445e65cf0 0x55d445e68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 4 client.15924 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d445e1fc90 con 0x55d445e65cf0 2023-10-16T14:36:40.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.082+0000 7eff6499a700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445e65cf0 0x55d445e68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.088+0000 7eff6499a700 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445e65cf0 0x55d445e68110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a0b56471482c9f server_cookie=324ca342c1b33143 in_seq=0 out_seq=0 2023-10-16T14:36:40.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.088+0000 7eff48ff9700 10 client.15924 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.091 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.089+0000 7eff48ff9700 1 -- 172.21.15.178:0/2719523611 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d445e1fc90 con 0x55d445e65cf0 2023-10-16T14:36:40.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.089+0000 7eff48ff9700 10 client.15924 handle_command_reply: tid=0 2023-10-16T14:36:40.092 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.089+0000 7eff7ac44b80 1 client.15924 shutdown 2023-10-16T14:36:40.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.101+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 msgr2=0x55d445e59ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.101+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7eff5000cd20 tx=0x7eff500047e0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 shutdown_connections 2023-10-16T14:36:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445e65cf0 0x55d445e68110 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e51880 0x55d445e61420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e59ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2719523611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e50990 0x55d445e5a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 >> 172.21.15.178:0/2719523611 conn(0x55d445d7de50 msgr2=0x55d445e4adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.103+0000 7eff48ff9700 0 client.15924 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.105+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 shutdown_connections 2023-10-16T14:36:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.105+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2719523611 wait complete. 2023-10-16T14:36:40.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.105+0000 7eff7ac44b80 20 client.15924 trim_cache size 0 max 16384 2023-10-16T14:36:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:39 smithi187 ceph-mon[135297]: pgmap v427: 129 pgs: 129 active+clean; 369 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 630 op/s 2023-10-16T14:36:40.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.187+0000 7eff7ac44b80 1 Processor -- start 2023-10-16T14:36:40.190 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.187+0000 7eff7ac44b80 1 -- start start 2023-10-16T14:36:40.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.188+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.189+0000 7eff6519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.189+0000 7eff6519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33526/0 (socket says 172.21.15.178:33526) 2023-10-16T14:36:40.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.190+0000 7eff6519b700 1 -- 172.21.15.178:0/2108752420 learned_addr learned my addr 172.21.15.178:0/2108752420 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.190+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50990 0x55d445e613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.190+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445e544d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.190+0000 7eff6499a700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50990 0x55d445e613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.191+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445c239a0 con 0x55d445e50030 2023-10-16T14:36:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.191+0000 7eff6599c700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445e544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.191+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445e5aef0 con 0x55d445e50990 2023-10-16T14:36:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.191+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445e52100 con 0x55d445e51880 2023-10-16T14:36:40.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6599c700 1 -- 172.21.15.178:0/2108752420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50990 msgr2=0x55d445e613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6599c700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50990 0x55d445e613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6599c700 1 -- 172.21.15.178:0/2108752420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 msgr2=0x55d445e5a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6599c700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6599c700 1 -- 172.21.15.178:0/2108752420 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d445e6cd60 con 0x55d445e51880 2023-10-16T14:36:40.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.192+0000 7eff6519b700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:36:40.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.193+0000 7eff6599c700 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445e544d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d445e51060 tx=0x7eff5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31bff34b1afc6b02 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.194+0000 7eff4bfff700 1 -- 172.21.15.178:0/2108752420 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c09a370 con 0x55d445e51880 2023-10-16T14:36:40.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.194+0000 7eff4bfff700 1 -- 172.21.15.178:0/2108752420 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff5c099c70 con 0x55d445e51880 2023-10-16T14:36:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.194+0000 7eff4bfff700 1 -- 172.21.15.178:0/2108752420 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00e5b0 con 0x55d445e51880 2023-10-16T14:36:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.195+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 msgr2=0x55d445e544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.195+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445e544d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55d445e51060 tx=0x7eff5c004750 comp rx=0 tx=0).stop 2023-10-16T14:36:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.195+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 shutdown_connections 2023-10-16T14:36:40.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.195+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445e544d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.195+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50030 0x55d445e5a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.196+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/2108752420 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50990 0x55d445e613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.196+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 >> 172.21.15.178:0/2108752420 conn(0x55d445d7de50 msgr2=0x55d445e59c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.196+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 shutdown_connections 2023-10-16T14:36:40.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.196+0000 7eff7ac44b80 1 -- 172.21.15.178:0/2108752420 wait complete. 2023-10-16T14:36:40.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.197+0000 7eff7ac44b80 1 -- start start 2023-10-16T14:36:40.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.198+0000 7eff7ac44b80 1 Processor -- start 2023-10-16T14:36:40.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 0x55d445e60880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445eb9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445eba070 con 0x55d445e50990 2023-10-16T14:36:40.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445eba1b0 con 0x55d445e50030 2023-10-16T14:36:40.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445eba320 con 0x55d445e51880 2023-10-16T14:36:40.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.199+0000 7eff7ac44b80 10 client.0 _ll_register_callbacks cb 0x7eff5000c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.200+0000 7eff6519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.200+0000 7eff6519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e60340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52754/0 (socket says 172.21.15.178:52754) 2023-10-16T14:36:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.200+0000 7eff6519b700 1 -- 172.21.15.178:0/879558191 learned_addr learned my addr 172.21.15.178:0/879558191 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6599c700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445eb9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 0x55d445e60880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 -- 172.21.15.178:0/879558191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 msgr2=0x55d445eb9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445eb9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 -- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 msgr2=0x55d445e60340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e60340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.201+0000 7eff6499a700 1 -- 172.21.15.178:0/879558191 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff60112240 con 0x55d445e50990 2023-10-16T14:36:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.202+0000 7eff6499a700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 0x55d445e60880 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55d445e68a10 tx=0x7eff580099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9050dabb174f4aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.202+0000 7eff48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:36:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.203+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5800e040 con 0x55d445e50990 2023-10-16T14:36:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.203+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7eff58003c00 con 0x55d445e50990 2023-10-16T14:36:40.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.203+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff580092d0 con 0x55d445e50990 2023-10-16T14:36:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.203+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff58003dc0 con 0x55d445e50990 2023-10-16T14:36:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.203+0000 7eff7ac44b80 10 client.15930 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7eff601b61d0 con 0x55d445e50990 2023-10-16T14:36:40.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff580092d0 con 0x55d445e50990 2023-10-16T14:36:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7eff58038070 con 0x55d445e50990 2023-10-16T14:36:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff7ac44b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:40.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff7ac44b80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff7ac44b80 20 client.15930 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.204+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445ebb8c0 0x55d445ebdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.205+0000 7eff6519b700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445ebb8c0 0x55d445ebdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.205+0000 7eff7ac44b80 4 client.15930 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:36:40.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.205+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d445d9bed0 con 0x55d445ebb8c0 2023-10-16T14:36:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.205+0000 7eff48ff9700 10 client.15930 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.206+0000 7eff6519b700 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445ebb8c0 0x55d445ebdce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a674bc85dc8799 server_cookie=3e70c4b5c142c1f5 in_seq=0 out_seq=0 2023-10-16T14:36:40.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.208+0000 7eff48ff9700 1 -- 172.21.15.178:0/879558191 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d445d9bed0 con 0x55d445ebb8c0 2023-10-16T14:36:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.208+0000 7eff48ff9700 10 client.15930 handle_command_reply: tid=0 2023-10-16T14:36:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.208+0000 7eff7ac44b80 1 client.15930 shutdown 2023-10-16T14:36:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 msgr2=0x55d445e60880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 0x55d445e60880 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x55d445e68a10 tx=0x7eff580099e0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 shutdown_connections 2023-10-16T14:36:40.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55d445ebb8c0 0x55d445ebdce0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d445e51880 0x55d445eb9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d445e50030 0x55d445e60340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 --2- 172.21.15.178:0/879558191 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d445e50990 0x55d445e60880 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 >> 172.21.15.178:0/879558191 conn(0x55d445d7de50 msgr2=0x55d445e544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.209+0000 7eff48ff9700 0 client.15930 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.210+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 shutdown_connections 2023-10-16T14:36:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.210+0000 7eff7ac44b80 1 -- 172.21.15.178:0/879558191 wait complete. 2023-10-16T14:36:40.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.210+0000 7eff7ac44b80 20 client.15930 trim_cache size 0 max 16384 2023-10-16T14:36:40.215 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:36:40.215 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:36:40.216 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "71a14aaa-7be0-41f2-81f0-d10cfdf98b60", 2023-10-16T14:36:40.216 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:36:40.216 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:36:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 msgr2=0x7eff4c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 0x7eff4c0a3af0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff50006fd0 tx=0x7eff50008040 comp rx=0 tx=0).stop 2023-10-16T14:36:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 msgr2=0x7eff601b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d980 tx=0x7eff5c00dd40 comp rx=0 tx=0).stop 2023-10-16T14:36:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 shutdown_connections 2023-10-16T14:36:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7eff4c0a1630 0x7eff4c0a3af0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.212+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7eff6010b040 0x7eff601b5a30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.213+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff601094b0 0x7eff6019d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.213+0000 7eff6619d700 1 --2- 172.21.15.178:0/2871700300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff6010a6a0 0x7eff6019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.213+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 >> 172.21.15.178:0/2871700300 conn(0x7eff6006fc70 msgr2=0x7eff6010dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.213+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 shutdown_connections 2023-10-16T14:36:40.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.213+0000 7eff6619d700 1 -- 172.21.15.178:0/2871700300 wait complete. 2023-10-16T14:36:40.252 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:36:40.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.610+0000 7fda73fff700 1 -- 172.21.15.178:0/4249184879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 msgr2=0x7fda74107790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.610+0000 7fda73fff700 1 --2- 172.21.15.178:0/4249184879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74107790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fda68009990 tx=0x7fda68009ca0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.613+0000 7fda73fff700 1 -- 172.21.15.178:0/4249184879 shutdown_connections 2023-10-16T14:36:40.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.614+0000 7fda73fff700 1 --2- 172.21.15.178:0/4249184879 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108f00 0x7fda7410f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.614+0000 7fda73fff700 1 --2- 172.21.15.178:0/4249184879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74107790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.614+0000 7fda73fff700 1 --2- 172.21.15.178:0/4249184879 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108560 0x7fda741089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.614+0000 7fda73fff700 1 -- 172.21.15.178:0/4249184879 >> 172.21.15.178:0/4249184879 conn(0x7fda74076310 msgr2=0x7fda74078750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.620+0000 7fda73fff700 1 -- 172.21.15.178:0/4249184879 shutdown_connections 2023-10-16T14:36:40.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.623+0000 7fda73fff700 1 -- 172.21.15.178:0/4249184879 wait complete. 2023-10-16T14:36:40.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.628+0000 7fda73fff700 1 Processor -- start 2023-10-16T14:36:40.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 -- start start 2023-10-16T14:36:40.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74081960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 0x7fda7407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 0x7fda7407ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda741ad250 con 0x7fda74108f00 2023-10-16T14:36:40.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7406a8d0 con 0x7fda74108560 2023-10-16T14:36:40.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.631+0000 7fda73fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7406aa90 con 0x7fda74107370 2023-10-16T14:36:40.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74081960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 0x7fda7407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74081960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:53178/0 (socket says 172.21.15.178:53178) 2023-10-16T14:36:40.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda72ffd700 1 -- 172.21.15.178:0/1064710397 learned_addr learned my addr 172.21.15.178:0/1064710397 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda727fc700 1 -- 172.21.15.178:0/1064710397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 msgr2=0x7fda74081960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda727fc700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74081960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.632+0000 7fda727fc700 1 -- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 msgr2=0x7fda7407ea30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:36:40.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.633+0000 7fda737fe700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 0x7fda7407ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.633+0000 7fda727fc700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 0x7fda7407ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.633+0000 7fda727fc700 1 -- 172.21.15.178:0/1064710397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680095f0 con 0x7fda74108560 2023-10-16T14:36:40.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.633+0000 7fda737fe700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 0x7fda7407ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:40.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.639+0000 7fda727fc700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 0x7fda7407e3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda6c007d60 tx=0x7fda6c00a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.640+0000 7fda5bfff700 1 -- 172.21.15.178:0/1064710397 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c00ad40 con 0x7fda74108560 2023-10-16T14:36:40.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.640+0000 7fda5bfff700 1 -- 172.21.15.178:0/1064710397 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda6c0128e0 con 0x7fda74108560 2023-10-16T14:36:40.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.640+0000 7fda5bfff700 1 -- 172.21.15.178:0/1064710397 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6c011690 con 0x7fda74108560 2023-10-16T14:36:40.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.641+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7406ad70 con 0x7fda74108560 2023-10-16T14:36:40.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.641+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7406b230 con 0x7fda74108560 2023-10-16T14:36:40.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.649+0000 7fda5bfff700 1 -- 172.21.15.178:0/1064710397 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fda6c012420 con 0x7fda74108560 2023-10-16T14:36:40.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.657+0000 7fda5bfff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 0x7fda5c0a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.657+0000 7fda5bfff700 1 -- 172.21.15.178:0/1064710397 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda6c0ccfb0 con 0x7fda74108560 2023-10-16T14:36:40.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.663+0000 7fda72ffd700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 0x7fda5c0a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.664+0000 7fda72ffd700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 0x7fda5c0a3b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda68006010 tx=0x7fda680058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.670+0000 7fda8cd94b80 1 Processor -- start 2023-10-16T14:36:40.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.674+0000 7fda8cd94b80 1 -- start start 2023-10-16T14:36:40.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e260220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e260c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e1d3c70 con 0x558e1e25efb0 2023-10-16T14:36:40.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e1b8c60 con 0x558e1e260800 2023-10-16T14:36:40.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.676+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e1b6580 con 0x558e1e25f910 2023-10-16T14:36:40.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.677+0000 7fda737fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e260c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.677+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.677+0000 7fda72ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:33570/0 (socket says 172.21.15.178:33570) 2023-10-16T14:36:40.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.677+0000 7fda72ffd700 1 -- 172.21.15.178:0/3190929899 learned_addr learned my addr 172.21.15.178:0/3190929899 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.679+0000 7fda72ffd700 1 -- 172.21.15.178:0/3190929899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 msgr2=0x558e1e260220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:36:40.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.680+0000 7fda727fc700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e260220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.680+0000 7fda72ffd700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e260220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.680+0000 7fda72ffd700 1 -- 172.21.15.178:0/3190929899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 msgr2=0x558e1e260c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.680+0000 7fda72ffd700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e260c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.681+0000 7fda72ffd700 1 -- 172.21.15.178:0/3190929899 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e1e16f3f0 con 0x558e1e25efb0 2023-10-16T14:36:40.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.681+0000 7fda727fc700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e260220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:40.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.681+0000 7fda737fe700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e260c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:40.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.681+0000 7fda72ffd700 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fda68005570 tx=0x7fda68020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5419d20dc648c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.683+0000 7fda597fa700 1 -- 172.21.15.178:0/3190929899 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800b7b0 con 0x558e1e25efb0 2023-10-16T14:36:40.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.684+0000 7fda597fa700 1 -- 172.21.15.178:0/3190929899 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda6801e040 con 0x558e1e25efb0 2023-10-16T14:36:40.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.684+0000 7fda597fa700 1 -- 172.21.15.178:0/3190929899 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800b7b0 con 0x558e1e25efb0 2023-10-16T14:36:40.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.687+0000 7fda8cd94b80 1 -- 172.21.15.178:0/3190929899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 msgr2=0x558e1e25f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.687+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fda68005570 tx=0x7fda68020000 comp rx=0 tx=0).stop 2023-10-16T14:36:40.693 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.689+0000 7fda8cd94b80 1 -- 172.21.15.178:0/3190929899 shutdown_connections 2023-10-16T14:36:40.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.689+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e260c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.689+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25efb0 0x558e1e25f3d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.689+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/3190929899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e260220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.694 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.689+0000 7fda8cd94b80 1 -- 172.21.15.178:0/3190929899 >> 172.21.15.178:0/3190929899 conn(0x558e1e18cdd0 msgr2=0x558e1e25b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.701 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.699+0000 7fda8cd94b80 1 -- 172.21.15.178:0/3190929899 shutdown_connections 2023-10-16T14:36:40.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.699+0000 7fda8cd94b80 1 -- 172.21.15.178:0/3190929899 wait complete. 2023-10-16T14:36:40.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.700+0000 7fda8cd94b80 1 -- start start 2023-10-16T14:36:40.705 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.702+0000 7fda8cd94b80 1 Processor -- start 2023-10-16T14:36:40.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e25f910 0x558e1e268f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 0x558e1e2694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda737fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda737fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:53196/0 (socket says 172.21.15.178:53196) 2023-10-16T14:36:40.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.704+0000 7fda737fe700 1 -- 172.21.15.178:0/160231469 learned_addr learned my addr 172.21.15.178:0/160231469 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.705+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e1b1e70 con 0x558e1e260800 2023-10-16T14:36:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.705+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e14e710 con 0x558e1e25f910 2023-10-16T14:36:40.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.705+0000 7fda727fc700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 0x558e1e2694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.705+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e14e3b0 con 0x558e1e26e500 2023-10-16T14:36:40.710 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda8cd94b80 10 client.0 _ll_register_callbacks cb 0x558e1dfab800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 -- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e25f910 msgr2=0x558e1e268f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:36:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e25f910 0x558e1e268f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 -- 172.21.15.178:0/160231469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 msgr2=0x558e1e2694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 0x558e1e2694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda74110100 con 0x558e1e26e500 2023-10-16T14:36:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.706+0000 7fda737fe700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda64011660 tx=0x7fda64011a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92a8be76be43279 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.707+0000 7fda727fc700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 0x558e1e2694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:36:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.712+0000 7fda56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:40.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.712+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64004a30 con 0x558e1e26e500 2023-10-16T14:36:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.712+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda64004b90 con 0x558e1e26e500 2023-10-16T14:36:40.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.714+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e1e1b6ef0 con 0x558e1e26e500 2023-10-16T14:36:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.718+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda64004d50 con 0x558e1e26e500 2023-10-16T14:36:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.719+0000 7fda8cd94b80 10 client.25981 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:40.722 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.719+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e1dfdcde0 con 0x558e1e26e500 2023-10-16T14:36:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.721+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda640054e0 con 0x558e1e26e500 2023-10-16T14:36:40.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.721+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fda64012cd0 con 0x558e1e26e500 2023-10-16T14:36:40.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 10 client.25981 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:40.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 10 client.25981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 20 client.25981 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2750b0 0x558e1e277530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 4 client.25981 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:36:40.732 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e1e1b6ef0 con 0x558e1e2750b0 2023-10-16T14:36:40.733 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.729+0000 7fda72ffd700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2750b0 0x558e1e277530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.732+0000 7fda72ffd700 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2750b0 0x558e1e277530 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17488e50dbcceff3 server_cookie=f62ddf18226f9397 in_seq=0 out_seq=0 2023-10-16T14:36:40.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.732+0000 7fda56ff5700 10 client.25981 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.742+0000 7fda56ff5700 1 -- 172.21.15.178:0/160231469 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558e1e1b6ef0 con 0x558e1e2750b0 2023-10-16T14:36:40.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.742+0000 7fda56ff5700 10 client.25981 handle_command_reply: tid=0 2023-10-16T14:36:40.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.742+0000 7fda8cd94b80 1 client.25981 shutdown 2023-10-16T14:36:40.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.743+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 msgr2=0x558e1e2708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.743+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fda64011660 tx=0x7fda64011a20 comp rx=0 tx=0).stop 2023-10-16T14:36:40.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.743+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 shutdown_connections 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2750b0 0x558e1e277530 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2708f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e25f910 0x558e1e268f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/160231469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e260800 0x558e1e2694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 >> 172.21.15.178:0/160231469 conn(0x558e1e18cdd0 msgr2=0x558e1e259d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.744+0000 7fda56ff5700 0 client.25981 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.745+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 shutdown_connections 2023-10-16T14:36:40.751 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.747+0000 7fda8cd94b80 1 -- 172.21.15.178:0/160231469 wait complete. 2023-10-16T14:36:40.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.756+0000 7fda8cd94b80 20 client.25981 trim_cache size 0 max 16384 2023-10-16T14:36:40.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.803+0000 7fda8cd94b80 1 Processor -- start 2023-10-16T14:36:40.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.803+0000 7fda8cd94b80 1 -- start start 2023-10-16T14:36:40.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e284d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e263a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e26e500 0x558e1e268f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e032920 con 0x558e1e26e500 2023-10-16T14:36:40.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e25ebf0 con 0x558e1e260800 2023-10-16T14:36:40.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e256700 con 0x558e1e25f910 2023-10-16T14:36:40.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e263a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e263a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52802/0 (socket says 172.21.15.178:52802) 2023-10-16T14:36:40.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda727fc700 1 -- 172.21.15.178:0/2680742138 learned_addr learned my addr 172.21.15.178:0/2680742138 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.809 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda737fe700 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e26e500 0x558e1e268f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.804+0000 7fda72ffd700 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e284d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.805+0000 7fda72ffd700 1 -- 172.21.15.178:0/2680742138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 msgr2=0x558e1e263a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.805+0000 7fda72ffd700 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e263a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.805+0000 7fda72ffd700 1 -- 172.21.15.178:0/2680742138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e26e500 msgr2=0x558e1e268f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.810 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.805+0000 7fda72ffd700 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e26e500 0x558e1e268f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.805+0000 7fda72ffd700 1 -- 172.21.15.178:0/2680742138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e1dfc5c60 con 0x558e1e25f910 2023-10-16T14:36:40.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.806+0000 7fda72ffd700 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e284d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda68026000 tx=0x7fda68029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee2b4fab027928c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.807+0000 7fda59ffb700 1 -- 172.21.15.178:0/2680742138 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6800b7b0 con 0x558e1e25f910 2023-10-16T14:36:40.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.807+0000 7fda59ffb700 1 -- 172.21.15.178:0/2680742138 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda68004570 con 0x558e1e25f910 2023-10-16T14:36:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.807+0000 7fda59ffb700 1 -- 172.21.15.178:0/2680742138 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda6803c3c0 con 0x558e1e25f910 2023-10-16T14:36:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.809+0000 7fda8cd94b80 1 -- 172.21.15.178:0/2680742138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 msgr2=0x558e1e284d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.809+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e284d50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fda68026000 tx=0x7fda68029040 comp rx=0 tx=0).stop 2023-10-16T14:36:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.809+0000 7fda8cd94b80 1 -- 172.21.15.178:0/2680742138 shutdown_connections 2023-10-16T14:36:40.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.809+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e26e500 0x558e1e268f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.810+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e25f910 0x558e1e284d50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.810+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/2680742138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e263a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.810+0000 7fda8cd94b80 1 -- 172.21.15.178:0/2680742138 >> 172.21.15.178:0/2680742138 conn(0x558e1e18cdd0 msgr2=0x558e1e267a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.810+0000 7fda8cd94b80 1 -- 172.21.15.178:0/2680742138 shutdown_connections 2023-10-16T14:36:40.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.811+0000 7fda8cd94b80 1 -- 172.21.15.178:0/2680742138 wait complete. 2023-10-16T14:36:40.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.811+0000 7fda8cd94b80 1 -- start start 2023-10-16T14:36:40.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.812+0000 7fda8cd94b80 1 Processor -- start 2023-10-16T14:36:40.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25f910 0x558e1e2c3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e2c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2c8ad0 con 0x558e1e25f910 2023-10-16T14:36:40.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2c8c40 con 0x558e1e260800 2023-10-16T14:36:40.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e1e2c8e00 con 0x558e1e26e500 2023-10-16T14:36:40.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.813+0000 7fda8cd94b80 10 client.0 _ll_register_callbacks cb 0x7fda64009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:36:40.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e2c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda727fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e2c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:52818/0 (socket says 172.21.15.178:52818) 2023-10-16T14:36:40.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda727fc700 1 -- 172.21.15.178:0/1371117333 learned_addr learned my addr 172.21.15.178:0/1371117333 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:36:40.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda737fe700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda72ffd700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25f910 0x558e1e2c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda737fe700 1 -- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 msgr2=0x558e1e2c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda737fe700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e2c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.815+0000 7fda737fe700 1 -- 172.21.15.178:0/1371117333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25f910 msgr2=0x558e1e2c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:36:40.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.816+0000 7fda737fe700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25f910 0x558e1e2c3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.818+0000 7fda737fe700 1 -- 172.21.15.178:0/1371117333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e1e16f3f0 con 0x558e1e26e500 2023-10-16T14:36:40.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.819+0000 7fda737fe700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2c8390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda640097f0 tx=0x7fda64005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f1667495379f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:36:40.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.819+0000 7fda56ff5700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:36:40.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.819+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda64005110 con 0x558e1e26e500 2023-10-16T14:36:40.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.819+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda64005270 con 0x558e1e26e500 2023-10-16T14:36:40.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda6801e040 con 0x558e1e26e500 2023-10-16T14:36:40.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda64005430 con 0x558e1e26e500 2023-10-16T14:36:40.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda8cd94b80 10 client.25987 fetch_fsmap learned FSMap version 30 2023-10-16T14:36:40.825 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x558e1e003250 con 0x558e1e26e500 2023-10-16T14:36:40.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda6801e040 con 0x558e1e26e500 2023-10-16T14:36:40.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.822+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fda64003830 con 0x558e1e26e500 2023-10-16T14:36:40.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 10 client.25987 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:36:40.826 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 10 client.25987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:36:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 20 client.25987 populate_metadata read hostname 'smithi178' 2023-10-16T14:36:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2ca7d0 0x558e1e2ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:36:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 4 client.25987 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:36:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558e1e22ec10 con 0x558e1e2ca7d0 2023-10-16T14:36:40.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.823+0000 7fda72ffd700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2ca7d0 0x558e1e2ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:36:40.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.826+0000 7fda72ffd700 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2ca7d0 0x558e1e2ccbf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b045db34b36b039 server_cookie=86e41b58b7f086fd in_seq=0 out_seq=0 2023-10-16T14:36:40.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda56ff5700 10 client.25987 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda56ff5700 1 -- 172.21.15.178:0/1371117333 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x558e1e22ec10 con 0x558e1e2ca7d0 2023-10-16T14:36:40.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda56ff5700 10 client.25987 handle_command_reply: tid=0 2023-10-16T14:36:40.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda8cd94b80 1 client.25987 shutdown 2023-10-16T14:36:40.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 msgr2=0x558e1e2c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.829+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2c8390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda640097f0 tx=0x7fda64005de0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 shutdown_connections 2023-10-16T14:36:40.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558e1e2ca7d0 0x558e1e2ccbf0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda56ff5700 0 client.25987 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:36:40.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e1e26e500 0x558e1e2c8390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558e1e25f910 0x558e1e2c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 --2- 172.21.15.178:0/1371117333 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558e1e260800 0x558e1e2c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.830+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 >> 172.21.15.178:0/1371117333 conn(0x558e1e18cdd0 msgr2=0x558e1e263a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.832+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 shutdown_connections 2023-10-16T14:36:40.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.832+0000 7fda8cd94b80 1 -- 172.21.15.178:0/1371117333 wait complete. 2023-10-16T14:36:40.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.833+0000 7fda8cd94b80 20 client.25987 trim_cache size 0 max 16384 2023-10-16T14:36:40.836 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:36:40.836 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "scrub active (6737 inodes in the stack)", 2023-10-16T14:36:40.837 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:36:40.837 INFO:teuthology.orchestra.run.smithi178.stdout: "71a14aaa-7be0-41f2-81f0-d10cfdf98b60": { 2023-10-16T14:36:40.837 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:36:40.837 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "71a14aaa-7be0-41f2-81f0-d10cfdf98b60", 2023-10-16T14:36:40.837 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:36:40.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:36:40.838 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:36:40.838 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:36:40.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.839+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 msgr2=0x7fda5c0a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.839+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 0x7fda5c0a3b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fda68006010 tx=0x7fda680058e0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.839+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 msgr2=0x7fda7407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:36:40.844 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.839+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 0x7fda7407e3d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fda6c007d60 tx=0x7fda6c00a340 comp rx=0 tx=0).stop 2023-10-16T14:36:40.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.841+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 shutdown_connections 2023-10-16T14:36:40.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda5c0a1690 0x7fda5c0a3b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda74108f00 0x7fda7407ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.845 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda74107370 0x7fda74081960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 --2- 172.21.15.178:0/1064710397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda74108560 0x7fda7407e3d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:36:40.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 >> 172.21.15.178:0/1064710397 conn(0x7fda74076310 msgr2=0x7fda740783f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:36:40.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 shutdown_connections 2023-10-16T14:36:40.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:36:40.842+0000 7fda73fff700 1 -- 172.21.15.178:0/1064710397 wait complete. 2023-10-16T14:36:40.910 INFO:tasks.cephfs.filesystem:scrub status for tag:71a14aaa-7be0-41f2-81f0-d10cfdf98b60 - {'path': '/', 'tag': '71a14aaa-7be0-41f2-81f0-d10cfdf98b60', 'options': 'recursive,force'} 2023-10-16T14:36:40.977 INFO:tasks.workunit.client.0.smithi178.stdout: 105 487061 876 327134 792 161879 736 2023-10-16T14:36:41.287 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:36:41.288 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.288 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:41.288 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:41.288 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.289 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:41.289 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:41 smithi178 ceph-mon[98495]: pgmap v428: 129 pgs: 129 active+clean; 382 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 700 op/s 2023-10-16T14:36:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:36:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:41.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:41.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:41 smithi186 ceph-mon[134892]: pgmap v428: 129 pgs: 129 active+clean; 382 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 700 op/s 2023-10-16T14:36:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:36:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:41.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:41.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:41 smithi187 ceph-mon[135297]: pgmap v428: 129 pgs: 129 active+clean; 382 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 700 op/s 2023-10-16T14:36:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:42.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:36:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:42 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:42.586 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:42.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:42.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:36:42.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.588 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:42 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:42.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:36:42.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:42.676 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:42 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:43.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:43 smithi178 ceph-mon[98495]: pgmap v429: 129 pgs: 129 active+clean; 383 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-10-16T14:36:43.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:43 smithi186 ceph-mon[134892]: pgmap v429: 129 pgs: 129 active+clean; 383 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-10-16T14:36:43.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:43 smithi187 ceph-mon[135297]: pgmap v429: 129 pgs: 129 active+clean; 383 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 513 op/s 2023-10-16T14:36:46.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:45 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:45 smithi178 ceph-mon[98495]: pgmap v430: 129 pgs: 129 active+clean; 387 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 458 op/s 2023-10-16T14:36:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:45 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:45 smithi186 ceph-mon[134892]: pgmap v430: 129 pgs: 129 active+clean; 387 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 458 op/s 2023-10-16T14:36:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:45 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:36:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:45 smithi187 ceph-mon[135297]: pgmap v430: 129 pgs: 129 active+clean; 387 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 458 op/s 2023-10-16T14:36:48.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:47 smithi178 ceph-mon[98495]: pgmap v431: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 414 op/s 2023-10-16T14:36:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:47 smithi186 ceph-mon[134892]: pgmap v431: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 414 op/s 2023-10-16T14:36:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:47 smithi187 ceph-mon[135297]: pgmap v431: 129 pgs: 129 active+clean; 389 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 414 op/s 2023-10-16T14:36:50.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:49 smithi186 ceph-mon[134892]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2023-10-16T14:36:50.081 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:49 smithi178 ceph-mon[98495]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2023-10-16T14:36:50.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:49 smithi187 ceph-mon[135297]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 295 op/s 2023-10-16T14:36:50.979 INFO:tasks.workunit.client.0.smithi178.stdout: 115 644559 515 435231 716 208122 1350 2023-10-16T14:36:51.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:51 smithi178 ceph-mon[98495]: pgmap v433: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-10-16T14:36:51.554 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:51 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:51 smithi186 ceph-mon[134892]: pgmap v433: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-10-16T14:36:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:51 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:51.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:51 smithi187 ceph-mon[135297]: pgmap v433: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 235 op/s 2023-10-16T14:36:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:51 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:52 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:52.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:52 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:36:52.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:52 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:52.587 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:52 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:36:52.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:52 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:52.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:52 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:36:53.560 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:53 smithi178 ceph-mon[98495]: pgmap v434: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2023-10-16T14:36:53.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:53 smithi186 ceph-mon[134892]: pgmap v434: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2023-10-16T14:36:53.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:53 smithi187 ceph-mon[135297]: pgmap v434: 129 pgs: 129 active+clean; 393 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 166 op/s 2023-10-16T14:36:56.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:55 smithi178 ceph-mon[98495]: pgmap v435: 129 pgs: 129 active+clean; 400 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-10-16T14:36:56.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:55 smithi186 ceph-mon[134892]: pgmap v435: 129 pgs: 129 active+clean; 400 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-10-16T14:36:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:55 smithi187 ceph-mon[135297]: pgmap v435: 129 pgs: 129 active+clean; 400 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 181 op/s 2023-10-16T14:36:58.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:57 smithi178 ceph-mon[98495]: pgmap v436: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-10-16T14:36:58.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:57 smithi186 ceph-mon[134892]: pgmap v436: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-10-16T14:36:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:57 smithi187 ceph-mon[135297]: pgmap v436: 129 pgs: 129 active+clean; 404 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2023-10-16T14:37:00.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:36:59 smithi178 ceph-mon[98495]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2023-10-16T14:37:00.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:36:59 smithi186 ceph-mon[134892]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2023-10-16T14:37:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:36:59 smithi187 ceph-mon[135297]: pgmap v437: 129 pgs: 129 active+clean; 405 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 183 op/s 2023-10-16T14:37:00.992 INFO:tasks.workunit.client.0.smithi178.stdout: 132 821136 816 562468 974 298974 2182 2023-10-16T14:37:01.561 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:01 smithi178 ceph-mon[98495]: pgmap v438: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 208 op/s 2023-10-16T14:37:01.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:01 smithi186 ceph-mon[134892]: pgmap v438: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 208 op/s 2023-10-16T14:37:01.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:01 smithi187 ceph-mon[135297]: pgmap v438: 129 pgs: 129 active+clean; 416 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 208 op/s 2023-10-16T14:37:04.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:03 smithi178 ceph-mon[98495]: pgmap v439: 129 pgs: 129 active+clean; 426 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2023-10-16T14:37:04.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:03 smithi186 ceph-mon[134892]: pgmap v439: 129 pgs: 129 active+clean; 426 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2023-10-16T14:37:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:03 smithi187 ceph-mon[135297]: pgmap v439: 129 pgs: 129 active+clean; 426 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 207 op/s 2023-10-16T14:37:06.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:05 smithi178 ceph-mon[98495]: pgmap v440: 129 pgs: 129 active+clean; 435 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-16T14:37:06.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:05 smithi186 ceph-mon[134892]: pgmap v440: 129 pgs: 129 active+clean; 435 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-16T14:37:06.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:05 smithi187 ceph-mon[135297]: pgmap v440: 129 pgs: 129 active+clean; 435 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2023-10-16T14:37:08.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:07 smithi178 ceph-mon[98495]: pgmap v441: 129 pgs: 129 active+clean; 450 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2023-10-16T14:37:08.082 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:07 smithi186 ceph-mon[134892]: pgmap v441: 129 pgs: 129 active+clean; 450 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2023-10-16T14:37:08.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:07 smithi187 ceph-mon[135297]: pgmap v441: 129 pgs: 129 active+clean; 450 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 271 op/s 2023-10-16T14:37:10.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:09 smithi178 ceph-mon[98495]: pgmap v442: 129 pgs: 129 active+clean; 475 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2023-10-16T14:37:10.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:09 smithi186 ceph-mon[134892]: pgmap v442: 129 pgs: 129 active+clean; 475 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2023-10-16T14:37:10.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:09 smithi187 ceph-mon[135297]: pgmap v442: 129 pgs: 129 active+clean; 475 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 350 op/s 2023-10-16T14:37:10.911 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:37:11.009 INFO:tasks.workunit.client.0.smithi178.stdout: 142 604403 655 417642 646 258963 1676 2023-10-16T14:37:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:37:11.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:11 smithi186 ceph-mon[134892]: pgmap v443: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 716 op/s 2023-10-16T14:37:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:37:11.584 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:11 smithi178 ceph-mon[98495]: pgmap v443: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 716 op/s 2023-10-16T14:37:11.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:37:11.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:11 smithi187 ceph-mon[135297]: pgmap v443: 129 pgs: 129 active+clean; 557 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 716 op/s 2023-10-16T14:37:14.065 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:13 smithi178 ceph-mon[98495]: pgmap v444: 129 pgs: 129 active+clean; 577 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-16T14:37:14.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:13 smithi186 ceph-mon[134892]: pgmap v444: 129 pgs: 129 active+clean; 577 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-16T14:37:14.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:13 smithi187 ceph-mon[135297]: pgmap v444: 129 pgs: 129 active+clean; 577 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 14 MiB/s wr, 790 op/s 2023-10-16T14:37:15.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.598+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2617682932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 msgr2=0x7fb0e8109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:15.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.598+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2617682932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 0x7fb0e8109ec0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009f90 tx=0x7fb0e000b330 comp rx=0 tx=0).stop 2023-10-16T14:37:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.603+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2617682932 shutdown_connections 2023-10-16T14:37:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.603+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2617682932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e810cf90 0x7fb0e8117df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.607 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.603+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2617682932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 0x7fb0e8109ec0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.603+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2617682932 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.603+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2617682932 >> 172.21.15.178:0/2617682932 conn(0x7fb0e806da40 msgr2=0x7fb0e806fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:15.608 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.604+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2617682932 shutdown_connections 2023-10-16T14:37:15.609 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.605+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2617682932 wait complete. 2023-10-16T14:37:15.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.610+0000 7fb0ef19d700 1 Processor -- start 2023-10-16T14:37:15.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 -- start start 2023-10-16T14:37:15.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 0x7fb0e8114340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e810cf90 0x7fb0e81b59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8117960 con 0x7fb0e8072cc0 2023-10-16T14:37:15.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8117ad0 con 0x7fb0e810a490 2023-10-16T14:37:15.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.622+0000 7fb0ef19d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8117c90 con 0x7fb0e810cf90 2023-10-16T14:37:15.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50474/0 (socket says 172.21.15.178:50474) 2023-10-16T14:37:15.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 -- 172.21.15.178:0/2086244479 learned_addr learned my addr 172.21.15.178:0/2086244479 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:15.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 -- 172.21.15.178:0/2086244479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e810cf90 msgr2=0x7fb0e81b59a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T14:37:15.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e810cf90 0x7fb0e81b59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 -- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 msgr2=0x7fb0e8114340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:37:15.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 0x7fb0e8114340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.627+0000 7fb0ed99a700 1 -- 172.21.15.178:0/2086244479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009c40 con 0x7fb0e810a490 2023-10-16T14:37:15.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.628+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9c0 tx=0x7fb0dc00ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:15.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.628+0000 7fb0e77fe700 1 -- 172.21.15.178:0/2086244479 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00cb70 con 0x7fb0e810a490 2023-10-16T14:37:15.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.628+0000 7fb0e77fe700 1 -- 172.21.15.178:0/2086244479 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0dc004cf0 con 0x7fb0e810a490 2023-10-16T14:37:15.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.628+0000 7fb0e77fe700 1 -- 172.21.15.178:0/2086244479 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc005580 con 0x7fb0e810a490 2023-10-16T14:37:15.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.628+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e81b5fd0 con 0x7fb0e810a490 2023-10-16T14:37:15.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.629+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e81b6460 con 0x7fb0e810a490 2023-10-16T14:37:15.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.646+0000 7fb0e77fe700 1 -- 172.21.15.178:0/2086244479 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb0dc00ccd0 con 0x7fb0e810a490 2023-10-16T14:37:15.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.656+0000 7fb103dc7b80 1 Processor -- start 2023-10-16T14:37:15.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 -- start start 2023-10-16T14:37:15.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfd6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bf49c70 con 0x56429bfd6800 2023-10-16T14:37:15.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bf2ec60 con 0x56429bfd4fb0 2023-10-16T14:37:15.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.657+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bf2c580 con 0x56429bfd5910 2023-10-16T14:37:15.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.658+0000 7fb0ee99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.658+0000 7fb0ee99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50476/0 (socket says 172.21.15.178:50476) 2023-10-16T14:37:15.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.658+0000 7fb0ee99c700 1 -- 172.21.15.178:0/920886508 learned_addr learned my addr 172.21.15.178:0/920886508 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:15.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.659+0000 7fb0ee99c700 1 -- 172.21.15.178:0/920886508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 msgr2=0x56429bfd6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:15.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.659+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfd6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.659+0000 7fb0ee99c700 1 -- 172.21.15.178:0/920886508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 msgr2=0x56429bfd6c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:37:15.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.659+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfd6c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.659+0000 7fb0ee99c700 1 -- 172.21.15.178:0/920886508 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429bee53f0 con 0x56429bfd4fb0 2023-10-16T14:37:15.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.660+0000 7fb0e77fe700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 0x7fb0d00a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.660+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802c960 tx=0x7fb0d802cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be96ee93d3b98378 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:15.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.660+0000 7fb0e77fe700 1 -- 172.21.15.178:0/2086244479 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb0dc014070 con 0x7fb0e810a490 2023-10-16T14:37:15.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.661+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 0x7fb0d00a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.662+0000 7fb0e4bfe700 1 -- 172.21.15.178:0/920886508 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d802ec20 con 0x56429bfd4fb0 2023-10-16T14:37:15.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.668+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 0x7fb0d00a3c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009550 tx=0x7fb0e0008c20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:15.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.669+0000 7fb0e4bfe700 1 -- 172.21.15.178:0/920886508 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d80044e0 con 0x56429bfd4fb0 2023-10-16T14:37:15.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.669+0000 7fb0e4bfe700 1 -- 172.21.15.178:0/920886508 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8005020 con 0x56429bfd4fb0 2023-10-16T14:37:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.674+0000 7fb103dc7b80 1 -- 172.21.15.178:0/920886508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 msgr2=0x56429bfd53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:15.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.674+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb0d802c960 tx=0x7fb0d802cc70 comp rx=0 tx=0).stop 2023-10-16T14:37:15.730 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.725+0000 7fb103dc7b80 1 -- 172.21.15.178:0/920886508 shutdown_connections 2023-10-16T14:37:15.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.725+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.725+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfd53d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.725+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/920886508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfd6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.726+0000 7fb103dc7b80 1 -- 172.21.15.178:0/920886508 >> 172.21.15.178:0/920886508 conn(0x56429bf02dd0 msgr2=0x56429bfd1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:15.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.789+0000 7fb103dc7b80 1 -- 172.21.15.178:0/920886508 shutdown_connections 2023-10-16T14:37:15.800 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.796+0000 7fb103dc7b80 1 -- 172.21.15.178:0/920886508 wait complete. 2023-10-16T14:37:15.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.797+0000 7fb103dc7b80 1 -- start start 2023-10-16T14:37:15.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.797+0000 7fb103dc7b80 1 Processor -- start 2023-10-16T14:37:15.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.827+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfdef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.827+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfdf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.827+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.828+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bf27e70 con 0x56429bfd6800 2023-10-16T14:37:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.828+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bec4710 con 0x56429bfd4fb0 2023-10-16T14:37:15.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.828+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bec43b0 con 0x56429bfd5910 2023-10-16T14:37:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.828+0000 7fb103dc7b80 10 client.0 _ll_register_callbacks cb 0x56429bd21800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.829+0000 7fb0ee99c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.829+0000 7fb0ee99c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45722/0 (socket says 172.21.15.178:45722) 2023-10-16T14:37:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.829+0000 7fb0ee99c700 1 -- 172.21.15.178:0/581410801 learned_addr learned my addr 172.21.15.178:0/581410801 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:15.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.830+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfdf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.830+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfdef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.834+0000 7fb0ee99c700 1 -- 172.21.15.178:0/581410801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 msgr2=0x56429bfdf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:15.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.834+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfdf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.834+0000 7fb0ee99c700 1 -- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 msgr2=0x56429bfdef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:15.841 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.834+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfdef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.834+0000 7fb0ee99c700 1 -- 172.21.15.178:0/581410801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d802c5c0 con 0x56429bfd6800 2023-10-16T14:37:15.842 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.835+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004940 tx=0x7fb0d8004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e630fc2d9b5b4f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:15.843 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.839+0000 7fb0cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:37:15.853 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.849+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8041070 con 0x56429bfd6800 2023-10-16T14:37:15.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.849+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb0d8004230 con 0x56429bfd6800 2023-10-16T14:37:15.854 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.850+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56429bf2cef0 con 0x56429bfd6800 2023-10-16T14:37:15.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.851+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0d8005b30 con 0x56429bfd6800 2023-10-16T14:37:15.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.851+0000 7fb103dc7b80 10 client.15945 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:15.855 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.851+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56429bd52de0 con 0x56429bfd6800 2023-10-16T14:37:15.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.863+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56429bf2cef0 con 0x56429bfd6800 2023-10-16T14:37:15.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.863+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb0d8038030 con 0x56429bfd6800 2023-10-16T14:37:15.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.870+0000 7fb103dc7b80 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:15.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.870+0000 7fb103dc7b80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:15.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.870+0000 7fb103dc7b80 20 client.15945 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:15.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.870+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429bfeabd0 0x56429bfecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:15.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.872+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429bfeabd0 0x56429bfecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:15.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.872+0000 7fb103dc7b80 4 client.15945 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:15.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.872+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56429bfa4c10 con 0x56429bfeabd0 2023-10-16T14:37:15.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.877+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429bfeabd0 0x56429bfecff0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d99d062e41ba4f server_cookie=8e6da5c3478752a7 in_seq=0 out_seq=0 2023-10-16T14:37:15.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.877+0000 7fb0cdffb700 10 client.15945 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:15.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.900+0000 7fb0cdffb700 1 -- 172.21.15.178:0/581410801 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56429bfa4c10 con 0x56429bfeabd0 2023-10-16T14:37:15.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.900+0000 7fb0cdffb700 10 client.15945 handle_command_reply: tid=0 2023-10-16T14:37:15.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.903+0000 7fb103dc7b80 1 client.15945 shutdown 2023-10-16T14:37:15.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 msgr2=0x56429bfe64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:15.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004940 tx=0x7fb0d8004970 comp rx=0 tx=0).stop 2023-10-16T14:37:15.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 shutdown_connections 2023-10-16T14:37:15.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429bfeabd0 0x56429bfecff0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd6800 0x56429bfe64d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd4fb0 0x56429bfdef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/581410801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd5910 0x56429bfdf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:15.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 >> 172.21.15.178:0/581410801 conn(0x56429bf02dd0 msgr2=0x56429bfcfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:15.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.916+0000 7fb0cdffb700 0 client.15945 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:15.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.926+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 shutdown_connections 2023-10-16T14:37:15.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.942+0000 7fb103dc7b80 1 -- 172.21.15.178:0/581410801 wait complete. 2023-10-16T14:37:15.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:15.963+0000 7fb103dc7b80 20 client.15945 trim_cache size 0 max 16384 2023-10-16T14:37:16.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:15 smithi178 ceph-mon[98495]: pgmap v445: 129 pgs: 129 active+clean; 580 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-16T14:37:16.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:15 smithi186 ceph-mon[134892]: pgmap v445: 129 pgs: 129 active+clean; 580 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-16T14:37:16.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:15 smithi187 ceph-mon[135297]: pgmap v445: 129 pgs: 129 active+clean; 580 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 794 op/s 2023-10-16T14:37:16.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.244+0000 7fb103dc7b80 1 Processor -- start 2023-10-16T14:37:16.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.255+0000 7fb103dc7b80 1 -- start start 2023-10-16T14:37:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.255+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.257+0000 7fb0ee19b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.257+0000 7fb0ee19b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45730/0 (socket says 172.21.15.178:45730) 2023-10-16T14:37:16.264 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.257+0000 7fb0ee19b700 1 -- 172.21.15.178:0/115455282 learned_addr learned my addr 172.21.15.178:0/115455282 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:16.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.257+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.259+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.257+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfd1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.259+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bfde960 con 0x56429bfd4fb0 2023-10-16T14:37:16.265 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.259+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bda8920 con 0x56429bfd5910 2023-10-16T14:37:16.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.259+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bfd5d60 con 0x56429bfd6800 2023-10-16T14:37:16.266 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.262+0000 7fb0ee19b700 1 -- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 msgr2=0x56429bfd1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:16.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfd1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfd1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee19b700 1 -- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 msgr2=0x56429bfe6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:16.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee19b700 1 -- 172.21.15.178:0/115455282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429bd3bc60 con 0x56429bfd4fb0 2023-10-16T14:37:16.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.263+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007a40 tx=0x7fb0e001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e80f77e6fedf911 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:16.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.264+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfd1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:37:16.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.268+0000 7fb0e53ff700 1 -- 172.21.15.178:0/115455282 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e000e040 con 0x56429bfd4fb0 2023-10-16T14:37:16.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.268+0000 7fb0e53ff700 1 -- 172.21.15.178:0/115455282 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e0050040 con 0x56429bfd4fb0 2023-10-16T14:37:16.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.268+0000 7fb0e53ff700 1 -- 172.21.15.178:0/115455282 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0051a50 con 0x56429bfd4fb0 2023-10-16T14:37:16.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.272+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:37:16.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.281+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 msgr2=0x56429bfd1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:16.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.281+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007a40 tx=0x7fb0e001e000 comp rx=0 tx=0).stop 2023-10-16T14:37:16.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.320+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 shutdown_connections 2023-10-16T14:37:16.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.320+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfd1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.320+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfd1540 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.320+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/115455282 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.320+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 >> 172.21.15.178:0/115455282 conn(0x56429bf02dd0 msgr2=0x56429bfdef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:16.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.330+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 shutdown_connections 2023-10-16T14:37:16.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.330+0000 7fb103dc7b80 1 -- 172.21.15.178:0/115455282 wait complete. 2023-10-16T14:37:16.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.331+0000 7fb103dc7b80 1 -- start start 2023-10-16T14:37:16.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.331+0000 7fb103dc7b80 1 Processor -- start 2023-10-16T14:37:16.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfe61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfe4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bfe5230 con 0x56429bfd4fb0 2023-10-16T14:37:16.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bfe53a0 con 0x56429bfd5910 2023-10-16T14:37:16.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.334+0000 7fb103dc7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429bfe5560 con 0x56429bfd6800 2023-10-16T14:37:16.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.335+0000 7fb0ed99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.335+0000 7fb0ed99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50516/0 (socket says 172.21.15.178:50516) 2023-10-16T14:37:16.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.335+0000 7fb0ed99a700 1 -- 172.21.15.178:0/1571655738 learned_addr learned my addr 172.21.15.178:0/1571655738 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:16.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.335+0000 7fb0ee99c700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfe4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.336+0000 7fb103dc7b80 10 client.0 _ll_register_callbacks cb 0x7fb0d802c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:16.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 -- 172.21.15.178:0/1571655738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 msgr2=0x56429bfe4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:16.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfe4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 -- 172.21.15.178:0/1571655738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 msgr2=0x56429bfe61a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:37:16.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfe61a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 -- 172.21.15.178:0/1571655738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e8118850 con 0x56429bfd5910 2023-10-16T14:37:16.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.340+0000 7fb0ed99a700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc096bc0 tx=0x7fb0dc09a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ac4f228c4d9ba6f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:16.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.346+0000 7fb0cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:37:16.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.346+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc09e320 con 0x56429bfd5910 2023-10-16T14:37:16.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.346+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb0dc0a24e0 con 0x56429bfd5910 2023-10-16T14:37:16.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.361+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0d80044e0 con 0x56429bfd5910 2023-10-16T14:37:16.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.373+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0dc020e30 con 0x56429bfd5910 2023-10-16T14:37:16.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.373+0000 7fb103dc7b80 10 client.26012 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:16.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.373+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56429bd79250 con 0x56429bfd5910 2023-10-16T14:37:16.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.376+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0d80044e0 con 0x56429bfd5910 2023-10-16T14:37:16.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.376+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb0dc099640 con 0x56429bfd5910 2023-10-16T14:37:16.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 10 client.26012 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:16.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 10 client.26012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:16.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 20 client.26012 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:16.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429c040160 0x56429c042580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:16.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 4 client.26012 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:37:16.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.377+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56429bf20e50 con 0x56429c040160 2023-10-16T14:37:16.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.378+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429c040160 0x56429c042580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:16.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.383+0000 7fb0ee19b700 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429c040160 0x56429c042580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e369dc00eaf03a server_cookie=3332d71b50c5f844 in_seq=0 out_seq=0 2023-10-16T14:37:16.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.383+0000 7fb0cdffb700 10 client.26012 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:16.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.384+0000 7fb0cdffb700 1 -- 172.21.15.178:0/1571655738 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56429bf20e50 con 0x56429c040160 2023-10-16T14:37:16.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.384+0000 7fb0cdffb700 10 client.26012 handle_command_reply: tid=0 2023-10-16T14:37:16.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.384+0000 7fb103dc7b80 1 client.26012 shutdown 2023-10-16T14:37:16.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 msgr2=0x56429bfe47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:16.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc096bc0 tx=0x7fb0dc09a200 comp rx=0 tx=0).stop 2023-10-16T14:37:16.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 shutdown_connections 2023-10-16T14:37:16.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56429c040160 0x56429c042580 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56429bfd6800 0x56429bfe4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56429bfd4fb0 0x56429bfe61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 --2- 172.21.15.178:0/1571655738 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56429bfd5910 0x56429bfe47b0 secure :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc096bc0 tx=0x7fb0dc09a200 comp rx=0 tx=0).stop 2023-10-16T14:37:16.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.396+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 >> 172.21.15.178:0/1571655738 conn(0x56429bf02dd0 msgr2=0x56429bfd1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:16.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.397+0000 7fb0cdffb700 0 client.26012 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:16.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.400+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 shutdown_connections 2023-10-16T14:37:16.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.407+0000 7fb103dc7b80 1 -- 172.21.15.178:0/1571655738 wait complete. 2023-10-16T14:37:16.450 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:37:16.450 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:37:16.451 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:37:16.451 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:37:16.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.434+0000 7fb103dc7b80 20 client.26012 trim_cache size 0 max 16384 2023-10-16T14:37:16.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.478+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 msgr2=0x7fb0d00a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:16.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.478+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 0x7fb0d00a3c20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009550 tx=0x7fb0e0008c20 comp rx=0 tx=0).stop 2023-10-16T14:37:16.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.478+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 msgr2=0x7fb0e8114880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:16.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.478+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9c0 tx=0x7fb0dc00ed80 comp rx=0 tx=0).stop 2023-10-16T14:37:16.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 shutdown_connections 2023-10-16T14:37:16.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb0d00a1760 0x7fb0d00a3c20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0e810cf90 0x7fb0e81b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0e8072cc0 0x7fb0e8114340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 --2- 172.21.15.178:0/2086244479 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb0e810a490 0x7fb0e8114880 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:16.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.482+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 >> 172.21.15.178:0/2086244479 conn(0x7fb0e806da40 msgr2=0x7fb0e8108740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:16.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.499+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 shutdown_connections 2023-10-16T14:37:16.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:16.505+0000 7fb0ef19d700 1 -- 172.21.15.178:0/2086244479 wait complete. 2023-10-16T14:37:16.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:37:16.714 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:37:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 -- 172.21.15.178:0/425000326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c101ed0 msgr2=0x7ff82c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 --2- 172.21.15.178:0/425000326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c101ed0 0x7ff82c102330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff820009990 tx=0x7ff820009ca0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 -- 172.21.15.178:0/425000326 shutdown_connections 2023-10-16T14:37:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 --2- 172.21.15.178:0/425000326 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c102870 0x7ff82c109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 --2- 172.21.15.178:0/425000326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c100ce0 0x7ff82c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.321+0000 7ff82b59e700 1 --2- 172.21.15.178:0/425000326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c101ed0 0x7ff82c102330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.322+0000 7ff82b59e700 1 -- 172.21.15.178:0/425000326 >> 172.21.15.178:0/425000326 conn(0x7ff82c0fc210 msgr2=0x7ff82c0fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:17.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.322+0000 7ff82b59e700 1 -- 172.21.15.178:0/425000326 shutdown_connections 2023-10-16T14:37:17.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.322+0000 7ff82b59e700 1 -- 172.21.15.178:0/425000326 wait complete. 2023-10-16T14:37:17.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 Processor -- start 2023-10-16T14:37:17.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 -- start start 2023-10-16T14:37:17.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c101ed0 0x7ff82c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c102870 0x7ff82c1a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a0750 con 0x7ff82c102870 2023-10-16T14:37:17.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a0890 con 0x7ff82c101ed0 2023-10-16T14:37:17.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82b59e700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1a0a20 con 0x7ff82c100ce0 2023-10-16T14:37:17.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82a59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82a59c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60442/0 (socket says 172.21.15.178:60442) 2023-10-16T14:37:17.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff82a59c700 1 -- 172.21.15.178:0/4136363450 learned_addr learned my addr 172.21.15.178:0/4136363450 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:17.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.323+0000 7ff829d9b700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c101ed0 0x7ff82c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82ad9d700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c102870 0x7ff82c1a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 -- 172.21.15.178:0/4136363450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c101ed0 msgr2=0x7ff82c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c101ed0 0x7ff82c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 -- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c102870 msgr2=0x7ff82c1a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c102870 0x7ff82c1a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 -- 172.21.15.178:0/4136363450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200095f0 con 0x7ff82c100ce0 2023-10-16T14:37:17.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82a59c700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e970 tx=0x7ff81c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c009790 con 0x7ff82c100ce0 2023-10-16T14:37:17.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c1a0d00 con 0x7ff82c100ce0 2023-10-16T14:37:17.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81c00eec0 con 0x7ff82c100ce0 2023-10-16T14:37:17.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.324+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c003d60 con 0x7ff82c100ce0 2023-10-16T14:37:17.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.325+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c1a11b0 con 0x7ff82c100ce0 2023-10-16T14:37:17.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.326+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c193330 con 0x7ff82c100ce0 2023-10-16T14:37:17.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.328+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff81c003ec0 con 0x7ff82c100ce0 2023-10-16T14:37:17.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.329+0000 7ff81b7fe700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 0x7ff8140a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.329+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff81c014070 con 0x7ff82c100ce0 2023-10-16T14:37:17.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.329+0000 7ff829d9b700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 0x7ff8140a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.330+0000 7ff829d9b700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 0x7ff8140a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b2e0 tx=0x7ff820005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.337+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff81c096540 con 0x7ff82c100ce0 2023-10-16T14:37:17.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.670+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff82c02cf70 con 0x7ff82c100ce0 2023-10-16T14:37:17.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.671+0000 7ff81b7fe700 1 -- 172.21.15.178:0/4136363450 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+11344 (secure 0 0 0) 0x7ff81c099b60 con 0x7ff82c100ce0 2023-10-16T14:37:17.675 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:37:17.675 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:35:50.922947+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:37:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 msgr2=0x7ff8140a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 0x7ff8140a39c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b2e0 tx=0x7ff820005fb0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 msgr2=0x7ff82c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e970 tx=0x7ff81c00ec80 comp rx=0 tx=0).stop 2023-10-16T14:37:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 shutdown_connections 2023-10-16T14:37:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff8140a1500 0x7ff8140a39c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff82c102870 0x7ff82c1a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff82c100ce0 0x7ff82c198bd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 --2- 172.21.15.178:0/4136363450 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff82c101ed0 0x7ff82c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 >> 172.21.15.178:0/4136363450 conn(0x7ff82c0fc210 msgr2=0x7ff82c1077e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:17.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 shutdown_connections 2023-10-16T14:37:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.674+0000 7ff82b59e700 1 -- 172.21.15.178:0/4136363450 wait complete. 2023-10-16T14:37:17.681 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 30 2023-10-16T14:37:17.702 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:37:17.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 -- 172.21.15.178:0/400354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 msgr2=0x7f1020103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 --2- 172.21.15.178:0/400354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020103530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f10100098f0 tx=0x7f1010009c00 comp rx=0 tx=0).stop 2023-10-16T14:37:17.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 -- 172.21.15.178:0/400354187 shutdown_connections 2023-10-16T14:37:17.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 --2- 172.21.15.178:0/400354187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020104ca0 0x7f102010b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 --2- 172.21.15.178:0/400354187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020103530 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 --2- 172.21.15.178:0/400354187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f1020104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 -- 172.21.15.178:0/400354187 >> 172.21.15.178:0/400354187 conn(0x7f10200fe6a0 msgr2=0x7f1020100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:17.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.928+0000 7f1025fff700 1 -- 172.21.15.178:0/400354187 shutdown_connections 2023-10-16T14:37:17.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.929+0000 7f1025fff700 1 -- 172.21.15.178:0/400354187 wait complete. 2023-10-16T14:37:17.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 Processor -- start 2023-10-16T14:37:17.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 -- start start 2023-10-16T14:37:17.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f10201993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020104ca0 0x7f10201a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10201a09f0 con 0x7f1020104ca0 2023-10-16T14:37:17.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10201a0b30 con 0x7f1020104300 2023-10-16T14:37:17.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.930+0000 7f1025fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10201a0cc0 con 0x7f1020103110 2023-10-16T14:37:17.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60462/0 (socket says 172.21.15.178:60462) 2023-10-16T14:37:17.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 -- 172.21.15.178:0/1513246389 learned_addr learned my addr 172.21.15.178:0/1513246389 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:17.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101effd700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f10201993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 -- 172.21.15.178:0/1513246389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 msgr2=0x7f10201993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f10201993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 -- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020104ca0 msgr2=0x7f10201a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020104ca0 0x7f10201a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 -- 172.21.15.178:0/1513246389 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010009520 con 0x7f1020103110 2023-10-16T14:37:17.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101effd700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f10201993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:17.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.931+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1010005e90 tx=0x7f101000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.932+0000 7f101cff9700 1 -- 172.21.15.178:0/1513246389 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001c070 con 0x7f1020103110 2023-10-16T14:37:17.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.932+0000 7f101cff9700 1 -- 172.21.15.178:0/1513246389 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101000f860 con 0x7f1020103110 2023-10-16T14:37:17.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.932+0000 7f101cff9700 1 -- 172.21.15.178:0/1513246389 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010003dc0 con 0x7f1020103110 2023-10-16T14:37:17.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.932+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10201a0f40 con 0x7f1020103110 2023-10-16T14:37:17.941 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.932+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10201a12b0 con 0x7f1020103110 2023-10-16T14:37:17.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.935+0000 7f101cff9700 1 -- 172.21.15.178:0/1513246389 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1010005040 con 0x7f1020103110 2023-10-16T14:37:17.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.937+0000 7f101cff9700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 0x7f10080a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.937+0000 7f103986fb80 1 Processor -- start 2023-10-16T14:37:17.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f101cff9700 1 -- 172.21.15.178:0/1513246389 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f10100cdfc0 con 0x7f1020103110 2023-10-16T14:37:17.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f101effd700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 0x7f10080a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 -- start start 2023-10-16T14:37:17.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b7191240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b7192aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b7105c70 con 0x5566b7190e20 2023-10-16T14:37:17.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b70eac60 con 0x5566b7192620 2023-10-16T14:37:17.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f103986fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b70e8580 con 0x5566b7191780 2023-10-16T14:37:17.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f101effd700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 0x7f10080a39c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1014005950 tx=0x7f101400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.938+0000 7f101effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b7192aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60474/0 (socket says 172.21.15.178:60474) 2023-10-16T14:37:17.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 -- 172.21.15.178:0/3886880826 learned_addr learned my addr 172.21.15.178:0/3886880826 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:17.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101f7fe700 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b7191240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 -- 172.21.15.178:0/3886880826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 msgr2=0x5566b7192aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:17.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b7192aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 -- 172.21.15.178:0/3886880826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 msgr2=0x5566b7191240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:17.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b7191240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.947 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101effd700 1 -- 172.21.15.178:0/3886880826 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b70a13f0 con 0x5566b7191780 2023-10-16T14:37:17.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.939+0000 7f101f7fe700 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b7191240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:37:17.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.942+0000 7f101effd700 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1014011760 tx=0x7f1014013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9d72d3ed470171 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.943+0000 7f1005ffb700 1 -- 172.21.15.178:0/3886880826 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101400bd40 con 0x5566b7191780 2023-10-16T14:37:17.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.943+0000 7f1005ffb700 1 -- 172.21.15.178:0/3886880826 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1014015070 con 0x5566b7191780 2023-10-16T14:37:17.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.943+0000 7f1005ffb700 1 -- 172.21.15.178:0/3886880826 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140269f0 con 0x5566b7191780 2023-10-16T14:37:17.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.946+0000 7f103986fb80 1 -- 172.21.15.178:0/3886880826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 msgr2=0x5566b7192040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.949 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.946+0000 7f103986fb80 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1014011760 tx=0x7f1014013000 comp rx=0 tx=0).stop 2023-10-16T14:37:17.950 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.947+0000 7f103986fb80 1 -- 172.21.15.178:0/3886880826 shutdown_connections 2023-10-16T14:37:17.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.947+0000 7f103986fb80 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b7192aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.947+0000 7f103986fb80 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b7191240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.947+0000 7f103986fb80 1 --2- 172.21.15.178:0/3886880826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7192040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.947+0000 7f103986fb80 1 -- 172.21.15.178:0/3886880826 >> 172.21.15.178:0/3886880826 conn(0x5566b70bedd0 msgr2=0x5566b718bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:17.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.948+0000 7f103986fb80 1 -- 172.21.15.178:0/3886880826 shutdown_connections 2023-10-16T14:37:17.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.948+0000 7f103986fb80 1 -- 172.21.15.178:0/3886880826 wait complete. 2023-10-16T14:37:17.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.949+0000 7f103986fb80 1 -- start start 2023-10-16T14:37:17.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.950+0000 7f103986fb80 1 Processor -- start 2023-10-16T14:37:17.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b719ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b719b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b70e3e70 con 0x5566b7190e20 2023-10-16T14:37:17.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b7080710 con 0x5566b7192620 2023-10-16T14:37:17.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.954+0000 7f103986fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b70803b0 con 0x5566b7191780 2023-10-16T14:37:17.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50566/0 (socket says 172.21.15.178:50566) 2023-10-16T14:37:17.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f101ffff700 1 -- 172.21.15.178:0/1281212194 learned_addr learned my addr 172.21.15.178:0/1281212194 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:17.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b719ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f103986fb80 10 client.0 _ll_register_callbacks cb 0x5566b6edd800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:17.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.955+0000 7f101effd700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b719b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.956+0000 7f101effd700 1 -- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 msgr2=0x5566b71a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:17.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.956+0000 7f101effd700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.956+0000 7f101effd700 1 -- 172.21.15.178:0/1281212194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 msgr2=0x5566b719ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:17.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.956+0000 7f101effd700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b719ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.956+0000 7f101effd700 1 -- 172.21.15.178:0/1281212194 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014011440 con 0x5566b7191780 2023-10-16T14:37:17.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b719ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:17.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f101effd700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b719b280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1014012fd0 tx=0x7f1014010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05e70e80f16c149 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:17.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f0ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:17.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014034070 con 0x5566b7191780 2023-10-16T14:37:17.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1014017070 con 0x5566b7191780 2023-10-16T14:37:17.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.957+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566b70e8ef0 con 0x5566b7191780 2023-10-16T14:37:17.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.958+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101402a040 con 0x5566b7191780 2023-10-16T14:37:17.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.959+0000 7f103986fb80 10 client.26011 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:17.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.959+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5566b6f0ede0 con 0x5566b7191780 2023-10-16T14:37:17.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.959+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566b70e8ef0 con 0x5566b7191780 2023-10-16T14:37:17.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.959+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1014037020 con 0x5566b7191780 2023-10-16T14:37:17.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 10 client.26011 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:17.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 10 client.26011 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:37:17.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 20 client.26011 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:17.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b71a6960 0x5566b71a8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 4 client.26011 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:17.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566b7160c10 con 0x5566b71a6960 2023-10-16T14:37:17.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.961+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b71a6960 0x5566b71a8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:17.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.962+0000 7f0ffeffd700 10 client.26011 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:37:17.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.962+0000 7f101f7fe700 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b71a6960 0x5566b71a8da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ec73038f341cec2 server_cookie=b2d56cb968571ec0 in_seq=0 out_seq=0 2023-10-16T14:37:17.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.963+0000 7f0ffeffd700 1 -- 172.21.15.178:0/1281212194 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5566b7160c10 con 0x5566b71a6960 2023-10-16T14:37:17.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.963+0000 7f0ffeffd700 10 client.26011 handle_command_reply: tid=0 2023-10-16T14:37:17.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.964+0000 7f103986fb80 1 client.26011 shutdown 2023-10-16T14:37:17.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.965+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 msgr2=0x5566b719b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:17.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.965+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b719b280 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1014012fd0 tx=0x7f1014010f10 comp rx=0 tx=0).stop 2023-10-16T14:37:17.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 shutdown_connections 2023-10-16T14:37:17.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b71a6960 0x5566b71a8da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b719ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 --2- 172.21.15.178:0/1281212194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b719b280 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f0ffeffd700 0 client.26011 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.966+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 >> 172.21.15.178:0/1281212194 conn(0x5566b70bedd0 msgr2=0x5566b718bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.967+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 shutdown_connections 2023-10-16T14:37:17.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.967+0000 7f103986fb80 1 -- 172.21.15.178:0/1281212194 wait complete. 2023-10-16T14:37:17.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.967+0000 7f103986fb80 20 client.26011 trim_cache size 0 max 16384 2023-10-16T14:37:17.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.993+0000 7f103986fb80 1 Processor -- start 2023-10-16T14:37:17.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.993+0000 7f103986fb80 1 -- start start 2023-10-16T14:37:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.994+0000 7f103986fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.994+0000 7f103986fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7195b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.994+0000 7f103986fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f103986fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b6f64920 con 0x5566b7190e20 2023-10-16T14:37:17.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f103986fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b719bc40 con 0x5566b7192620 2023-10-16T14:37:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f103986fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b719a8e0 con 0x5566b7191780 2023-10-16T14:37:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50574/0 (socket says 172.21.15.178:50574) 2023-10-16T14:37:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.995+0000 7f101ffff700 1 -- 172.21.15.178:0/769525351 learned_addr learned my addr 172.21.15.178:0/769525351 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.996+0000 7f101ffff700 1 -- 172.21.15.178:0/769525351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 msgr2=0x5566b7195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.996+0000 7f101ffff700 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.996+0000 7f101ffff700 1 -- 172.21.15.178:0/769525351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 msgr2=0x5566b71955c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:37:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.997+0000 7f101f7fe700 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.998+0000 7f101ffff700 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.998+0000 7f101ffff700 1 -- 172.21.15.178:0/769525351 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566b6ef7c60 con 0x5566b7192620 2023-10-16T14:37:18.003 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.998+0000 7f101f7fe700 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:37:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.998+0000 7f101ffff700 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1018009530 tx=0x7f1018009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc40f989aa88dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.999+0000 7f10067fc700 1 -- 172.21.15.178:0/769525351 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101800e040 con 0x5566b7192620 2023-10-16T14:37:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.999+0000 7f10067fc700 1 -- 172.21.15.178:0/769525351 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101800f040 con 0x5566b7192620 2023-10-16T14:37:18.004 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:17.999+0000 7f10067fc700 1 -- 172.21.15.178:0/769525351 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018016430 con 0x5566b7192620 2023-10-16T14:37:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 -- 172.21.15.178:0/769525351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 msgr2=0x5566b71a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1018009530 tx=0x7f1018009560 comp rx=0 tx=0).stop 2023-10-16T14:37:18.005 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 -- 172.21.15.178:0/769525351 shutdown_connections 2023-10-16T14:37:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a2280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 --2- 172.21.15.178:0/769525351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b7195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.000+0000 7f103986fb80 1 -- 172.21.15.178:0/769525351 >> 172.21.15.178:0/769525351 conn(0x5566b70bedd0 msgr2=0x5566b7199950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.001+0000 7f103986fb80 1 -- 172.21.15.178:0/769525351 shutdown_connections 2023-10-16T14:37:18.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.001+0000 7f103986fb80 1 -- 172.21.15.178:0/769525351 wait complete. 2023-10-16T14:37:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.002+0000 7f103986fb80 1 -- start start 2023-10-16T14:37:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.002+0000 7f103986fb80 1 Processor -- start 2023-10-16T14:37:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71a0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b71a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b71a1ce0 con 0x5566b7190e20 2023-10-16T14:37:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b71a1e50 con 0x5566b7192620 2023-10-16T14:37:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.003+0000 7f103986fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566b71a1fc0 con 0x5566b7191780 2023-10-16T14:37:18.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.004+0000 7f103986fb80 10 client.0 _ll_register_callbacks cb 0x7f1014011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.004+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.004+0000 7f101ffff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50582/0 (socket says 172.21.15.178:50582) 2023-10-16T14:37:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.004+0000 7f101ffff700 1 -- 172.21.15.178:0/2102715355 learned_addr learned my addr 172.21.15.178:0/2102715355 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.004+0000 7f101effd700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b71a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101f7fe700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101effd700 1 -- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 msgr2=0x5566b71a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101effd700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101effd700 1 -- 172.21.15.178:0/2102715355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 msgr2=0x5566b71a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.012 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101effd700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101effd700 1 -- 172.21.15.178:0/2102715355 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10180091b0 con 0x5566b7191780 2023-10-16T14:37:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.005+0000 7f101f7fe700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:37:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.006+0000 7f101effd700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b71a0790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1014011dd0 tx=0x7f1014010f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=751fdc6ca902fd3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.006+0000 7f0ffeffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:18.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.006+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014034030 con 0x5566b7191780 2023-10-16T14:37:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.006+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1014035070 con 0x5566b7191780 2023-10-16T14:37:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.007+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1014015070 con 0x5566b7191780 2023-10-16T14:37:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.007+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101402a040 con 0x5566b7191780 2023-10-16T14:37:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.007+0000 7f103986fb80 10 client.26014 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:18.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.007+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5566b6f35250 con 0x5566b7191780 2023-10-16T14:37:18.015 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.007+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1014015070 con 0x5566b7191780 2023-10-16T14:37:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.008+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1014011a70 con 0x5566b7191780 2023-10-16T14:37:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.009+0000 7f103986fb80 10 client.26014 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.009+0000 7f103986fb80 10 client.26014 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:37:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.010+0000 7f103986fb80 20 client.26014 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:18.016 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.010+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b7214d40 0x5566b7217160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.010+0000 7f103986fb80 4 client.26014 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:37:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.010+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566b70dce50 con 0x5566b7214d40 2023-10-16T14:37:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.010+0000 7f101f7fe700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b7214d40 0x5566b7217160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.011+0000 7f0ffeffd700 10 client.26014 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:37:18.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.011+0000 7f101f7fe700 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b7214d40 0x5566b7217160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=980d5d42ca98a5e1 server_cookie=e8faecfbbfd0711 in_seq=0 out_seq=0 2023-10-16T14:37:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.013+0000 7f0ffeffd700 1 -- 172.21.15.178:0/2102715355 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566b70dce50 con 0x5566b7214d40 2023-10-16T14:37:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.013+0000 7f0ffeffd700 10 client.26014 handle_command_reply: tid=0 2023-10-16T14:37:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.013+0000 7f103986fb80 1 client.26014 shutdown 2023-10-16T14:37:18.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 msgr2=0x5566b71a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b71a0790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1014011dd0 tx=0x7f1014010f10 comp rx=0 tx=0).stop 2023-10-16T14:37:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 shutdown_connections 2023-10-16T14:37:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5566b7214d40 0x5566b7217160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5566b7192620 0x5566b71a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5566b7190e20 0x5566b71a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 --2- 172.21.15.178:0/2102715355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566b7191780 0x5566b71a0790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 >> 172.21.15.178:0/2102715355 conn(0x5566b70bedd0 msgr2=0x5566b71955c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.014+0000 7f0ffeffd700 0 client.26014 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:37:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.015+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 shutdown_connections 2023-10-16T14:37:18.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.015+0000 7f103986fb80 1 -- 172.21.15.178:0/2102715355 wait complete. 2023-10-16T14:37:18.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.015+0000 7f103986fb80 20 client.26014 trim_cache size 0 max 16384 2023-10-16T14:37:18.022 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:37:18.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.016+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 msgr2=0x7f10080a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.016+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 0x7f10080a39c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1014005950 tx=0x7f101400b410 comp rx=0 tx=0).stop 2023-10-16T14:37:18.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.016+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 msgr2=0x7f1020198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.016+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1010005e90 tx=0x7f101000b790 comp rx=0 tx=0).stop 2023-10-16T14:37:18.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 shutdown_connections 2023-10-16T14:37:18.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f10080a1500 0x7f10080a39c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1020104ca0 0x7f10201a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1020103110 0x7f1020198e70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 --2- 172.21.15.178:0/1513246389 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1020104300 0x7f10201993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.018+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 >> 172.21.15.178:0/1513246389 conn(0x7f10200fe6a0 msgr2=0x7f1020107850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.019+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 shutdown_connections 2023-10-16T14:37:18.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.019+0000 7f1025fff700 1 -- 172.21.15.178:0/1513246389 wait complete. 2023-10-16T14:37:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:17 smithi178 ceph-mon[98495]: pgmap v446: 129 pgs: 129 active+clean; 584 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-10-16T14:37:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:17 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4136363450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:37:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:37:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:37:18.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:17 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:37:18.058 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:37:18.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:17 smithi186 ceph-mon[134892]: pgmap v446: 129 pgs: 129 active+clean; 584 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-10-16T14:37:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:17 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4136363450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:37:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:37:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:37:18.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:17 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:37:18.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:17 smithi187 ceph-mon[135297]: pgmap v446: 129 pgs: 129 active+clean; 584 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 813 op/s 2023-10-16T14:37:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:17 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4136363450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:37:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:37:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:37:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:17 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:37:18.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 -- 172.21.15.178:0/3498163518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824100e80 msgr2=0x7f28241012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 --2- 172.21.15.178:0/3498163518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824100e80 0x7f28241012a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28140098f0 tx=0x7f2814009c00 comp rx=0 tx=0).stop 2023-10-16T14:37:18.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 -- 172.21.15.178:0/3498163518 shutdown_connections 2023-10-16T14:37:18.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 --2- 172.21.15.178:0/3498163518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102a20 0x7f2824109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 --2- 172.21.15.178:0/3498163518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824100e80 0x7f28241012a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 --2- 172.21.15.178:0/3498163518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824102060 0x7f28241024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 -- 172.21.15.178:0/3498163518 >> 172.21.15.178:0/3498163518 conn(0x7f28240fc400 msgr2=0x7f28240fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 -- 172.21.15.178:0/3498163518 shutdown_connections 2023-10-16T14:37:18.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.279+0000 7f2829bff700 1 -- 172.21.15.178:0/3498163518 wait complete. 2023-10-16T14:37:18.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.280+0000 7f2829bff700 1 Processor -- start 2023-10-16T14:37:18.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.280+0000 7f2829bff700 1 -- start start 2023-10-16T14:37:18.287 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824100e80 0x7f2824198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824102a20 0x7f28241a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28241a09b0 con 0x7f2824100e80 2023-10-16T14:37:18.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28241a0af0 con 0x7f2824102060 2023-10-16T14:37:18.288 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2829bff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28241a0c80 con 0x7f2824102a20 2023-10-16T14:37:18.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50596/0 (socket says 172.21.15.178:50596) 2023-10-16T14:37:18.289 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 -- 172.21.15.178:0/90273886 learned_addr learned my addr 172.21.15.178:0/90273886 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2823fff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824102a20 0x7f28241a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f28237fe700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824100e80 0x7f2824198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 -- 172.21.15.178:0/90273886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824102a20 msgr2=0x7f28241a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824102a20 0x7f28241a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 -- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824100e80 msgr2=0x7f2824198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824100e80 0x7f2824198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 -- 172.21.15.178:0/90273886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2814009520 con 0x7f2824102060 2023-10-16T14:37:18.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.281+0000 7f2822ffd700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f281800e970 tx=0x7f281800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.282+0000 7f2820ff9700 1 -- 172.21.15.178:0/90273886 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818009790 con 0x7f2824102060 2023-10-16T14:37:18.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.282+0000 7f2820ff9700 1 -- 172.21.15.178:0/90273886 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281800c7e0 con 0x7f2824102060 2023-10-16T14:37:18.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.282+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28241a0f60 con 0x7f2824102060 2023-10-16T14:37:18.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.282+0000 7f2820ff9700 1 -- 172.21.15.178:0/90273886 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818003c70 con 0x7f2824102060 2023-10-16T14:37:18.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.282+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28241a1430 con 0x7f2824102060 2023-10-16T14:37:18.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.286+0000 7f2820ff9700 1 -- 172.21.15.178:0/90273886 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f281800cc50 con 0x7f2824102060 2023-10-16T14:37:18.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.287+0000 7f283d5f6b80 1 Processor -- start 2023-10-16T14:37:18.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 -- start start 2023-10-16T14:37:18.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 0x563750d61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d62040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d62aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750cd5c70 con 0x563750d61780 2023-10-16T14:37:18.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750cbac60 con 0x563750d60e20 2023-10-16T14:37:18.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750cb8580 con 0x563750d62620 2023-10-16T14:37:18.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f2823fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 0x563750d61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f2823fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 0x563750d61240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50606/0 (socket says 172.21.15.178:50606) 2023-10-16T14:37:18.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f2823fff700 1 -- 172.21.15.178:0/3576635574 learned_addr learned my addr 172.21.15.178:0/3576635574 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f28237fe700 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d62040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.288+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d62aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2820ff9700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 -- 172.21.15.178:0/3576635574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 msgr2=0x563750d61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 0x563750d61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 -- 172.21.15.178:0/3576635574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 msgr2=0x563750d62040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d62040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2820ff9700 1 -- 172.21.15.178:0/90273886 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2818014070 con 0x7f2824102060 2023-10-16T14:37:18.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 -- 172.21.15.178:0/3576635574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563750c713f0 con 0x563750d62620 2023-10-16T14:37:18.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2823fff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d62aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28180ca900 tx=0x7f2818003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b909c55d7cd8060 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.300 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.289+0000 7f2809ffb700 1 -- 172.21.15.178:0/3576635574 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281809a850 con 0x563750d62620 2023-10-16T14:37:18.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.290+0000 7f2809ffb700 1 -- 172.21.15.178:0/3576635574 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281809a150 con 0x563750d62620 2023-10-16T14:37:18.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.290+0000 7f2809ffb700 1 -- 172.21.15.178:0/3576635574 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180cf0c0 con 0x563750d62620 2023-10-16T14:37:18.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.290+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3576635574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 msgr2=0x563750d62aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.301 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.290+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d62aa0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f28180ca900 tx=0x7f2818003680 comp rx=0 tx=0).stop 2023-10-16T14:37:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.290+0000 7f2823fff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 0x7f280c0a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f281c006220 tx=0x7f281c009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3576635574 shutdown_connections 2023-10-16T14:37:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d62aa0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d60e20 0x563750d61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3576635574 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d62040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3576635574 >> 172.21.15.178:0/3576635574 conn(0x563750c8edd0 msgr2=0x563750d5bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3576635574 shutdown_connections 2023-10-16T14:37:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3576635574 wait complete. 2023-10-16T14:37:18.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.291+0000 7f283d5f6b80 1 -- start start 2023-10-16T14:37:18.304 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 Processor -- start 2023-10-16T14:37:18.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d6abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d72170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.308 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750cb3e70 con 0x563750d60e20 2023-10-16T14:37:18.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750c50710 con 0x563750d62620 2023-10-16T14:37:18.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750c503b0 con 0x563750d61780 2023-10-16T14:37:18.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f283d5f6b80 10 client.0 _ll_register_callbacks cb 0x563750aad800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:18.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f2823fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d72170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f28237fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d6abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60524/0 (socket says 172.21.15.178:60524) 2023-10-16T14:37:18.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.292+0000 7f2822ffd700 1 -- 172.21.15.178:0/2527164022 learned_addr learned my addr 172.21.15.178:0/2527164022 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 -- 172.21.15.178:0/2527164022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 msgr2=0x563750d72170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d72170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 -- 172.21.15.178:0/2527164022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 msgr2=0x563750d6abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d6abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 -- 172.21.15.178:0/2527164022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28241a1430 con 0x563750d61780 2023-10-16T14:37:18.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2823fff700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d72170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:18.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2822ffd700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2818096bc0 tx=0x7f281809a850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0ab0d4bd9432280 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:18.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180cd030 con 0x563750d61780 2023-10-16T14:37:18.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563750cb8ef0 con 0x563750d61780 2023-10-16T14:37:18.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f281809e2a0 con 0x563750d61780 2023-10-16T14:37:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f281809a3b0 con 0x563750d61780 2023-10-16T14:37:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.293+0000 7f283d5f6b80 10 client.26026 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563750adede0 con 0x563750d61780 2023-10-16T14:37:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563750cb8ef0 con 0x563750d61780 2023-10-16T14:37:18.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f28180a2050 con 0x563750d61780 2023-10-16T14:37:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 10 client.26026 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 10 client.26026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:37:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 20 client.26026 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750d76800 0x563750d78c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 4 client.26026 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563750d30c10 con 0x563750d76800 2023-10-16T14:37:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.294+0000 7f28237fe700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750d76800 0x563750d78c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.295+0000 7f2802ffd700 10 client.26026 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:37:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.295+0000 7f28237fe700 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750d76800 0x563750d78c20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6329da1f14f3517 server_cookie=7c8c7293df3ca2c8 in_seq=0 out_seq=0 2023-10-16T14:37:18.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.297+0000 7f2802ffd700 1 -- 172.21.15.178:0/2527164022 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563750d30c10 con 0x563750d76800 2023-10-16T14:37:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.297+0000 7f2802ffd700 10 client.26026 handle_command_reply: tid=0 2023-10-16T14:37:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.297+0000 7f283d5f6b80 1 client.26026 shutdown 2023-10-16T14:37:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 msgr2=0x563750d6b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2818096bc0 tx=0x7f281809a850 comp rx=0 tx=0).stop 2023-10-16T14:37:18.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 shutdown_connections 2023-10-16T14:37:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750d76800 0x563750d78c20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d72170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d6abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.317 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f2802ffd700 0 client.26026 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:37:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/2527164022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d61780 0x563750d6b130 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 >> 172.21.15.178:0/2527164022 conn(0x563750c8edd0 msgr2=0x563750d5bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 shutdown_connections 2023-10-16T14:37:18.318 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 1 -- 172.21.15.178:0/2527164022 wait complete. 2023-10-16T14:37:18.319 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.298+0000 7f283d5f6b80 20 client.26026 trim_cache size 0 max 16384 2023-10-16T14:37:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 Processor -- start 2023-10-16T14:37:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 -- start start 2023-10-16T14:37:18.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750b34920 con 0x563750d60e20 2023-10-16T14:37:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750d624d0 con 0x563750d61780 2023-10-16T14:37:18.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750d6a0a0 con 0x563750d62620 2023-10-16T14:37:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f28237fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:50618/0 (socket says 172.21.15.178:50618) 2023-10-16T14:37:18.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f28237fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45844/0 (socket says 172.21.15.178:45844) 2023-10-16T14:37:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f2822ffd700 1 -- 172.21.15.178:0/3375404909 learned_addr learned my addr 172.21.15.178:0/3375404909 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.313+0000 7f2823fff700 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 -- 172.21.15.178:0/3375404909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 msgr2=0x563750d5bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d5bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 -- 172.21.15.178:0/3375404909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 msgr2=0x563750d62a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 -- 172.21.15.178:0/3375404909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563750ac7c60 con 0x563750d61780 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f28237fe700 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f2822ffd700 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f281809a0e0 tx=0x7f2818003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba4c5e9bee7ff961 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.327 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f280a7fc700 1 -- 172.21.15.178:0/3375404909 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180cd040 con 0x563750d61780 2023-10-16T14:37:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f280a7fc700 1 -- 172.21.15.178:0/3375404909 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281809e2a0 con 0x563750d61780 2023-10-16T14:37:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.314+0000 7f280a7fc700 1 -- 172.21.15.178:0/3375404909 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818099640 con 0x563750d61780 2023-10-16T14:37:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3375404909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 msgr2=0x563750d72130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f281809a0e0 tx=0x7f2818003680 comp rx=0 tx=0).stop 2023-10-16T14:37:18.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3375404909 shutdown_connections 2023-10-16T14:37:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d62620 0x563750d5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d60e20 0x563750d62a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/3375404909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d61780 0x563750d72130 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3375404909 >> 172.21.15.178:0/3375404909 conn(0x563750c8edd0 msgr2=0x563750d698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3375404909 shutdown_connections 2023-10-16T14:37:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- 172.21.15.178:0/3375404909 wait complete. 2023-10-16T14:37:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.315+0000 7f283d5f6b80 1 -- start start 2023-10-16T14:37:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 Processor -- start 2023-10-16T14:37:18.330 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d6e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d71260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750d717a0 con 0x563750d61780 2023-10-16T14:37:18.331 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750d71910 con 0x563750d62620 2023-10-16T14:37:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563750d71ad0 con 0x563750d60e20 2023-10-16T14:37:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f283d5f6b80 10 client.0 _ll_register_callbacks cb 0x7f2818005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.316+0000 7f28237fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60542/0 (socket says 172.21.15.178:60542) 2023-10-16T14:37:18.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 -- 172.21.15.178:0/1969463124 learned_addr learned my addr 172.21.15.178:0/1969463124 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f2823fff700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d71260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f2822ffd700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d6e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 -- 172.21.15.178:0/1969463124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 msgr2=0x563750d71260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d71260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 -- 172.21.15.178:0/1969463124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 msgr2=0x563750d6e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d6e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 -- 172.21.15.178:0/1969463124 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2824109c70 con 0x563750d60e20 2023-10-16T14:37:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f28237fe700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28140098c0 tx=0x7f28140271a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e6cb4910be79bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f2802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.317+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814027350 con 0x563750d60e20 2023-10-16T14:37:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f281809a150 con 0x563750d60e20 2023-10-16T14:37:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f28140274b0 con 0x563750d60e20 2023-10-16T14:37:18.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2814027a80 con 0x563750d60e20 2023-10-16T14:37:18.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 10 client.26032 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:18.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x563750adede0 con 0x563750d60e20 2023-10-16T14:37:18.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f281809a150 con 0x563750d60e20 2023-10-16T14:37:18.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f2814009920 con 0x563750d60e20 2023-10-16T14:37:18.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 10 client.26032 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 10 client.26032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:37:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 20 client.26032 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750dcb430 0x563750dcd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 4 client.26032 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:37:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.318+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563750cace50 con 0x563750dcb430 2023-10-16T14:37:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.319+0000 7f2822ffd700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750dcb430 0x563750dcd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.319+0000 7f2822ffd700 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750dcb430 0x563750dcd850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4c8dbad8a7d15e0 server_cookie=67bbce7c96f51823 in_seq=0 out_seq=0 2023-10-16T14:37:18.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.319+0000 7f2802ffd700 10 client.26032 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:37:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f2802ffd700 1 -- 172.21.15.178:0/1969463124 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563750cace50 con 0x563750dcb430 2023-10-16T14:37:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f2802ffd700 10 client.26032 handle_command_reply: tid=0 2023-10-16T14:37:18.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 client.26032 shutdown 2023-10-16T14:37:18.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 msgr2=0x563750d6e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28140098c0 tx=0x7f28140271a0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 shutdown_connections 2023-10-16T14:37:18.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x563750dcb430 0x563750dcd850 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563750d62620 0x563750d71260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563750d60e20 0x563750d6e360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 --2- 172.21.15.178:0/1969463124 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563750d61780 0x563750d6e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 >> 172.21.15.178:0/1969463124 conn(0x563750c8edd0 msgr2=0x563750d6ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f2802ffd700 0 client.26032 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:37:18.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 shutdown_connections 2023-10-16T14:37:18.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.322+0000 7f283d5f6b80 1 -- 172.21.15.178:0/1969463124 wait complete. 2023-10-16T14:37:18.343 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f283d5f6b80 20 client.26032 trim_cache size 0 max 16384 2023-10-16T14:37:18.344 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:37:18.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 msgr2=0x7f280c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 0x7f280c0a39c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f281c006220 tx=0x7f281c009040 comp rx=0 tx=0).stop 2023-10-16T14:37:18.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 msgr2=0x7f2824199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f281800e970 tx=0x7f281800ec80 comp rx=0 tx=0).stop 2023-10-16T14:37:18.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 shutdown_connections 2023-10-16T14:37:18.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f280c0a1500 0x7f280c0a39c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2824102a20 0x7f28241a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2824100e80 0x7f2824198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 --2- 172.21.15.178:0/90273886 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f2824102060 0x7f2824199370 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 >> 172.21.15.178:0/90273886 conn(0x7f28240fc400 msgr2=0x7f28240fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 shutdown_connections 2023-10-16T14:37:18.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.323+0000 7f2829bff700 1 -- 172.21.15.178:0/90273886 wait complete. 2023-10-16T14:37:18.348 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:37:18.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 -- 172.21.15.178:0/3334346520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c100e80 msgr2=0x7f1f6c1012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3334346520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c100e80 0x7f1f6c1012a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009990 tx=0x7f1f5c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 -- 172.21.15.178:0/3334346520 shutdown_connections 2023-10-16T14:37:18.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3334346520 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 0x7f1f6c109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3334346520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c100e80 0x7f1f6c1012a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3334346520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c102060 0x7f1f6c1024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.562+0000 7f1f729ff700 1 -- 172.21.15.178:0/3334346520 >> 172.21.15.178:0/3334346520 conn(0x7f1f6c0fc400 msgr2=0x7f1f6c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.563+0000 7f1f729ff700 1 -- 172.21.15.178:0/3334346520 shutdown_connections 2023-10-16T14:37:18.568 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.563+0000 7f1f729ff700 1 -- 172.21.15.178:0/3334346520 wait complete. 2023-10-16T14:37:18.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.563+0000 7f1f729ff700 1 Processor -- start 2023-10-16T14:37:18.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 -- start start 2023-10-16T14:37:18.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 0x7f1f6c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 0x7f1f6c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1a09a0 con 0x7f1f6c100e80 2023-10-16T14:37:18.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1a0ae0 con 0x7f1f6c102a20 2023-10-16T14:37:18.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f729ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c1a0c70 con 0x7f1f6c102060 2023-10-16T14:37:18.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 0x7f1f6c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 0x7f1f6c198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45868/0 (socket says 172.21.15.178:45868) 2023-10-16T14:37:18.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f6bfff700 1 -- 172.21.15.178:0/3241938436 learned_addr learned my addr 172.21.15.178:0/3241938436 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60554/0 (socket says 172.21.15.178:60554) 2023-10-16T14:37:18.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.564+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 0x7f1f6c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.574 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3241938436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 msgr2=0x7f1f6c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 0x7f1f6c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 msgr2=0x7f1f6c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 0x7f1f6c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.575 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3241938436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0095f0 con 0x7f1f6c102060 2023-10-16T14:37:18.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e9c0 tx=0x7f1f6000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f697fa700 1 -- 172.21.15.178:0/3241938436 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60004220 con 0x7f1f6c102060 2023-10-16T14:37:18.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f697fa700 1 -- 172.21.15.178:0/3241938436 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f60004380 con 0x7f1f6c102060 2023-10-16T14:37:18.576 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c1a0f50 con 0x7f1f6c102060 2023-10-16T14:37:18.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f697fa700 1 -- 172.21.15.178:0/3241938436 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60003be0 con 0x7f1f6c102060 2023-10-16T14:37:18.577 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.565+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1a1420 con 0x7f1f6c102060 2023-10-16T14:37:18.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.568+0000 7f1f697fa700 1 -- 172.21.15.178:0/3241938436 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f60003d40 con 0x7f1f6c102060 2023-10-16T14:37:18.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.569+0000 7f1f86353b80 1 Processor -- start 2023-10-16T14:37:18.584 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 -- start start 2023-10-16T14:37:18.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f697fa700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 0x7f1f540a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f697fa700 1 -- 172.21.15.178:0/3241938436 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f60014070 con 0x7f1f6c102060 2023-10-16T14:37:18.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.585 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c371620 0x55b44c371aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c2e4c70 con 0x55b44c370780 2023-10-16T14:37:18.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c2c9c60 con 0x55b44c36fe20 2023-10-16T14:37:18.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f86353b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c2c7580 con 0x55b44c371620 2023-10-16T14:37:18.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 0x7f1f540a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.586 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45874/0 (socket says 172.21.15.178:45874) 2023-10-16T14:37:18.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.570+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1310457198 learned_addr learned my addr 172.21.15.178:0/1310457198 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c371620 0x55b44c371aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.587 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1310457198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c371620 msgr2=0x55b44c371aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c371620 0x55b44c371aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1310457198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 msgr2=0x55b44c370240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1310457198 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b44c2803f0 con 0x55b44c370780 2023-10-16T14:37:18.588 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f6009a1a0 tx=0x7f1f60099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce770176944253f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 0x7f1f540a39c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009ec0 tx=0x7f1f640134a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f527fc700 1 -- 172.21.15.178:0/1310457198 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6009e170 con 0x55b44c370780 2023-10-16T14:37:18.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f527fc700 1 -- 172.21.15.178:0/1310457198 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f6009a210 con 0x55b44c370780 2023-10-16T14:37:18.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.571+0000 7f1f527fc700 1 -- 172.21.15.178:0/1310457198 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60099700 con 0x55b44c370780 2023-10-16T14:37:18.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 -- 172.21.15.178:0/1310457198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 msgr2=0x55b44c371040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f6009a1a0 tx=0x7f1f60099bf0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 -- 172.21.15.178:0/1310457198 shutdown_connections 2023-10-16T14:37:18.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c371620 0x55b44c371aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1310457198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c370780 0x55b44c371040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 -- 172.21.15.178:0/1310457198 >> 172.21.15.178:0/1310457198 conn(0x55b44c29ddd0 msgr2=0x55b44c36afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 -- 172.21.15.178:0/1310457198 shutdown_connections 2023-10-16T14:37:18.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.572+0000 7f1f86353b80 1 -- 172.21.15.178:0/1310457198 wait complete. 2023-10-16T14:37:18.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 -- start start 2023-10-16T14:37:18.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 Processor -- start 2023-10-16T14:37:18.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c379cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c381240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c2c2e70 con 0x55b44c36fe20 2023-10-16T14:37:18.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c25f710 con 0x55b44c371620 2023-10-16T14:37:18.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c25f3b0 con 0x55b44c370780 2023-10-16T14:37:18.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.573+0000 7f1f86353b80 10 client.0 _ll_register_callbacks cb 0x55b44c0bc800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:18.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c379cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c379cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45886/0 (socket says 172.21.15.178:45886) 2023-10-16T14:37:18.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6bfff700 1 -- 172.21.15.178:0/1587239401 learned_addr learned my addr 172.21.15.178:0/1587239401 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c381240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 msgr2=0x55b44c381240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c381240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1587239401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 msgr2=0x55b44c379cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c379cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/1587239401 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c1a1420 con 0x55b44c370780 2023-10-16T14:37:18.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37a200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f6009a1a0 tx=0x7f1f60096680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aba2dcfbf447442 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:18.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b1070 con 0x55b44c370780 2023-10-16T14:37:18.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.574+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f6009e360 con 0x55b44c370780 2023-10-16T14:37:18.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b44c2c7ef0 con 0x55b44c370780 2023-10-16T14:37:18.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f60099910 con 0x55b44c370780 2023-10-16T14:37:18.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f86353b80 10 client.26044 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:18.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b44c0edde0 con 0x55b44c370780 2023-10-16T14:37:18.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b44c2c7ef0 con 0x55b44c370780 2023-10-16T14:37:18.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1f6001fbd0 con 0x55b44c370780 2023-10-16T14:37:18.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f86353b80 10 client.26044 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:18.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.575+0000 7f1f86353b80 10 client.26044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:18.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f86353b80 20 client.26044 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:18.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c385610 0x55b44c387a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f86353b80 4 client.26044 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:18.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b44c33fc10 con 0x55b44c385610 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c385610 0x55b44c387a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f4b7fe700 10 client.26044 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.576+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c385610 0x55b44c387a30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1811539452c780c4 server_cookie=6aae964dad62d093 in_seq=0 out_seq=0 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/1587239401 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b44c33fc10 con 0x55b44c385610 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f4b7fe700 10 client.26044 handle_command_reply: tid=0 2023-10-16T14:37:18.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 client.26044 shutdown 2023-10-16T14:37:18.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 msgr2=0x55b44c37a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37a200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1f6009a1a0 tx=0x7f1f60096680 comp rx=0 tx=0).stop 2023-10-16T14:37:18.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 shutdown_connections 2023-10-16T14:37:18.602 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c385610 0x55b44c387a30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c381240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c379cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 --2- 172.21.15.178:0/1587239401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37a200 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f4b7fe700 0 client.26044 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 >> 172.21.15.178:0/1587239401 conn(0x55b44c29ddd0 msgr2=0x55b44c36abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 shutdown_connections 2023-10-16T14:37:18.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.577+0000 7f1f86353b80 1 -- 172.21.15.178:0/1587239401 wait complete. 2023-10-16T14:37:18.604 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.578+0000 7f1f86353b80 20 client.26044 trim_cache size 0 max 16384 2023-10-16T14:37:18.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 Processor -- start 2023-10-16T14:37:18.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 -- start start 2023-10-16T14:37:18.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c381200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 0x55b44c3697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c143920 con 0x55b44c371620 2023-10-16T14:37:18.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c36fac0 con 0x55b44c36fe20 2023-10-16T14:37:18.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.592+0000 7f1f86353b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c3714a0 con 0x55b44c370780 2023-10-16T14:37:18.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f70f9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 0x55b44c3697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c381200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f70f9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 0x55b44c3697a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:45900/0 (socket says 172.21.15.178:45900) 2023-10-16T14:37:18.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f70f9c700 1 -- 172.21.15.178:0/395597747 learned_addr learned my addr 172.21.15.178:0/395597747 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/395597747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 msgr2=0x55b44c370200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/395597747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 msgr2=0x55b44c3697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 0x55b44c3697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/395597747 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b44c0d6c60 con 0x55b44c370780 2023-10-16T14:37:18.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c381200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f60098560 tx=0x7f1f6000cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f30ec8ccd68f1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f52ffd700 1 -- 172.21.15.178:0/395597747 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b1070 con 0x55b44c370780 2023-10-16T14:37:18.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f52ffd700 1 -- 172.21.15.178:0/395597747 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f6009e170 con 0x55b44c370780 2023-10-16T14:37:18.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.593+0000 7f1f52ffd700 1 -- 172.21.15.178:0/395597747 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a2370 con 0x55b44c370780 2023-10-16T14:37:18.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 -- 172.21.15.178:0/395597747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 msgr2=0x55b44c381200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c381200 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f60098560 tx=0x7f1f6000cf10 comp rx=0 tx=0).stop 2023-10-16T14:37:18.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 -- 172.21.15.178:0/395597747 shutdown_connections 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c371620 0x55b44c3697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c36fe20 0x55b44c370200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 --2- 172.21.15.178:0/395597747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c381200 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 -- 172.21.15.178:0/395597747 >> 172.21.15.178:0/395597747 conn(0x55b44c29ddd0 msgr2=0x55b44c378950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 -- 172.21.15.178:0/395597747 shutdown_connections 2023-10-16T14:37:18.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.594+0000 7f1f86353b80 1 -- 172.21.15.178:0/395597747 wait complete. 2023-10-16T14:37:18.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 -- start start 2023-10-16T14:37:18.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 Processor -- start 2023-10-16T14:37:18.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c37ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c3803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c380920 con 0x55b44c36fe20 2023-10-16T14:37:18.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c380a90 con 0x55b44c371620 2023-10-16T14:37:18.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44c380c50 con 0x55b44c370780 2023-10-16T14:37:18.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f86353b80 10 client.0 _ll_register_callbacks cb 0x7f1f60005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:18.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f6b7fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60598/0 (socket says 172.21.15.178:60598) 2023-10-16T14:37:18.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3276009971 learned_addr learned my addr 172.21.15.178:0/3276009971 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f6bfff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c37ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.595+0000 7f1f70f9c700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c3803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 msgr2=0x55b44c3803e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c3803e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.620 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3276009971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 msgr2=0x55b44c37ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c37ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 -- 172.21.15.178:0/3276009971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b44c0edde0 con 0x55b44c370780 2023-10-16T14:37:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c37ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:37:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f6b7fe700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009f90 tx=0x7f1f60003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38da60522e9f5ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:18.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f4b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:18.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f600b1050 con 0x55b44c370780 2023-10-16T14:37:18.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1f600a25a0 con 0x55b44c370780 2023-10-16T14:37:18.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f6009a210 con 0x55b44c370780 2023-10-16T14:37:18.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6009e360 con 0x55b44c370780 2023-10-16T14:37:18.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f86353b80 10 client.26050 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.596+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55b44c114250 con 0x55b44c370780 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f6009a210 con 0x55b44c370780 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f1f60099270 con 0x55b44c370780 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:18.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 20 client.26050 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:18.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c3d9d30 0x55b44c3dc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:18.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 4 client.26050 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:37:18.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b44c2bbe50 con 0x55b44c3d9d30 2023-10-16T14:37:18.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.597+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c3d9d30 0x55b44c3dc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:18.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f6bfff700 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c3d9d30 0x55b44c3dc150 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc0dfdde04fb265 server_cookie=fec7302f134013b5 in_seq=0 out_seq=0 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f4b7fe700 10 client.26050 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f4b7fe700 1 -- 172.21.15.178:0/3276009971 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b44c2bbe50 con 0x55b44c3d9d30 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f4b7fe700 10 client.26050 handle_command_reply: tid=0 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 client.26050 shutdown 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 msgr2=0x55b44c37d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009f90 tx=0x7f1f60003680 comp rx=0 tx=0).stop 2023-10-16T14:37:18.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 shutdown_connections 2023-10-16T14:37:18.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55b44c3d9d30 0x55b44c3dc150 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b44c371620 0x55b44c3803e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f4b7fe700 0 client.26050 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:18.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b44c36fe20 0x55b44c37ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 --2- 172.21.15.178:0/3276009971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b44c370780 0x55b44c37d3c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 >> 172.21.15.178:0/3276009971 conn(0x55b44c29ddd0 msgr2=0x55b44c378950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 shutdown_connections 2023-10-16T14:37:18.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 1 -- 172.21.15.178:0/3276009971 wait complete. 2023-10-16T14:37:18.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.598+0000 7f1f86353b80 20 client.26050 trim_cache size 0 max 16384 2023-10-16T14:37:18.628 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:37:18.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 msgr2=0x7f1f540a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 0x7f1f540a39c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009ec0 tx=0x7f1f640134a0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 msgr2=0x7f1f6c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000e9c0 tx=0x7f1f6000ecd0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 shutdown_connections 2023-10-16T14:37:18.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f540a1500 0x7f1f540a39c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f6c102a20 0x7f1f6c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f6c100e80 0x7f1f6c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 --2- 172.21.15.178:0/3241938436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f6c102060 0x7f1f6c199360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 >> 172.21.15.178:0/3241938436 conn(0x7f1f6c0fc400 msgr2=0x7f1f6c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:18.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 shutdown_connections 2023-10-16T14:37:18.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:18.599+0000 7f1f729ff700 1 -- 172.21.15.178:0/3241938436 wait complete. 2023-10-16T14:37:19.633 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:37:19.633 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:37:19.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 -- 172.21.15.178:0/2491565057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 msgr2=0x7f25e8103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 --2- 172.21.15.178:0/2491565057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e8103d70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009990 tx=0x7f25d8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 -- 172.21.15.178:0/2491565057 shutdown_connections 2023-10-16T14:37:19.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 --2- 172.21.15.178:0/2491565057 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 0x7f25e8109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.863 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 --2- 172.21.15.178:0/2491565057 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e8103d70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 --2- 172.21.15.178:0/2491565057 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 0x7f25e81066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 -- 172.21.15.178:0/2491565057 >> 172.21.15.178:0/2491565057 conn(0x7f25e80fb370 msgr2=0x7f25e80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 -- 172.21.15.178:0/2491565057 shutdown_connections 2023-10-16T14:37:19.864 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.858+0000 7f25ecdff700 1 -- 172.21.15.178:0/2491565057 wait complete. 2023-10-16T14:37:19.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.859+0000 7f25ecdff700 1 Processor -- start 2023-10-16T14:37:19.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 -- start start 2023-10-16T14:37:19.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 0x7f25e80ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 0x7f25e80ffe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8100610 con 0x7f25e81042b0 2023-10-16T14:37:19.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8100780 con 0x7f25e8106be0 2023-10-16T14:37:19.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25ecdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81a0c10 con 0x7f25e8101980 2023-10-16T14:37:19.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:44644/0 (socket says 172.21.15.178:44644) 2023-10-16T14:37:19.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 -- 172.21.15.178:0/232611370 learned_addr learned my addr 172.21.15.178:0/232611370 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:19.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25dffff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 0x7f25e80ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 0x7f25e80ffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 -- 172.21.15.178:0/232611370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 msgr2=0x7f25e80ffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 0x7f25e80ffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 -- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 msgr2=0x7f25e80ff930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 0x7f25e80ff930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.860+0000 7f25e659c700 1 -- 172.21.15.178:0/232611370 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80095f0 con 0x7f25e8101980 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25e659c700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25df7fe700 1 -- 172.21.15.178:0/232611370 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d801d070 con 0x7f25e8101980 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25df7fe700 1 -- 172.21.15.178:0/232611370 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d800fb90 con 0x7f25e8101980 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e81a0e90 con 0x7f25e8101980 2023-10-16T14:37:19.870 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25df7fe700 1 -- 172.21.15.178:0/232611370 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80173f0 con 0x7f25e8101980 2023-10-16T14:37:19.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.861+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a1260 con 0x7f25e8101980 2023-10-16T14:37:19.873 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.865+0000 7f25df7fe700 1 -- 172.21.15.178:0/232611370 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f25d8021410 con 0x7f25e8101980 2023-10-16T14:37:19.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 Processor -- start 2023-10-16T14:37:19.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 -- start start 2023-10-16T14:37:19.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb8f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb8fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bb02cf0 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bae7ce0 con 0x55ec4bb8e800 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f2600778b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bae5600 con 0x55ec4bb8dea0 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.867+0000 7f25df7fe700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 0x7f25d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25df7fe700 1 -- 172.21.15.178:0/232611370 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25d80ce360 con 0x7f25e8101980 2023-10-16T14:37:19.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:44656/0 (socket says 172.21.15.178:44656) 2023-10-16T14:37:19.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3801904482 learned_addr learned my addr 172.21.15.178:0/3801904482 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:19.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 0x7f25d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25dffff700 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb8fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e659c700 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb8f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3801904482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 msgr2=0x55ec4bb8f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb8f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3801904482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 msgr2=0x55ec4bb8fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb8fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3801904482 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4ba9e470 con 0x55ec4bb8dea0 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.868+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e0008f20 tx=0x7f25e001a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555cd2ba2cabb792 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 0x7f25d40a3940 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f25e001aa10 tx=0x7f25e000e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f25dcff9700 1 -- 172.21.15.178:0/3801904482 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0027070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f25dcff9700 1 -- 172.21.15.178:0/3801904482 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e001f070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.878 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f25dcff9700 1 -- 172.21.15.178:0/3801904482 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000e070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f2600778b80 1 -- 172.21.15.178:0/3801904482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 msgr2=0x55ec4bb8e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.869+0000 7f2600778b80 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f25e0008f20 tx=0x7f25e001a6c0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 -- 172.21.15.178:0/3801904482 shutdown_connections 2023-10-16T14:37:19.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb8fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb8e2c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 --2- 172.21.15.178:0/3801904482 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb8f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 -- 172.21.15.178:0/3801904482 >> 172.21.15.178:0/3801904482 conn(0x55ec4babbe50 msgr2=0x55ec4bb89060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 -- 172.21.15.178:0/3801904482 shutdown_connections 2023-10-16T14:37:19.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 -- 172.21.15.178:0/3801904482 wait complete. 2023-10-16T14:37:19.880 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.870+0000 7f2600778b80 1 -- start start 2023-10-16T14:37:19.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 Processor -- start 2023-10-16T14:37:19.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb98290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bae0ef0 con 0x55ec4bb8dea0 2023-10-16T14:37:19.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4ba7d790 con 0x55ec4bb8e800 2023-10-16T14:37:19.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4ba7d430 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f2600778b80 10 client.0 _ll_register_callbacks cb 0x55ec4b8da880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:19.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f25e659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.885 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f25e6d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f25e659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb97d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:46238/0 (socket says 172.21.15.178:46238) 2023-10-16T14:37:19.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f25e659c700 1 -- 172.21.15.178:0/3837797345 learned_addr learned my addr 172.21.15.178:0/3837797345 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:19.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.871+0000 7f25dffff700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb98290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 msgr2=0x55ec4bb98290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.886 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb98290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3837797345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 msgr2=0x55ec4bb97d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb97d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 -- 172.21.15.178:0/3837797345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e0008ad0 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f2d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25e001e000 tx=0x7f25e001bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d15089dab7db728 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:19.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0027040 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec4bae5f70 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.872+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25e0009210 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25e0003e10 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 10 client.26062 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:19.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ec4b90be60 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec4bae5f70 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f25e0007aa0 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:19.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 10 client.26062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:19.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 20 client.26062 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:19.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bba5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 4 client.26062 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:19.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.873+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec4bb5dc90 con 0x55ec4bba3b70 2023-10-16T14:37:19.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.874+0000 7f25e659c700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bba5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.874+0000 7f25e659c700 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bba5f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89210c4b8cda8887 server_cookie=da7e2504356b7169 in_seq=0 out_seq=0 2023-10-16T14:37:19.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.874+0000 7f25cdffb700 10 client.26062 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:19.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f25cdffb700 1 -- 172.21.15.178:0/3837797345 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ec4bb5dc90 con 0x55ec4bba3b70 2023-10-16T14:37:19.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f25cdffb700 10 client.26062 handle_command_reply: tid=0 2023-10-16T14:37:19.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 client.26062 shutdown 2023-10-16T14:37:19.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 msgr2=0x55ec4bb9f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f2d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25e001e000 tx=0x7f25e001bd70 comp rx=0 tx=0).stop 2023-10-16T14:37:19.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 shutdown_connections 2023-10-16T14:37:19.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bba5f90 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f2d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f25cdffb700 0 client.26062 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:19.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb97d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 --2- 172.21.15.178:0/3837797345 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb98290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 >> 172.21.15.178:0/3837797345 conn(0x55ec4babbe50 msgr2=0x55ec4bb88c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 shutdown_connections 2023-10-16T14:37:19.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.875+0000 7f2600778b80 1 -- 172.21.15.178:0/3837797345 wait complete. 2023-10-16T14:37:19.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.876+0000 7f2600778b80 20 client.26062 trim_cache size 0 max 16384 2023-10-16T14:37:19.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 Processor -- start 2023-10-16T14:37:19.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 -- start start 2023-10-16T14:37:19.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4b9619a0 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bae5f70 con 0x55ec4bb8e800 2023-10-16T14:37:19.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f2600778b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bb96790 con 0x55ec4bb8dea0 2023-10-16T14:37:19.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25e6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25e659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25e6d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:46244/0 (socket says 172.21.15.178:46244) 2023-10-16T14:37:19.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25e659c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:44674/0 (socket says 172.21.15.178:44674) 2023-10-16T14:37:19.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25e6d9d700 1 -- 172.21.15.178:0/2884340092 learned_addr learned my addr 172.21.15.178:0/2884340092 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:19.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.897+0000 7f25dffff700 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 -- 172.21.15.178:0/2884340092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 msgr2=0x55ec4bb89120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb89120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 -- 172.21.15.178:0/2884340092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 msgr2=0x55ec4bb9f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 -- 172.21.15.178:0/2884340092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec4b8f4ce0 con 0x55ec4bb8dea0 2023-10-16T14:37:19.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25dffff700 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:19.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25e659c700 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006010 tx=0x7f25d8005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=922d765404b391c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25dd7fa700 1 -- 172.21.15.178:0/2884340092 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d809bb40 con 0x55ec4bb8dea0 2023-10-16T14:37:19.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25dd7fa700 1 -- 172.21.15.178:0/2884340092 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25d809b440 con 0x55ec4bb8dea0 2023-10-16T14:37:19.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.898+0000 7f25dd7fa700 1 -- 172.21.15.178:0/2884340092 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80ce1b0 con 0x55ec4bb8dea0 2023-10-16T14:37:19.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- 172.21.15.178:0/2884340092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 msgr2=0x55ec4bb88be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f25d8006010 tx=0x7f25d8005ef0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- 172.21.15.178:0/2884340092 shutdown_connections 2023-10-16T14:37:19.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bb9f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb88be0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 --2- 172.21.15.178:0/2884340092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8e800 0x55ec4bb89120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- 172.21.15.178:0/2884340092 >> 172.21.15.178:0/2884340092 conn(0x55ec4babbe50 msgr2=0x55ec4bb92610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- 172.21.15.178:0/2884340092 shutdown_connections 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- 172.21.15.178:0/2884340092 wait complete. 2023-10-16T14:37:19.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.899+0000 7f2600778b80 1 -- start start 2023-10-16T14:37:19.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 Processor -- start 2023-10-16T14:37:19.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8e800 0x55ec4bb9e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bbf79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bbf7f10 con 0x55ec4bb8e800 2023-10-16T14:37:19.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bbf8050 con 0x55ec4bb8dea0 2023-10-16T14:37:19.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec4bbf8190 con 0x55ec4bb8f6a0 2023-10-16T14:37:19.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f2600778b80 10 client.0 _ll_register_callbacks cb 0x7f25d0009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:19.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f25e659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f25e659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:36136/0 (socket says 172.21.15.178:36136) 2023-10-16T14:37:19.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f25e659c700 1 -- 172.21.15.178:0/2580484971 learned_addr learned my addr 172.21.15.178:0/2580484971 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:19.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f25e6d9d700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bbf79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.900+0000 7f25dffff700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8e800 0x55ec4bb9e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 -- 172.21.15.178:0/2580484971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 msgr2=0x55ec4bbf79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bbf79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 -- 172.21.15.178:0/2580484971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8e800 msgr2=0x55ec4bb9e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:19.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8e800 0x55ec4bb9e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 -- 172.21.15.178:0/2580484971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d0009b20 con 0x55ec4bb8dea0 2023-10-16T14:37:19.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25e659c700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f25d8097eb0 tx=0x7f25d8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e259bd33d9c5c5b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:19.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80a3740 con 0x55ec4bb8dea0 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25d80b0070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25e001f070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d80ce190 con 0x55ec4bb8dea0 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f2600778b80 10 client.26090 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:19.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.901+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f25d80b3e70 con 0x55ec4bb8dea0 2023-10-16T14:37:19.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25e001f070 con 0x55ec4bb8dea0 2023-10-16T14:37:19.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7f25d809a930 con 0x55ec4bb8dea0 2023-10-16T14:37:19.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:19.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 10 client.26090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:19.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 20 client.26090 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:19.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bbfb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:19.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 4 client.26090 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:37:19.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ec4bad9ed0 con 0x55ec4bba3b70 2023-10-16T14:37:19.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f25dffff700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bbfb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:19.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f25cdffb700 10 client.26090 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:19.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.902+0000 7f25dffff700 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bbfb940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f393bc0281ed6c35 server_cookie=e4b26a33e7eeca1b in_seq=0 out_seq=0 2023-10-16T14:37:19.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.903+0000 7f25cdffb700 1 -- 172.21.15.178:0/2580484971 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ec4bad9ed0 con 0x55ec4bba3b70 2023-10-16T14:37:19.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f25cdffb700 10 client.26090 handle_command_reply: tid=0 2023-10-16T14:37:19.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 client.26090 shutdown 2023-10-16T14:37:19.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 msgr2=0x55ec4bb9e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f25d8097eb0 tx=0x7f25d8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 shutdown_connections 2023-10-16T14:37:19.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ec4bba3b70 0x55ec4bbfb940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec4bb8f6a0 0x55ec4bbf79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ec4bb8dea0 0x55ec4bb9e3f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 --2- 172.21.15.178:0/2580484971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ec4bb8e800 0x55ec4bb9e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 >> 172.21.15.178:0/2580484971 conn(0x55ec4babbe50 msgr2=0x55ec4bb8a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f25cdffb700 0 client.26090 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:19.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 shutdown_connections 2023-10-16T14:37:19.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 1 -- 172.21.15.178:0/2580484971 wait complete. 2023-10-16T14:37:19.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.904+0000 7f2600778b80 20 client.26090 trim_cache size 0 max 16384 2023-10-16T14:37:19.922 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:37:19.923 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:37:19.923 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "1d3bdc57-7ad4-47f9-84f2-e67faf5948b0", 2023-10-16T14:37:19.923 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:37:19.923 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:37:19.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 msgr2=0x7f25d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 0x7f25d40a3940 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f25e001aa10 tx=0x7f25e000e000 comp rx=0 tx=0).stop 2023-10-16T14:37:19.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 msgr2=0x7f25e81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:19.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f25d8000c00 tx=0x7f25d80057a0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 shutdown_connections 2023-10-16T14:37:19.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f25d40a1480 0x7f25d40a3940 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f25e8106be0 0x7f25e80ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e8101980 0x7f25e81012f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 --2- 172.21.15.178:0/232611370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e81042b0 0x7f25e80ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:19.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 >> 172.21.15.178:0/232611370 conn(0x7f25e80fb370 msgr2=0x7f25e81025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:19.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 shutdown_connections 2023-10-16T14:37:19.926 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:19.905+0000 7f25ecdff700 1 -- 172.21.15.178:0/232611370 wait complete. 2023-10-16T14:37:19.932 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:37:19.997 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:19 smithi178 ceph-mon[98495]: pgmap v447: 129 pgs: 129 active+clean; 592 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-10-16T14:37:20.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:19 smithi186 ceph-mon[134892]: pgmap v447: 129 pgs: 129 active+clean; 592 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-10-16T14:37:20.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.145+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1490044138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102a40 msgr2=0x7fb4b0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.145+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1490044138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102a40 0x7fb4b0109360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b0a0 tx=0x7fb4a800b3b0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1490044138 shutdown_connections 2023-10-16T14:37:20.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1490044138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102a40 0x7fb4b0109360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1490044138 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1490044138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102080 0x7fb4b0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1490044138 >> 172.21.15.178:0/1490044138 conn(0x7fb4b00fc420 msgr2=0x7fb4b00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1490044138 shutdown_connections 2023-10-16T14:37:20.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.146+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1490044138 wait complete. 2023-10-16T14:37:20.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 Processor -- start 2023-10-16T14:37:20.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 -- start start 2023-10-16T14:37:20.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102080 0x7fb4b0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102a40 0x7fb4b01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b01a0ac0 con 0x7fb4b0102a40 2023-10-16T14:37:20.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b01a0c00 con 0x7fb4b0100ea0 2023-10-16T14:37:20.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b73ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b01a0d90 con 0x7fb4b0102080 2023-10-16T14:37:20.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:36156/0 (socket says 172.21.15.178:36156) 2023-10-16T14:37:20.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.147+0000 7fb4b519b700 1 -- 172.21.15.178:0/1496333675 learned_addr learned my addr 172.21.15.178:0/1496333675 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:20.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b599c700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102a40 0x7fb4b01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b499a700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102080 0x7fb4b0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 -- 172.21.15.178:0/1496333675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102080 msgr2=0x7fb4b0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102080 0x7fb4b0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 -- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102a40 msgr2=0x7fb4b01a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102a40 0x7fb4b01a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 -- 172.21.15.178:0/1496333675 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a8009d00 con 0x7fb4b0100ea0 2023-10-16T14:37:20.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00e970 tx=0x7fb4ac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4a67fc700 1 -- 172.21.15.178:0/1496333675 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac009790 con 0x7fb4b0100ea0 2023-10-16T14:37:20.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b01a1070 con 0x7fb4b0100ea0 2023-10-16T14:37:20.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4a67fc700 1 -- 172.21.15.178:0/1496333675 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ac00eec0 con 0x7fb4b0100ea0 2023-10-16T14:37:20.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.148+0000 7fb4a67fc700 1 -- 172.21.15.178:0/1496333675 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac003cf0 con 0x7fb4b0100ea0 2023-10-16T14:37:20.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.149+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b01a1580 con 0x7fb4b0100ea0 2023-10-16T14:37:20.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.152+0000 7fb4a67fc700 1 -- 172.21.15.178:0/1496333675 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb4ac00ccd0 con 0x7fb4b0100ea0 2023-10-16T14:37:20.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 Processor -- start 2023-10-16T14:37:20.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 -- start start 2023-10-16T14:37:20.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083d7fc70 con 0x558083e0b780 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083d64c60 con 0x558083e0ae20 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083d62580 con 0x558083e0c620 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.153+0000 7fb4a67fc700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 0x7fb49c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4a67fc700 1 -- 172.21.15.178:0/1496333675 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb4ac014070 con 0x7fb4b0100ea0 2023-10-16T14:37:20.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b499a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b599c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:46292/0 (socket says 172.21.15.178:46292) 2023-10-16T14:37:20.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 -- 172.21.15.178:0/111556945 learned_addr learned my addr 172.21.15.178:0/111556945 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:20.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b599c700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 0x7fb49c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 -- 172.21.15.178:0/111556945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 msgr2=0x558083e0caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e0caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 -- 172.21.15.178:0/111556945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 msgr2=0x558083e0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e0b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 -- 172.21.15.178:0/111556945 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558083d1b3f0 con 0x558083e0b780 2023-10-16T14:37:20.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b499a700 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e0caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:20.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.154+0000 7fb4b519b700 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac096790 tx=0x7fb4ac099bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed1970ac8e30ac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb49b7fe700 1 -- 172.21.15.178:0/111556945 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac094ea0 con 0x558083e0b780 2023-10-16T14:37:20.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4b599c700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 0x7fb49c0a3940 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b780 tx=0x7fb4a8006230 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb49b7fe700 1 -- 172.21.15.178:0/111556945 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ac09e3e0 con 0x558083e0b780 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb49b7fe700 1 -- 172.21.15.178:0/111556945 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac09a4f0 con 0x558083e0b780 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 -- 172.21.15.178:0/111556945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 msgr2=0x558083e0c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac096790 tx=0x7fb4ac099bf0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 -- 172.21.15.178:0/111556945 shutdown_connections 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e0caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/111556945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e0c040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.155+0000 7fb4cade7b80 1 -- 172.21.15.178:0/111556945 >> 172.21.15.178:0/111556945 conn(0x558083d38dd0 msgr2=0x558083e05fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.156+0000 7fb4cade7b80 1 -- 172.21.15.178:0/111556945 shutdown_connections 2023-10-16T14:37:20.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.156+0000 7fb4cade7b80 1 -- 172.21.15.178:0/111556945 wait complete. 2023-10-16T14:37:20.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.156+0000 7fb4cade7b80 1 -- start start 2023-10-16T14:37:20.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.156+0000 7fb4cade7b80 1 Processor -- start 2023-10-16T14:37:20.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 0x558083e14c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 0x558083e151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083d5de70 con 0x558083e0ae20 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083cfa710 con 0x558083e0c620 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083cfa3b0 con 0x558083e0b780 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4cade7b80 10 client.0 _ll_register_callbacks cb 0x558083b57800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 0x558083e14c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4b499a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 0x558083e151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4b499a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 0x558083e151b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:44716/0 (socket says 172.21.15.178:44716) 2023-10-16T14:37:20.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4b499a700 1 -- 172.21.15.178:0/3943180853 learned_addr learned my addr 172.21.15.178:0/3943180853 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:20.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.157+0000 7fb4b599c700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 -- 172.21.15.178:0/3943180853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 msgr2=0x558083e151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 0x558083e151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 -- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 msgr2=0x558083e1c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 -- 172.21.15.178:0/3943180853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a0009830 con 0x558083e0ae20 2023-10-16T14:37:20.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b599c700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:37:20.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 0x558083e14c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac096790 tx=0x7fb4ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6301ff12a444652 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:37:20.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac094ea0 con 0x558083e0ae20 2023-10-16T14:37:20.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558083d62ef0 con 0x558083e0ae20 2023-10-16T14:37:20.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.158+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb4ac0a2620 con 0x558083e0ae20 2023-10-16T14:37:20.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ac09a6f0 con 0x558083e0ae20 2023-10-16T14:37:20.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 10 client.16029 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:20.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb4b01a1580 con 0x558083e0ae20 2023-10-16T14:37:20.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558083d62ef0 con 0x558083e0ae20 2023-10-16T14:37:20.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb4ac096080 con 0x558083e0ae20 2023-10-16T14:37:20.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:20.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:20.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 20 client.16029 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:20.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e20660 0x558083e22a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 4 client.16029 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:20.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.159+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558083ddac10 con 0x558083e20660 2023-10-16T14:37:20.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.160+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e20660 0x558083e22a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.160+0000 7fb498ff9700 10 client.16029 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:20.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.160+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e20660 0x558083e22a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65278ffe7c208b12 server_cookie=f62751dcc6458f83 in_seq=0 out_seq=0 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.163+0000 7fb498ff9700 1 -- 172.21.15.178:0/3943180853 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558083ddac10 con 0x558083e20660 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.163+0000 7fb498ff9700 10 client.16029 handle_command_reply: tid=0 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.163+0000 7fb4cade7b80 1 client.16029 shutdown 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 msgr2=0x558083e14c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 0x558083e14c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac096790 tx=0x7fb4ac004750 comp rx=0 tx=0).stop 2023-10-16T14:37:20.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 shutdown_connections 2023-10-16T14:37:20.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e20660 0x558083e22a80 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0ae20 0x558083e14c70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3943180853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0b780 0x558083e151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 >> 172.21.15.178:0/3943180853 conn(0x558083d38dd0 msgr2=0x558083e05bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb498ff9700 0 client.16029 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:20.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 shutdown_connections 2023-10-16T14:37:20.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3943180853 wait complete. 2023-10-16T14:37:20.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.164+0000 7fb4cade7b80 20 client.16029 trim_cache size 0 max 16384 2023-10-16T14:37:20.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:19 smithi187 ceph-mon[135297]: pgmap v447: 129 pgs: 129 active+clean; 592 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 810 op/s 2023-10-16T14:37:20.188 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 Processor -- start 2023-10-16T14:37:20.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 -- start start 2023-10-16T14:37:20.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.189 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e12cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083bde920 con 0x558083e0b780 2023-10-16T14:37:20.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083d62ef0 con 0x558083e0ae20 2023-10-16T14:37:20.191 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.185+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083e14120 con 0x558083e0c620 2023-10-16T14:37:20.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b499a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:36186/0 (socket says 172.21.15.178:36186) 2023-10-16T14:37:20.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 -- 172.21.15.178:0/1272797258 learned_addr learned my addr 172.21.15.178:0/1272797258 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:20.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b599c700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e12cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 -- 172.21.15.178:0/1272797258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 msgr2=0x558083e12cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e12cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 -- 172.21.15.178:0/1272797258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 msgr2=0x558083e1c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 -- 172.21.15.178:0/1272797258 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558083b88de0 con 0x558083e0ae20 2023-10-16T14:37:20.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b599c700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:20.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b499a700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:20.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.186+0000 7fb4b519b700 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0985e0 tx=0x7fb4ac0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69da069eaa6aba23 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb49bfff700 1 -- 172.21.15.178:0/1272797258 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac094ea0 con 0x558083e0ae20 2023-10-16T14:37:20.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb49bfff700 1 -- 172.21.15.178:0/1272797258 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ac0a2620 con 0x558083e0ae20 2023-10-16T14:37:20.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb49bfff700 1 -- 172.21.15.178:0/1272797258 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac09a750 con 0x558083e0ae20 2023-10-16T14:37:20.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 -- 172.21.15.178:0/1272797258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 msgr2=0x558083e127b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0985e0 tx=0x7fb4ac0047c0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 -- 172.21.15.178:0/1272797258 shutdown_connections 2023-10-16T14:37:20.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0c620 0x558083e12cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0ae20 0x558083e127b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/1272797258 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.187+0000 7fb4cade7b80 1 -- 172.21.15.178:0/1272797258 >> 172.21.15.178:0/1272797258 conn(0x558083d38dd0 msgr2=0x558083e1cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.188+0000 7fb4cade7b80 1 -- 172.21.15.178:0/1272797258 shutdown_connections 2023-10-16T14:37:20.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.188+0000 7fb4cade7b80 1 -- 172.21.15.178:0/1272797258 wait complete. 2023-10-16T14:37:20.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.188+0000 7fb4cade7b80 1 -- start start 2023-10-16T14:37:20.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.188+0000 7fb4cade7b80 1 Processor -- start 2023-10-16T14:37:20.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083e1a970 con 0x558083e0b780 2023-10-16T14:37:20.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083e1aae0 con 0x558083e0c620 2023-10-16T14:37:20.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x558083e1aca0 con 0x558083e0ae20 2023-10-16T14:37:20.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4cade7b80 10 client.0 _ll_register_callbacks cb 0x7fb4a0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:44726/0 (socket says 172.21.15.178:44726) 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 -- 172.21.15.178:0/3669693477 learned_addr learned my addr 172.21.15.178:0/3669693477 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b599c700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.201 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 -- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 msgr2=0x558083e1a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 -- 172.21.15.178:0/3669693477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 msgr2=0x558083e1c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:20.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 -- 172.21.15.178:0/3669693477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a0009d30 con 0x558083e0ae20 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb4b519b700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0cade0 tx=0x7fb4ac096420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc6f217e56473f23 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb498ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.189+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac094ea0 con 0x558083e0ae20 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4ac09e3e0 con 0x558083e0ae20 2023-10-16T14:37:20.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb4ac09e120 con 0x558083e0ae20 2023-10-16T14:37:20.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4ac099bf0 con 0x558083e0ae20 2023-10-16T14:37:20.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 10 client.26086 fetch_fsmap learned FSMap version 30 2023-10-16T14:37:20.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fb4b0109c90 con 0x558083e0ae20 2023-10-16T14:37:20.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4ac09e3e0 con 0x558083e0ae20 2023-10-16T14:37:20.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 30) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fb4ac083170 con 0x558083e0ae20 2023-10-16T14:37:20.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 30 2023-10-16T14:37:20.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 10 client.26086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:20.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 20 client.26086 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:20.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e753a0 0x558083e777c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:20.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 4 client.26086 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:37:20.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558083d56e50 con 0x558083e753a0 2023-10-16T14:37:20.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.190+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e753a0 0x558083e777c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:20.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.191+0000 7fb4b499a700 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e753a0 0x558083e777c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b56d02253ce7f8 server_cookie=8540cd02ada5de73 in_seq=0 out_seq=0 2023-10-16T14:37:20.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.191+0000 7fb498ff9700 10 client.26086 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:20.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb498ff9700 1 -- 172.21.15.178:0/3669693477 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x558083d56e50 con 0x558083e753a0 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb498ff9700 10 client.26086 handle_command_reply: tid=0 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 client.26086 shutdown 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 msgr2=0x558083e1bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac0cade0 tx=0x7fb4ac096420 comp rx=0 tx=0).stop 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 shutdown_connections 2023-10-16T14:37:20.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x558083e753a0 0x558083e777c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x558083e0c620 0x558083e1a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558083e0ae20 0x558083e1bc50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 --2- 172.21.15.178:0/3669693477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x558083e0b780 0x558083e1c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 >> 172.21.15.178:0/3669693477 conn(0x558083d38dd0 msgr2=0x558083e127b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb498ff9700 0 client.26086 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:20.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.198+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 shutdown_connections 2023-10-16T14:37:20.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4cade7b80 1 -- 172.21.15.178:0/3669693477 wait complete. 2023-10-16T14:37:20.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4cade7b80 20 client.26086 trim_cache size 0 max 16384 2023-10-16T14:37:20.210 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:37:20.211 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "scrub active (2421 inodes in the stack)", 2023-10-16T14:37:20.211 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:37:20.211 INFO:teuthology.orchestra.run.smithi178.stdout: "1d3bdc57-7ad4-47f9-84f2-e67faf5948b0": { 2023-10-16T14:37:20.211 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:37:20.211 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "1d3bdc57-7ad4-47f9-84f2-e67faf5948b0", 2023-10-16T14:37:20.212 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:37:20.212 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:37:20.212 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:37:20.212 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:37:20.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 msgr2=0x7fb49c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 0x7fb49c0a3940 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800b780 tx=0x7fb4a8006230 comp rx=0 tx=0).stop 2023-10-16T14:37:20.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 msgr2=0x7fb4b0198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:20.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00e970 tx=0x7fb4ac00ec80 comp rx=0 tx=0).stop 2023-10-16T14:37:20.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.199+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 shutdown_connections 2023-10-16T14:37:20.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb49c0a1480 0x7fb49c0a3940 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4b0102a40 0x7fb4b01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4b0100ea0 0x7fb4b0198f40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 --2- 172.21.15.178:0/1496333675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4b0102080 0x7fb4b0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:20.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 >> 172.21.15.178:0/1496333675 conn(0x7fb4b00fc420 msgr2=0x7fb4b00fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:20.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 shutdown_connections 2023-10-16T14:37:20.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:20.200+0000 7fb4b73ff700 1 -- 172.21.15.178:0/1496333675 wait complete. 2023-10-16T14:37:20.222 INFO:tasks.cephfs.filesystem:scrub status for tag:1d3bdc57-7ad4-47f9-84f2-e67faf5948b0 - {'path': '/', 'tag': '1d3bdc57-7ad4-47f9-84f2-e67faf5948b0', 'options': 'recursive,force'} 2023-10-16T14:37:20.995 INFO:tasks.workunit.client.0.smithi178.stdout: 156 414484 703 286328 714 180971 1271 2023-10-16T14:37:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:20 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:37:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:20 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:20 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:37:21.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:20 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:37:21.054 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:20 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:37:21.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:20 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:37:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:20 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:20 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:37:21.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:20 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:37:21.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:20 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:37:21.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:20 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:37:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:20 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:20 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:37:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:20 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:37:21.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:20 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:37:22.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:21 smithi178 ceph-mon[98495]: pgmap v448: 129 pgs: 129 active+clean; 596 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 748 op/s 2023-10-16T14:37:22.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:21 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:22.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:21 smithi186 ceph-mon[134892]: pgmap v448: 129 pgs: 129 active+clean; 596 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 748 op/s 2023-10-16T14:37:22.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:21 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:22.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:21 smithi187 ceph-mon[135297]: pgmap v448: 129 pgs: 129 active+clean; 596 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 748 op/s 2023-10-16T14:37:22.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:21 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:23.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:22 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:23.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:22 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:23.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:22 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:24.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:23 smithi178 ceph-mon[98495]: pgmap v449: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-10-16T14:37:24.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:23 smithi186 ceph-mon[134892]: pgmap v449: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-10-16T14:37:24.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:23 smithi187 ceph-mon[135297]: pgmap v449: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.8 MiB/s wr, 375 op/s 2023-10-16T14:37:25.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:24 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:37:25.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:24 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:25.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:24 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:37:25.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:24 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:25.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:24 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:37:25.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:24 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:26.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:25 smithi178 ceph-mon[98495]: pgmap v450: 129 pgs: 129 active+clean; 602 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-10-16T14:37:26.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:25 smithi186 ceph-mon[134892]: pgmap v450: 129 pgs: 129 active+clean; 602 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-10-16T14:37:26.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:25 smithi187 ceph-mon[135297]: pgmap v450: 129 pgs: 129 active+clean; 602 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 278 op/s 2023-10-16T14:37:27.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:26 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:27.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:26 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:27.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:26 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:37:28.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:27 smithi178 ceph-mon[98495]: pgmap v451: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-10-16T14:37:28.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:27 smithi186 ceph-mon[134892]: pgmap v451: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-10-16T14:37:28.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:27 smithi187 ceph-mon[135297]: pgmap v451: 129 pgs: 129 active+clean; 601 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2023-10-16T14:37:30.057 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:29 smithi178 ceph-mon[98495]: pgmap v452: 129 pgs: 129 active+clean; 603 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2023-10-16T14:37:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:29 smithi186 ceph-mon[134892]: pgmap v452: 129 pgs: 129 active+clean; 603 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2023-10-16T14:37:30.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:29 smithi187 ceph-mon[135297]: pgmap v452: 129 pgs: 129 active+clean; 603 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2023-10-16T14:37:30.998 INFO:tasks.workunit.client.0.smithi178.stdout: 169 462346 680 323527 754 199128 1247 2023-10-16T14:37:31.559 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:31 smithi178 ceph-mon[98495]: pgmap v453: 129 pgs: 129 active+clean; 610 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-10-16T14:37:31.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:31 smithi186 ceph-mon[134892]: pgmap v453: 129 pgs: 129 active+clean; 610 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-10-16T14:37:31.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:31 smithi187 ceph-mon[135297]: pgmap v453: 129 pgs: 129 active+clean; 610 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2023-10-16T14:37:34.063 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:33 smithi178 ceph-mon[98495]: pgmap v454: 129 pgs: 129 active+clean; 613 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-10-16T14:37:34.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:33 smithi186 ceph-mon[134892]: pgmap v454: 129 pgs: 129 active+clean; 613 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-10-16T14:37:34.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:33 smithi187 ceph-mon[135297]: pgmap v454: 129 pgs: 129 active+clean; 613 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2023-10-16T14:37:36.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:35 smithi178 ceph-mon[98495]: pgmap v455: 129 pgs: 129 active+clean; 620 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-10-16T14:37:36.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:35 smithi186 ceph-mon[134892]: pgmap v455: 129 pgs: 129 active+clean; 620 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-10-16T14:37:36.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:35 smithi187 ceph-mon[135297]: pgmap v455: 129 pgs: 129 active+clean; 620 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2023-10-16T14:37:37.055 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:36 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:37:37.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:36 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:37:37.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:36 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:37:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:36 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:37:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:36 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:37:37.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:36 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:37:37.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:36 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:37:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:36 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:37:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:36 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:37:38.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:37 smithi186 ceph-mon[134892]: pgmap v456: 129 pgs: 129 active+clean; 639 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 245 op/s 2023-10-16T14:37:38.089 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:37 smithi178 ceph-mon[98495]: pgmap v456: 129 pgs: 129 active+clean; 639 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 245 op/s 2023-10-16T14:37:38.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:37 smithi187 ceph-mon[135297]: pgmap v456: 129 pgs: 129 active+clean; 639 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 245 op/s 2023-10-16T14:37:40.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:39 smithi178 ceph-mon[98495]: pgmap v457: 129 pgs: 129 active+clean; 647 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 283 op/s 2023-10-16T14:37:40.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:39 smithi186 ceph-mon[134892]: pgmap v457: 129 pgs: 129 active+clean; 647 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 283 op/s 2023-10-16T14:37:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:39 smithi187 ceph-mon[135297]: pgmap v457: 129 pgs: 129 active+clean; 647 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.5 MiB/s wr, 283 op/s 2023-10-16T14:37:40.998 INFO:tasks.workunit.client.0.smithi178.stdout: 182 582424 859 413289 547 248323 1901 2023-10-16T14:37:41.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:41 smithi186 ceph-mon[134892]: pgmap v458: 129 pgs: 129 active+clean; 700 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 539 op/s 2023-10-16T14:37:41.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:41 smithi187 ceph-mon[135297]: pgmap v458: 129 pgs: 129 active+clean; 700 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 539 op/s 2023-10-16T14:37:41.841 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:41 smithi178 ceph-mon[98495]: pgmap v458: 129 pgs: 129 active+clean; 700 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.0 MiB/s wr, 539 op/s 2023-10-16T14:37:43.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:43 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:37:43.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:43 smithi178 ceph-mon[98495]: pgmap v459: 129 pgs: 129 active+clean; 717 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 620 op/s 2023-10-16T14:37:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:43 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:37:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:43 smithi186 ceph-mon[134892]: pgmap v459: 129 pgs: 129 active+clean; 717 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 620 op/s 2023-10-16T14:37:43.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:43 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:37:43.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:43 smithi187 ceph-mon[135297]: pgmap v459: 129 pgs: 129 active+clean; 717 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 620 op/s 2023-10-16T14:37:46.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:45 smithi178 ceph-mon[98495]: pgmap v460: 129 pgs: 129 active+clean; 748 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 719 op/s 2023-10-16T14:37:46.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:45 smithi186 ceph-mon[134892]: pgmap v460: 129 pgs: 129 active+clean; 748 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 719 op/s 2023-10-16T14:37:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:45 smithi187 ceph-mon[135297]: pgmap v460: 129 pgs: 129 active+clean; 748 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 12 MiB/s wr, 719 op/s 2023-10-16T14:37:48.062 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:47 smithi178 ceph-mon[98495]: pgmap v461: 129 pgs: 129 active+clean; 781 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 15 MiB/s wr, 888 op/s 2023-10-16T14:37:48.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:47 smithi186 ceph-mon[134892]: pgmap v461: 129 pgs: 129 active+clean; 781 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 15 MiB/s wr, 888 op/s 2023-10-16T14:37:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:47 smithi187 ceph-mon[135297]: pgmap v461: 129 pgs: 129 active+clean; 781 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 15 MiB/s wr, 888 op/s 2023-10-16T14:37:50.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:49 smithi178 ceph-mon[98495]: pgmap v462: 129 pgs: 129 active+clean; 787 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-16T14:37:50.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:49 smithi186 ceph-mon[134892]: pgmap v462: 129 pgs: 129 active+clean; 787 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-16T14:37:50.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:49 smithi187 ceph-mon[135297]: pgmap v462: 129 pgs: 129 active+clean; 787 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 14 MiB/s wr, 862 op/s 2023-10-16T14:37:50.223 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:37:51.031 INFO:tasks.workunit.client.0.smithi178.stdout: 199 615881 1023 430402 815 273563 2007 2023-10-16T14:37:51.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:51 smithi186 ceph-mon[134892]: pgmap v463: 129 pgs: 129 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-10-16T14:37:51.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:51 smithi187 ceph-mon[135297]: pgmap v463: 129 pgs: 129 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-10-16T14:37:51.810 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:51 smithi178 ceph-mon[98495]: pgmap v463: 129 pgs: 129 active+clean; 789 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 13 MiB/s wr, 902 op/s 2023-10-16T14:37:54.064 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:53 smithi178 ceph-mon[98495]: pgmap v464: 129 pgs: 129 active+clean; 794 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 642 op/s 2023-10-16T14:37:54.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:53 smithi186 ceph-mon[134892]: pgmap v464: 129 pgs: 129 active+clean; 794 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 642 op/s 2023-10-16T14:37:54.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:53 smithi187 ceph-mon[135297]: pgmap v464: 129 pgs: 129 active+clean; 794 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 642 op/s 2023-10-16T14:37:56.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:55 smithi178 ceph-mon[98495]: pgmap v465: 129 pgs: 129 active+clean; 797 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 562 op/s 2023-10-16T14:37:56.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:55 smithi186 ceph-mon[134892]: pgmap v465: 129 pgs: 129 active+clean; 797 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 562 op/s 2023-10-16T14:37:56.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.086+0000 7f74887ff700 1 -- 172.21.15.178:0/1549140497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748010a690 msgr2=0x7f748010cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:56.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.086+0000 7f74887ff700 1 --2- 172.21.15.178:0/1549140497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748010a690 0x7f748010cb20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f747001ac00 tx=0x7f747001af10 comp rx=0 tx=0).stop 2023-10-16T14:37:56.095 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.093+0000 7f74887ff700 1 -- 172.21.15.178:0/1549140497 shutdown_connections 2023-10-16T14:37:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.093+0000 7f74887ff700 1 --2- 172.21.15.178:0/1549140497 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f748010d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.093+0000 7f74887ff700 1 --2- 172.21.15.178:0/1549140497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7480107c90 0x7f748010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.093+0000 7f74887ff700 1 --2- 172.21.15.178:0/1549140497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f748010a690 0x7f748010cb20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.093+0000 7f74887ff700 1 -- 172.21.15.178:0/1549140497 >> 172.21.15.178:0/1549140497 conn(0x7f748006dd20 msgr2=0x7f74800701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:56.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.128+0000 7f74887ff700 1 -- 172.21.15.178:0/1549140497 shutdown_connections 2023-10-16T14:37:56.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.131+0000 7f74887ff700 1 -- 172.21.15.178:0/1549140497 wait complete. 2023-10-16T14:37:56.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.135+0000 7f74887ff700 1 Processor -- start 2023-10-16T14:37:56.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.144+0000 7f74887ff700 1 -- start start 2023-10-16T14:37:56.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.144+0000 7f74887ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.144+0000 7f74887ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.144+0000 7f74887ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.145+0000 7f748659b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.145+0000 7f748659b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41724/0 (socket says 172.21.15.178:41724) 2023-10-16T14:37:56.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.145+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.148+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:48104/0 (socket says 172.21.15.178:48104) 2023-10-16T14:37:56.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.158+0000 7f74887ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801a4eb0 con 0x7f7480107c90 2023-10-16T14:37:56.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.158+0000 7f74887ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801a5020 con 0x7f748010d170 2023-10-16T14:37:56.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.158+0000 7f74887ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801a51e0 con 0x7f748010a690 2023-10-16T14:37:56.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.158+0000 7f7486d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:42978/0 (socket says 172.21.15.178:42978) 2023-10-16T14:37:56.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 -- 172.21.15.178:0/3483932409 learned_addr learned my addr 172.21.15.178:0/3483932409 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:56.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 -- 172.21.15.178:0/3483932409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 msgr2=0x7f748019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:56.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 -- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 msgr2=0x7f748019d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:56.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.159+0000 7f7486d9c700 1 -- 172.21.15.178:0/3483932409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747001a600 con 0x7f748010d170 2023-10-16T14:37:56.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.161+0000 7f7486d9c700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b4a0 tx=0x7f747c00b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:56.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.163+0000 7f74777fe700 1 -- 172.21.15.178:0/3483932409 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c011830 con 0x7f748010d170 2023-10-16T14:37:56.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.163+0000 7f74777fe700 1 -- 172.21.15.178:0/3483932409 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f747c011e50 con 0x7f748010d170 2023-10-16T14:37:56.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.163+0000 7f74777fe700 1 -- 172.21.15.178:0/3483932409 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00f2e0 con 0x7f748010d170 2023-10-16T14:37:56.166 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.163+0000 7f748659b700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-16T14:37:56.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.163+0000 7f7485d9a700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-16T14:37:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:55 smithi187 ceph-mon[135297]: pgmap v465: 129 pgs: 129 active+clean; 797 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.2 MiB/s wr, 562 op/s 2023-10-16T14:37:56.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.168+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74801a54c0 con 0x7f748010d170 2023-10-16T14:37:56.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.169+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74801a5a00 con 0x7f748010d170 2023-10-16T14:37:56.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.212+0000 7f74777fe700 1 -- 172.21.15.178:0/3483932409 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f747c011990 con 0x7f748010d170 2023-10-16T14:37:56.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.223+0000 7f74777fe700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 0x7f746c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.227+0000 7f74777fe700 1 -- 172.21.15.178:0/3483932409 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f747c0cc9d0 con 0x7f748010d170 2023-10-16T14:37:56.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.232+0000 7f749c04fb80 1 Processor -- start 2023-10-16T14:37:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.245+0000 7f748659b700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 0x7f746c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.246+0000 7f748659b700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 0x7f746c0a3b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7480072160 tx=0x7f74780073a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:56.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.273+0000 7f749c04fb80 1 -- start start 2023-10-16T14:37:56.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.273+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c931240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c932aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f749c04fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c8a5c70 con 0x556e8c931780 2023-10-16T14:37:56.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f749c04fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c88ac60 con 0x556e8c932620 2023-10-16T14:37:56.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f749c04fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c888580 con 0x556e8c930e20 2023-10-16T14:37:56.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.274+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41730/0 (socket says 172.21.15.178:41730) 2023-10-16T14:37:56.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 -- 172.21.15.178:0/3725280913 learned_addr learned my addr 172.21.15.178:0/3725280913 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:56.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 -- 172.21.15.178:0/3725280913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 msgr2=0x556e8c931240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:37:56.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c931240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 -- 172.21.15.178:0/3725280913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 msgr2=0x556e8c932aa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:37:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c932aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.275+0000 7f7485d9a700 1 -- 172.21.15.178:0/3725280913 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e8c8413f0 con 0x556e8c931780 2023-10-16T14:37:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.276+0000 7f7485d9a700 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74700036c0 tx=0x7f74700036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d43ac5876ccb2e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:56.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.277+0000 7f7474ff9700 1 -- 172.21.15.178:0/3725280913 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747002f070 con 0x556e8c931780 2023-10-16T14:37:56.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.277+0000 7f7474ff9700 1 -- 172.21.15.178:0/3725280913 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7470005220 con 0x556e8c931780 2023-10-16T14:37:56.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.277+0000 7f7474ff9700 1 -- 172.21.15.178:0/3725280913 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747001cd50 con 0x556e8c931780 2023-10-16T14:37:56.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.292+0000 7f749c04fb80 1 -- 172.21.15.178:0/3725280913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 msgr2=0x556e8c932040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:56.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.293+0000 7f749c04fb80 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f74700036c0 tx=0x7f74700036f0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.323+0000 7f749c04fb80 1 -- 172.21.15.178:0/3725280913 shutdown_connections 2023-10-16T14:37:56.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.323+0000 7f749c04fb80 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c932aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.323+0000 7f749c04fb80 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c931240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.323+0000 7f749c04fb80 1 --2- 172.21.15.178:0/3725280913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c932040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.323+0000 7f749c04fb80 1 -- 172.21.15.178:0/3725280913 >> 172.21.15.178:0/3725280913 conn(0x556e8c85edd0 msgr2=0x556e8c92bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:56.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.340+0000 7f749c04fb80 1 -- 172.21.15.178:0/3725280913 shutdown_connections 2023-10-16T14:37:56.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.340+0000 7f749c04fb80 1 -- 172.21.15.178:0/3725280913 wait complete. 2023-10-16T14:37:56.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.341+0000 7f749c04fb80 1 -- start start 2023-10-16T14:37:56.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.341+0000 7f749c04fb80 1 Processor -- start 2023-10-16T14:37:56.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c942280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c883e70 con 0x556e8c931780 2023-10-16T14:37:56.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c820710 con 0x556e8c930e20 2023-10-16T14:37:56.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c8203b0 con 0x556e8c932620 2023-10-16T14:37:56.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.345+0000 7f749c04fb80 10 client.0 _ll_register_callbacks cb 0x556e8c67d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:56.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.349+0000 7f748659b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.349+0000 7f748659b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:42992/0 (socket says 172.21.15.178:42992) 2023-10-16T14:37:56.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.349+0000 7f748659b700 1 -- 172.21.15.178:0/4248408397 learned_addr learned my addr 172.21.15.178:0/4248408397 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:56.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.352+0000 7f7486d9c700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c942280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.353+0000 7f7485d9a700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f748659b700 1 -- 172.21.15.178:0/4248408397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 msgr2=0x556e8c942280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:56.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f748659b700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c942280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f748659b700 1 -- 172.21.15.178:0/4248408397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 msgr2=0x556e8c93b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:56.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f748659b700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f748659b700 1 -- 172.21.15.178:0/4248408397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748010e240 con 0x556e8c930e20 2023-10-16T14:37:56.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.354+0000 7f7485d9a700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:37:56.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.359+0000 7f748659b700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f747800fb30 tx=0x7f7478004b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4ef6fea13bb8ae server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:56.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.366+0000 7f7469ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:37:56.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.366+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800e9b0 con 0x556e8c930e20 2023-10-16T14:37:56.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.366+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f747800eb10 con 0x556e8c930e20 2023-10-16T14:37:56.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.378+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e8c888ef0 con 0x556e8c930e20 2023-10-16T14:37:56.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.382+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747800ecd0 con 0x556e8c930e20 2023-10-16T14:37:56.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.384+0000 7f749c04fb80 10 client.26114 fetch_fsmap learned FSMap version 31 2023-10-16T14:37:56.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.384+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556e8c6aede0 con 0x556e8c930e20 2023-10-16T14:37:56.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e8c888ef0 con 0x556e8c930e20 2023-10-16T14:37:56.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f74780243a0 con 0x556e8c930e20 2023-10-16T14:37:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 10 client.26114 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:37:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 20 client.26114 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c946b10 0x556e8c948f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:56.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 4 client.26114 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:37:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.385+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e8c900c10 con 0x556e8c946b10 2023-10-16T14:37:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.387+0000 7f7485d9a700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c946b10 0x556e8c948f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.388+0000 7f7485d9a700 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c946b10 0x556e8c948f30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f36bfe6aa389e1 server_cookie=8bc923e45584a87b in_seq=0 out_seq=0 2023-10-16T14:37:56.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.388+0000 7f7469ffb700 10 client.26114 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:56.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.398+0000 7f7469ffb700 1 -- 172.21.15.178:0/4248408397 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556e8c900c10 con 0x556e8c946b10 2023-10-16T14:37:56.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.398+0000 7f7469ffb700 10 client.26114 handle_command_reply: tid=0 2023-10-16T14:37:56.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.401+0000 7f749c04fb80 1 client.26114 shutdown 2023-10-16T14:37:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.401+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 msgr2=0x556e8c93ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:56.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.401+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f747800fb30 tx=0x7f7478004b60 comp rx=0 tx=0).stop 2023-10-16T14:37:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 shutdown_connections 2023-10-16T14:37:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c946b10 0x556e8c948f30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c942280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c930e20 0x556e8c93ad00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 --2- 172.21.15.178:0/4248408397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:56.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.426+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 >> 172.21.15.178:0/4248408397 conn(0x556e8c85edd0 msgr2=0x556e8c92bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.438+0000 7f7469ffb700 0 client.26114 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:56.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.438+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 shutdown_connections 2023-10-16T14:37:56.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.446+0000 7f749c04fb80 1 -- 172.21.15.178:0/4248408397 wait complete. 2023-10-16T14:37:56.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:56.458+0000 7f749c04fb80 20 client.26114 trim_cache size 0 max 16384 2023-10-16T14:37:57.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.102+0000 7f749c04fb80 1 Processor -- start 2023-10-16T14:37:57.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 -- start start 2023-10-16T14:37:57.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c931780 0x556e8c942210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c935590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c704920 con 0x556e8c930e20 2023-10-16T14:37:57.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c9324d0 con 0x556e8c931780 2023-10-16T14:37:57.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.103+0000 7f749c04fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c93a870 con 0x556e8c932620 2023-10-16T14:37:57.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.106+0000 7f748659b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:57.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.106+0000 7f748659b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41748/0 (socket says 172.21.15.178:41748) 2023-10-16T14:37:57.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.106+0000 7f748659b700 1 -- 172.21.15.178:0/782658305 learned_addr learned my addr 172.21.15.178:0/782658305 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:57.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.108+0000 7f748659b700 1 -- 172.21.15.178:0/782658305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 msgr2=0x556e8c935590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:37:57.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.108+0000 7f748659b700 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c935590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.108+0000 7f748659b700 1 -- 172.21.15.178:0/782658305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c931780 msgr2=0x556e8c942210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:37:57.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.108+0000 7f748659b700 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c931780 0x556e8c942210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.108+0000 7f748659b700 1 -- 172.21.15.178:0/782658305 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e8c928580 con 0x556e8c930e20 2023-10-16T14:37:57.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.109+0000 7f748659b700 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f747800e680 tx=0x7f7478007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=639ca733f4d99f74 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:57.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.140+0000 7f74757fa700 1 -- 172.21.15.178:0/782658305 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780243a0 con 0x556e8c930e20 2023-10-16T14:37:57.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.140+0000 7f74757fa700 1 -- 172.21.15.178:0/782658305 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7478024500 con 0x556e8c930e20 2023-10-16T14:37:57.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.140+0000 7f74757fa700 1 -- 172.21.15.178:0/782658305 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747800ebf0 con 0x556e8c930e20 2023-10-16T14:37:57.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.163+0000 7f749c04fb80 1 -- 172.21.15.178:0/782658305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 msgr2=0x556e8c939950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:57.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.163+0000 7f749c04fb80 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f747800e680 tx=0x7f7478007410 comp rx=0 tx=0).stop 2023-10-16T14:37:57.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.198+0000 7f749c04fb80 1 -- 172.21.15.178:0/782658305 shutdown_connections 2023-10-16T14:37:57.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.198+0000 7f749c04fb80 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c932620 0x556e8c935590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.198+0000 7f749c04fb80 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c930e20 0x556e8c939950 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.199+0000 7f749c04fb80 1 --2- 172.21.15.178:0/782658305 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c931780 0x556e8c942210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.199+0000 7f749c04fb80 1 -- 172.21.15.178:0/782658305 >> 172.21.15.178:0/782658305 conn(0x556e8c85edd0 msgr2=0x556e8c938a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:57.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.254+0000 7f749c04fb80 1 -- 172.21.15.178:0/782658305 shutdown_connections 2023-10-16T14:37:57.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.281+0000 7f749c04fb80 1 -- 172.21.15.178:0/782658305 wait complete. 2023-10-16T14:37:57.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.283+0000 7f749c04fb80 1 -- start start 2023-10-16T14:37:57.286 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.283+0000 7f749c04fb80 1 Processor -- start 2023-10-16T14:37:57.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c93f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c940f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c941440 con 0x556e8c931780 2023-10-16T14:37:57.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c9415b0 con 0x556e8c932620 2023-10-16T14:37:57.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f749c04fb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e8c941770 con 0x556e8c930e20 2023-10-16T14:37:57.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.287+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:57.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.288+0000 7f7485d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41752/0 (socket says 172.21.15.178:41752) 2023-10-16T14:37:57.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.288+0000 7f7485d9a700 1 -- 172.21.15.178:0/299937063 learned_addr learned my addr 172.21.15.178:0/299937063 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:37:57.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f749c04fb80 10 client.0 _ll_register_callbacks cb 0x7f747800f4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:37:57.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f7485d9a700 1 -- 172.21.15.178:0/299937063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 msgr2=0x556e8c93f9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:37:57.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f7485d9a700 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c93f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f7485d9a700 1 -- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 msgr2=0x556e8c940f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:37:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f7485d9a700 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c940f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.289+0000 7f7485d9a700 1 -- 172.21.15.178:0/299937063 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e8c6aede0 con 0x556e8c931780 2023-10-16T14:37:57.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.303+0000 7f7485d9a700 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f747001ac00 tx=0x7f747001af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee8a094796cd6f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:37:57.309 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.306+0000 7f7469ffb700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:37:57.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.306+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747002f070 con 0x556e8c931780 2023-10-16T14:37:57.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.306+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7470005410 con 0x556e8c931780 2023-10-16T14:37:57.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.307+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7470005220 con 0x556e8c931780 2023-10-16T14:37:57.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.333+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747002d040 con 0x556e8c931780 2023-10-16T14:37:57.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.347+0000 7f749c04fb80 10 client.16047 fetch_fsmap learned FSMap version 31 2023-10-16T14:37:57.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.347+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556e8c697c60 con 0x556e8c931780 2023-10-16T14:37:57.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.351+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7470005220 con 0x556e8c931780 2023-10-16T14:37:57.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.351+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f747003b070 con 0x556e8c931780 2023-10-16T14:37:57.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 10 client.16047 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:37:57.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:37:57.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 20 client.16047 populate_metadata read hostname 'smithi178' 2023-10-16T14:37:57.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c99af00 0x556e8c99d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:37:57.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 4 client.16047 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:37:57.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.357+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556e8c87ce50 con 0x556e8c99af00 2023-10-16T14:37:57.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.359+0000 7f748659b700 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c99af00 0x556e8c99d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:37:57.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.361+0000 7f748659b700 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c99af00 0x556e8c99d320 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=335c49151213970d server_cookie=3616c0e9c74c3c6c in_seq=0 out_seq=0 2023-10-16T14:37:57.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.361+0000 7f7469ffb700 10 client.16047 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:57.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.366+0000 7f7469ffb700 1 -- 172.21.15.178:0/299937063 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556e8c87ce50 con 0x556e8c99af00 2023-10-16T14:37:57.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.366+0000 7f7469ffb700 10 client.16047 handle_command_reply: tid=0 2023-10-16T14:37:57.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.366+0000 7f749c04fb80 1 client.16047 shutdown 2023-10-16T14:37:57.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.371+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 msgr2=0x556e8c93fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:57.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.371+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f747001ac00 tx=0x7f747001af10 comp rx=0 tx=0).stop 2023-10-16T14:37:57.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 shutdown_connections 2023-10-16T14:37:57.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x556e8c99af00 0x556e8c99d320 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x556e8c932620 0x556e8c940f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e8c930e20 0x556e8c93f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 --2- 172.21.15.178:0/299937063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x556e8c931780 0x556e8c93fef0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 >> 172.21.15.178:0/299937063 conn(0x556e8c85edd0 msgr2=0x556e8c939950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:57.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f7469ffb700 0 client.16047 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:37:57.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.407+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 shutdown_connections 2023-10-16T14:37:57.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.413+0000 7f749c04fb80 1 -- 172.21.15.178:0/299937063 wait complete. 2023-10-16T14:37:57.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.413+0000 7f749c04fb80 20 client.16047 trim_cache size 0 max 16384 2023-10-16T14:37:57.417 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:37:57.417 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:37:57.417 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:37:57.418 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:37:57.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.416+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 msgr2=0x7f746c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:57.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.416+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 0x7f746c0a3b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7480072160 tx=0x7f74780073a0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.416+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 msgr2=0x7f74801a48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:37:57.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.416+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f747c00b4a0 tx=0x7f747c00b860 comp rx=0 tx=0).stop 2023-10-16T14:37:57.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 shutdown_connections 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f746c0a1680 0x7f746c0a3b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f748010d170 0x7f74801a48a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7480107c90 0x7f748019d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 --2- 172.21.15.178:0/3483932409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f748010a690 0x7f748019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 >> 172.21.15.178:0/3483932409 conn(0x7f748006dd20 msgr2=0x7f748010b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:37:57.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 shutdown_connections 2023-10-16T14:37:57.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:37:57.417+0000 7f74887ff700 1 -- 172.21.15.178:0/3483932409 wait complete. 2023-10-16T14:37:57.750 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:37:57.750 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:37:58.058 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:57 smithi178 ceph-mon[98495]: pgmap v466: 129 pgs: 129 active+clean; 802 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2023-10-16T14:37:58.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:57 smithi186 ceph-mon[134892]: pgmap v466: 129 pgs: 129 active+clean; 802 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2023-10-16T14:37:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:57 smithi187 ceph-mon[135297]: pgmap v466: 129 pgs: 129 active+clean; 802 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2023-10-16T14:38:00.056 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:37:59 smithi178 ceph-mon[98495]: pgmap v467: 129 pgs: 129 active+clean; 808 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-10-16T14:38:00.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:37:59 smithi186 ceph-mon[134892]: pgmap v467: 129 pgs: 129 active+clean; 808 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-10-16T14:38:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:37:59 smithi187 ceph-mon[135297]: pgmap v467: 129 pgs: 129 active+clean; 808 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 279 op/s 2023-10-16T14:38:01.025 INFO:tasks.workunit.client.0.smithi178.stdout: 220 732874 1042 518804 1107 340900 3068 2023-10-16T14:38:01.575 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:01 smithi186 ceph-mon[134892]: pgmap v468: 129 pgs: 129 active+clean; 809 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2023-10-16T14:38:01.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:01 smithi187 ceph-mon[135297]: pgmap v468: 129 pgs: 129 active+clean; 809 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2023-10-16T14:38:01.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:01 smithi178 ceph-mon[98495]: pgmap v468: 129 pgs: 129 active+clean; 809 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 259 op/s 2023-10-16T14:38:02.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.639+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1836633125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc072cd0 msgr2=0x7f97bc109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:02.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.639+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1836633125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc072cd0 0x7f97bc109ec0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f97b00098f0 tx=0x7f97b0009c00 comp rx=0 tx=0).stop 2023-10-16T14:38:02.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.660+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1836633125 shutdown_connections 2023-10-16T14:38:02.665 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.660+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1836633125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10cf90 0x7f97bc117df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:02.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.660+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1836633125 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc072cd0 0x7f97bc109ec0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:02.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.660+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1836633125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc10a490 0x7f97bc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:02.666 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.660+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1836633125 >> 172.21.15.178:0/1836633125 conn(0x7f97bc06da50 msgr2=0x7f97bc06fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:02.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.695+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1836633125 shutdown_connections 2023-10-16T14:38:02.702 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.696+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1836633125 wait complete. 2023-10-16T14:38:02.703 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.700+0000 7f97c0b9d700 1 Processor -- start 2023-10-16T14:38:02.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.715+0000 7f97c0b9d700 1 -- start start 2023-10-16T14:38:02.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc072cd0 0x7f97bc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:02.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 0x7f97bc1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:02.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 0x7f97bc1a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:02.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc1a4bf0 con 0x7f97bc072cd0 2023-10-16T14:38:02.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc1a4d60 con 0x7f97bc10cf90 2023-10-16T14:38:02.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.716+0000 7f97c0b9d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc1a4f20 con 0x7f97bc10a490 2023-10-16T14:38:02.726 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.717+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 0x7f97bc1a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.717+0000 7f97bbfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 0x7f97bc1a4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:35374/0 (socket says 172.21.15.178:35374) 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.717+0000 7f97bbfff700 1 -- 172.21.15.178:0/1842620850 learned_addr learned my addr 172.21.15.178:0/1842620850 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.717+0000 7f97baffd700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 0x7f97bc1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.718+0000 7f97baffd700 1 -- 172.21.15.178:0/1842620850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 msgr2=0x7f97bc1a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.718+0000 7f97baffd700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 0x7f97bc1a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:02.727 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.718+0000 7f97baffd700 1 -- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc072cd0 msgr2=0x7f97bc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:02.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.718+0000 7f97baffd700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc072cd0 0x7f97bc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:02.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.718+0000 7f97baffd700 1 -- 172.21.15.178:0/1842620850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b0009520 con 0x7f97bc10a490 2023-10-16T14:38:02.728 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.719+0000 7f97baffd700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 0x7f97bc1a1d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97b400bdf0 tx=0x7f97b4009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.749+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400cc90 con 0x7f97bc10a490 2023-10-16T14:38:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.749+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b400cdf0 con 0x7f97bc10a490 2023-10-16T14:38:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.749+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40127e0 con 0x7f97bc10a490 2023-10-16T14:38:02.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.751+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc1a5200 con 0x7f97bc10a490 2023-10-16T14:38:02.775 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.752+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc1a5740 con 0x7f97bc10a490 2023-10-16T14:38:02.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.827+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f97b4012940 con 0x7f97bc10a490 2023-10-16T14:38:02.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.835+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc06f5d0 con 0x7f97bc10a490 2023-10-16T14:38:02.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.873+0000 7f97b8ff9700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 0x7f97a40a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:02.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.873+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f97b40cce60 con 0x7f97bc10a490 2023-10-16T14:38:02.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.880+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f97b4014660 con 0x7f97bc10a490 2023-10-16T14:38:02.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.884+0000 7f97bb7fe700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 0x7f97a40a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:02.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:02.885+0000 7f97bb7fe700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 0x7f97a40a3c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009810 tx=0x7f97b0005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:04.059 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:03 smithi178 ceph-mon[98495]: pgmap v469: 129 pgs: 129 active+clean; 813 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 172 op/s 2023-10-16T14:38:04.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:03 smithi186 ceph-mon[134892]: pgmap v469: 129 pgs: 129 active+clean; 813 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 172 op/s 2023-10-16T14:38:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:03 smithi187 ceph-mon[135297]: pgmap v469: 129 pgs: 129 active+clean; 813 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 172 op/s 2023-10-16T14:38:06.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:05 smithi186 ceph-mon[134892]: pgmap v470: 129 pgs: 129 active+clean; 827 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 201 op/s 2023-10-16T14:38:06.090 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:05 smithi178 ceph-mon[98495]: pgmap v470: 129 pgs: 129 active+clean; 827 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 201 op/s 2023-10-16T14:38:06.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:05 smithi187 ceph-mon[135297]: pgmap v470: 129 pgs: 129 active+clean; 827 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 201 op/s 2023-10-16T14:38:08.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:07 smithi178 ceph-mon[98495]: pgmap v471: 129 pgs: 129 active+clean; 836 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-10-16T14:38:08.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:07 smithi186 ceph-mon[134892]: pgmap v471: 129 pgs: 129 active+clean; 836 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-10-16T14:38:08.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:07 smithi187 ceph-mon[135297]: pgmap v471: 129 pgs: 129 active+clean; 836 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2023-10-16T14:38:10.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:09 smithi186 ceph-mon[134892]: pgmap v472: 129 pgs: 129 active+clean; 847 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 328 op/s 2023-10-16T14:38:10.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:09 smithi187 ceph-mon[135297]: pgmap v472: 129 pgs: 129 active+clean; 847 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 328 op/s 2023-10-16T14:38:10.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:09 smithi178 ceph-mon[98495]: pgmap v472: 129 pgs: 129 active+clean; 847 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.0 MiB/s wr, 328 op/s 2023-10-16T14:38:11.031 INFO:tasks.workunit.client.0.smithi178.stdout: 236 606045 772 429671 686 285219 1924 2023-10-16T14:38:11.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:11 smithi187 ceph-mon[135297]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 488 op/s 2023-10-16T14:38:11.808 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:11 smithi178 ceph-mon[98495]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 488 op/s 2023-10-16T14:38:11.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:11 smithi186 ceph-mon[134892]: pgmap v473: 129 pgs: 129 active+clean; 875 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.2 MiB/s wr, 488 op/s 2023-10-16T14:38:12.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.830+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9798005180 con 0x7f97bc10a490 2023-10-16T14:38:12.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.832+0000 7f97b8ff9700 1 -- 172.21.15.178:0/1842620850 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+11343 (secure 0 0 0) 0x7f97b4019090 con 0x7f97bc10a490 2023-10-16T14:38:12.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:38:12.843 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:37:42.452715+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:38:12.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.863+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 msgr2=0x7f97a40a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:12.868 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.863+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 0x7f97a40a3c20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009810 tx=0x7f97b0005d20 comp rx=0 tx=0).stop 2023-10-16T14:38:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.863+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 msgr2=0x7f97bc1a1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:12.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.863+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 0x7f97bc1a1d40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f97b400bdf0 tx=0x7f97b4009f70 comp rx=0 tx=0).stop 2023-10-16T14:38:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 shutdown_connections 2023-10-16T14:38:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f97a40a1760 0x7f97a40a3c20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f97bc10cf90 0x7f97bc1a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:12.871 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97bc072cd0 0x7f97bc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 --2- 172.21.15.178:0/1842620850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97bc10a490 0x7f97bc1a1d40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:12.872 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.865+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 >> 172.21.15.178:0/1842620850 conn(0x7f97bc06da50 msgr2=0x7f97bc1087f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:12.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.881+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 shutdown_connections 2023-10-16T14:38:12.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:12.883+0000 7f97c0b9d700 1 -- 172.21.15.178:0/1842620850 wait complete. 2023-10-16T14:38:12.900 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 31 2023-10-16T14:38:13.054 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:38:13.800 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:13 smithi178 ceph-mon[98495]: pgmap v474: 129 pgs: 129 active+clean; 880 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 466 op/s 2023-10-16T14:38:13.800 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:13 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1842620850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:38:13.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.932+0000 7f8633dff700 1 -- 172.21.15.178:0/2865733518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c107eb0 msgr2=0x7f862c10a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:13.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.932+0000 7f8633dff700 1 --2- 172.21.15.178:0/2865733518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c107eb0 0x7f862c10a2e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f861c009990 tx=0x7f861c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.933+0000 7f8633dff700 1 -- 172.21.15.178:0/2865733518 shutdown_connections 2023-10-16T14:38:13.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.933+0000 7f8633dff700 1 --2- 172.21.15.178:0/2865733518 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10d1f0 0x7f862c10d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.933+0000 7f8633dff700 1 --2- 172.21.15.178:0/2865733518 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c107eb0 0x7f862c10a2e0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f861c009990 tx=0x7f861c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.933+0000 7f8633dff700 1 --2- 172.21.15.178:0/2865733518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10a820 0x7f862c10ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.933+0000 7f8633dff700 1 -- 172.21.15.178:0/2865733518 >> 172.21.15.178:0/2865733518 conn(0x7f862c06dd30 msgr2=0x7f862c0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:13.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.935+0000 7f8633dff700 1 -- 172.21.15.178:0/2865733518 shutdown_connections 2023-10-16T14:38:13.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.935+0000 7f8633dff700 1 -- 172.21.15.178:0/2865733518 wait complete. 2023-10-16T14:38:13.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.945+0000 7f8633dff700 1 Processor -- start 2023-10-16T14:38:13.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.947+0000 7f8633dff700 1 -- start start 2023-10-16T14:38:13.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.947+0000 7f8633dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10a820 0x7f862c1a5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.948+0000 7f8633dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.948+0000 7f8633dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c1ab210 0x7f862c1ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.949+0000 7f8633dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c1adbd0 con 0x7f862c10a820 2023-10-16T14:38:13.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.949+0000 7f8633dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c10f610 con 0x7f862c1ab210 2023-10-16T14:38:13.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f8633dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c10f7d0 con 0x7f862c10d1f0 2023-10-16T14:38:13.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:13.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:34958/0 (socket says 172.21.15.178:34958) 2023-10-16T14:38:13.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 -- 172.21.15.178:0/1845582287 learned_addr learned my addr 172.21.15.178:0/1845582287 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:13.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f8631b9b700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10a820 0x7f862c1a5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:13.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 -- 172.21.15.178:0/1845582287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c1ab210 msgr2=0x7f862c1ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:13.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c1ab210 0x7f862c1ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 -- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10a820 msgr2=0x7f862c1a5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:13.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10a820 0x7f862c1a5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.950+0000 7f863139a700 1 -- 172.21.15.178:0/1845582287 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0095f0 con 0x7f862c10d1f0 2023-10-16T14:38:13.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.955+0000 7f863139a700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f862800d980 tx=0x7f862800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:13.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.956+0000 7f8622ffd700 1 -- 172.21.15.178:0/1845582287 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628004020 con 0x7f862c10d1f0 2023-10-16T14:38:13.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.957+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c10fab0 con 0x7f862c10d1f0 2023-10-16T14:38:13.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.957+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c10ffe0 con 0x7f862c10d1f0 2023-10-16T14:38:13.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.957+0000 7f8622ffd700 1 -- 172.21.15.178:0/1845582287 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8628009730 con 0x7f862c10d1f0 2023-10-16T14:38:13.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.957+0000 7f8622ffd700 1 -- 172.21.15.178:0/1845582287 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8628004020 con 0x7f862c10d1f0 2023-10-16T14:38:13.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.965+0000 7f8622ffd700 1 -- 172.21.15.178:0/1845582287 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8628004180 con 0x7f862c10d1f0 2023-10-16T14:38:13.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.973+0000 7f8622ffd700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 0x7f86180a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.974+0000 7f8622ffd700 1 -- 172.21.15.178:0/1845582287 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f86280cc850 con 0x7f862c10d1f0 2023-10-16T14:38:13.983 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.980+0000 7f8631b9b700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 0x7f86180a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:13.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.981+0000 7f8647786b80 1 Processor -- start 2023-10-16T14:38:13.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.985+0000 7f8647786b80 1 -- start start 2023-10-16T14:38:13.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.986+0000 7f8647786b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8647786b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d7e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8647786b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7e620 0x55d8e5d7eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:13.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8647786b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5cf1c70 con 0x55d8e5d7ce20 2023-10-16T14:38:13.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8647786b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5cd6c60 con 0x55d8e5d7d780 2023-10-16T14:38:13.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8647786b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5cd4580 con 0x55d8e5d7e620 2023-10-16T14:38:13.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:13.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35206/0 (socket says 172.21.15.178:35206) 2023-10-16T14:38:13.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.987+0000 7f8631b9b700 1 -- 172.21.15.178:0/848840423 learned_addr learned my addr 172.21.15.178:0/848840423 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:13.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 -- 172.21.15.178:0/848840423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7e620 msgr2=0x55d8e5d7eaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:38:13.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7e620 0x55d8e5d7eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 -- 172.21.15.178:0/848840423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 msgr2=0x55d8e5d7e040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:38:13.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d7e040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 -- 172.21.15.178:0/848840423 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e5c8d3f0 con 0x55d8e5d7ce20 2023-10-16T14:38:13.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.988+0000 7f8631b9b700 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c010 tx=0x7f861c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63424d7b1ed2049d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:13.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.991+0000 7f8617fff700 1 -- 172.21.15.178:0/848840423 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c003e70 con 0x55d8e5d7ce20 2023-10-16T14:38:13.995 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.991+0000 7f8617fff700 1 -- 172.21.15.178:0/848840423 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861c01e040 con 0x55d8e5d7ce20 2023-10-16T14:38:13.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.992+0000 7f8647786b80 1 -- 172.21.15.178:0/848840423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 msgr2=0x55d8e5d7d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:13.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.992+0000 7f8647786b80 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c010 tx=0x7f861c009310 comp rx=0 tx=0).stop 2023-10-16T14:38:13.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.993+0000 7f8631b9b700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 0x7f86180a3a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098e0 tx=0x7f861c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:13.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.994+0000 7f8647786b80 1 -- 172.21.15.178:0/848840423 shutdown_connections 2023-10-16T14:38:13.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.994+0000 7f8647786b80 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7e620 0x55d8e5d7eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.994+0000 7f8647786b80 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7d240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.994+0000 7f8647786b80 1 --2- 172.21.15.178:0/848840423 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d7e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:13.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.994+0000 7f8647786b80 1 -- 172.21.15.178:0/848840423 >> 172.21.15.178:0/848840423 conn(0x55d8e5caadd0 msgr2=0x55d8e5d77fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:13.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.995+0000 7f8647786b80 1 -- 172.21.15.178:0/848840423 shutdown_connections 2023-10-16T14:38:13.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.995+0000 7f8647786b80 1 -- 172.21.15.178:0/848840423 wait complete. 2023-10-16T14:38:14.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.995+0000 7f8647786b80 1 -- start start 2023-10-16T14:38:14.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.997+0000 7f8647786b80 1 Processor -- start 2023-10-16T14:38:14.002 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:13.998+0000 7f8647786b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d86c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.006 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5ccfe70 con 0x55d8e5d7e620 2023-10-16T14:38:14.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5c6c710 con 0x55d8e5d7d780 2023-10-16T14:38:14.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5c6c3b0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f8647786b80 10 client.0 _ll_register_callbacks cb 0x55d8e5ac9800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:14.007 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.002+0000 7f863239c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.005+0000 7f863239c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35220/0 (socket says 172.21.15.178:35220) 2023-10-16T14:38:14.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.005+0000 7f863239c700 1 -- 172.21.15.178:0/981251644 learned_addr learned my addr 172.21.15.178:0/981251644 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.005+0000 7f8631b9b700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d86c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.006+0000 7f8631b9b700 1 -- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 msgr2=0x55d8e5d871a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:14.010 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.006+0000 7f8631b9b700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d871a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.006+0000 7f8631b9b700 1 -- 172.21.15.178:0/981251644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 msgr2=0x55d8e5d8e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.006+0000 7f8631b9b700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.007+0000 7f8631b9b700 1 -- 172.21.15.178:0/981251644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c10e240 con 0x55d8e5d7ce20 2023-10-16T14:38:14.017 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.013+0000 7f8631b9b700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d86c60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b5c0 tx=0x7f861c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2763abbf63ee8c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.013+0000 7f86157fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:38:14.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.013+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c005610 con 0x55d8e5d7ce20 2023-10-16T14:38:14.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.014+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f861c02f470 con 0x55d8e5d7ce20 2023-10-16T14:38:14.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.014+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8e5cd4ef0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.014+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f861c02f630 con 0x55d8e5d7ce20 2023-10-16T14:38:14.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.014+0000 7f8647786b80 10 client.26104 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:14.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.015+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d8e5afade0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8e5cd4ef0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f861c0438c0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f8647786b80 10 client.26104 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:14.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f8647786b80 10 client.26104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:38:14.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f8647786b80 20 client.26104 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:14.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.017+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5d92550 0x55d8e5d94970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.018+0000 7f863139a700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5d92550 0x55d8e5d94970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.018+0000 7f8647786b80 4 client.26104 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:38:14.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.018+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8e5d4cc10 con 0x55d8e5d92550 2023-10-16T14:38:14.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.019+0000 7f863139a700 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5d92550 0x55d8e5d94970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bde4cf86fa58e1ff server_cookie=e2ef967ab345e9d3 in_seq=0 out_seq=0 2023-10-16T14:38:14.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.019+0000 7f86157fa700 10 client.26104 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:38:14.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.022+0000 7f86157fa700 1 -- 172.21.15.178:0/981251644 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d8e5d4cc10 con 0x55d8e5d92550 2023-10-16T14:38:14.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.022+0000 7f86157fa700 10 client.26104 handle_command_reply: tid=0 2023-10-16T14:38:14.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.023+0000 7f8647786b80 1 client.26104 shutdown 2023-10-16T14:38:14.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.027+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 msgr2=0x55d8e5d86c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.027+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d86c60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f861c00b5c0 tx=0x7f861c00f690 comp rx=0 tx=0).stop 2023-10-16T14:38:14.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 shutdown_connections 2023-10-16T14:38:14.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5d92550 0x55d8e5d94970 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d86c60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 --2- 172.21.15.178:0/981251644 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d871a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 >> 172.21.15.178:0/981251644 conn(0x55d8e5caadd0 msgr2=0x55d8e5d77bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.028+0000 7f86157fa700 0 client.26104 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:38:14.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.029+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 shutdown_connections 2023-10-16T14:38:14.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.029+0000 7f8647786b80 1 -- 172.21.15.178:0/981251644 wait complete. 2023-10-16T14:38:14.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.035+0000 7f8647786b80 20 client.26104 trim_cache size 0 max 16384 2023-10-16T14:38:14.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:13 smithi186 ceph-mon[134892]: pgmap v474: 129 pgs: 129 active+clean; 880 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 466 op/s 2023-10-16T14:38:14.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:13 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1842620850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:38:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.091+0000 7f8647786b80 1 Processor -- start 2023-10-16T14:38:14.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.091+0000 7f8647786b80 1 -- start start 2023-10-16T14:38:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.092+0000 7f8647786b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.092+0000 7f8647786b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.093+0000 7f8647786b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f8647786b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5d86830 con 0x55d8e5d7ce20 2023-10-16T14:38:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f8647786b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5b50920 con 0x55d8e5d7e620 2023-10-16T14:38:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f8647786b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5d72c00 con 0x55d8e5d7d780 2023-10-16T14:38:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f863239c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f863239c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:41148/0 (socket says 172.21.15.178:41148) 2023-10-16T14:38:14.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.094+0000 7f863239c700 1 -- 172.21.15.178:0/1757883187 learned_addr learned my addr 172.21.15.178:0/1757883187 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.095+0000 7f863239c700 1 -- 172.21.15.178:0/1757883187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7d780 msgr2=0x55d8e5d8e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.095+0000 7f863239c700 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.095+0000 7f863239c700 1 -- 172.21.15.178:0/1757883187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 msgr2=0x55d8e5d7ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.095+0000 7f863239c700 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.095+0000 7f863239c700 1 -- 172.21.15.178:0/1757883187 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8e5ae3c60 con 0x55d8e5d7e620 2023-10-16T14:38:14.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.096+0000 7f863239c700 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8624007ee0 tx=0x7f862400c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffee8f028e27b887 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.100+0000 7f8620bff700 1 -- 172.21.15.178:0/1757883187 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8624011040 con 0x55d8e5d7e620 2023-10-16T14:38:14.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.100+0000 7f8620bff700 1 -- 172.21.15.178:0/1757883187 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f862400ae70 con 0x55d8e5d7e620 2023-10-16T14:38:14.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.100+0000 7f8620bff700 1 -- 172.21.15.178:0/1757883187 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86240087f0 con 0x55d8e5d7e620 2023-10-16T14:38:14.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.101+0000 7f8647786b80 1 -- 172.21.15.178:0/1757883187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 msgr2=0x55d8e5d94560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.101+0000 7f8647786b80 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8624007ee0 tx=0x7f862400c040 comp rx=0 tx=0).stop 2023-10-16T14:38:14.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.104+0000 7f8647786b80 1 -- 172.21.15.178:0/1757883187 shutdown_connections 2023-10-16T14:38:14.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.104+0000 7f8647786b80 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7e620 0x55d8e5d94560 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.104+0000 7f8647786b80 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d7ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.104+0000 7f8647786b80 1 --2- 172.21.15.178:0/1757883187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.104+0000 7f8647786b80 1 -- 172.21.15.178:0/1757883187 >> 172.21.15.178:0/1757883187 conn(0x55d8e5caadd0 msgr2=0x55d8e5d77240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.105+0000 7f8647786b80 1 -- 172.21.15.178:0/1757883187 shutdown_connections 2023-10-16T14:38:14.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.106+0000 7f8647786b80 1 -- 172.21.15.178:0/1757883187 wait complete. 2023-10-16T14:38:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.107+0000 7f8647786b80 1 -- start start 2023-10-16T14:38:14.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.108+0000 7f8647786b80 1 Processor -- start 2023-10-16T14:38:14.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.112+0000 7f8647786b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.112+0000 7f8647786b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.112+0000 7f8647786b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.118 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8647786b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5d8d860 con 0x55d8e5d7e620 2023-10-16T14:38:14.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8647786b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5d8d9d0 con 0x55d8e5d7d780 2023-10-16T14:38:14.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8647786b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8e5d8db90 con 0x55d8e5d7ce20 2023-10-16T14:38:14.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8631b9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:34984/0 (socket says 172.21.15.178:34984) 2023-10-16T14:38:14.119 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8631b9b700 1 -- 172.21.15.178:0/796394434 learned_addr learned my addr 172.21.15.178:0/796394434 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.114+0000 7f8647786b80 10 client.0 _ll_register_callbacks cb 0x7f861c00b850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:14.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f863139a700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.120 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f863239c700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f8631b9b700 1 -- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 msgr2=0x55d8e5d8c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f8631b9b700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f8631b9b700 1 -- 172.21.15.178:0/796394434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 msgr2=0x55d8e5d8d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f8631b9b700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.115+0000 7f8631b9b700 1 -- 172.21.15.178:0/796394434 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8624005e70 con 0x55d8e5d7ce20 2023-10-16T14:38:14.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.116+0000 7f863239c700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:38:14.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.118+0000 7f8631b9b700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c010 tx=0x7f861c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9685ab9fe978e231 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.118+0000 7f86157fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:38:14.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.119+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c043ad0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.119+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f861c000e40 con 0x55d8e5d7ce20 2023-10-16T14:38:14.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.121+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f861c01e040 con 0x55d8e5d7ce20 2023-10-16T14:38:14.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.123+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f861c043da0 con 0x55d8e5d7ce20 2023-10-16T14:38:14.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.123+0000 7f8647786b80 10 client.26107 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:14.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.123+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d8e5b21250 con 0x55d8e5d7ce20 2023-10-16T14:38:14.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.124+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f861c01e040 con 0x55d8e5d7ce20 2023-10-16T14:38:14.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.125+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f861c026b60 con 0x55d8e5d7ce20 2023-10-16T14:38:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 10 client.26107 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 10 client.26107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:38:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 20 client.26107 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5de6c60 0x55d8e5de9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 4 client.26107 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:38:14.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.128+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8e5cc8e50 con 0x55d8e5de6c60 2023-10-16T14:38:14.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.129+0000 7f863139a700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5de6c60 0x55d8e5de9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.131+0000 7f863139a700 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5de6c60 0x55d8e5de9080 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be3e6a04ee4e04c7 server_cookie=572de85a312cd9af in_seq=0 out_seq=0 2023-10-16T14:38:14.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.131+0000 7f86157fa700 10 client.26107 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:38:14.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.133+0000 7f86157fa700 1 -- 172.21.15.178:0/796394434 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8e5cc8e50 con 0x55d8e5de6c60 2023-10-16T14:38:14.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.133+0000 7f86157fa700 10 client.26107 handle_command_reply: tid=0 2023-10-16T14:38:14.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.133+0000 7f8647786b80 1 client.26107 shutdown 2023-10-16T14:38:14.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.133+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 msgr2=0x55d8e5d8bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.133+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f861c00c010 tx=0x7f861c005050 comp rx=0 tx=0).stop 2023-10-16T14:38:14.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 shutdown_connections 2023-10-16T14:38:14.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55d8e5de6c60 0x55d8e5de9080 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55d8e5d7e620 0x55d8e5d8d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d8e5d7ce20 0x55d8e5d8bdd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 --2- 172.21.15.178:0/796394434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55d8e5d7d780 0x55d8e5d8c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.134+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 >> 172.21.15.178:0/796394434 conn(0x55d8e5caadd0 msgr2=0x55d8e5d94560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.136+0000 7f86157fa700 0 client.26107 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:38:14.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.136+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 shutdown_connections 2023-10-16T14:38:14.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.136+0000 7f8647786b80 1 -- 172.21.15.178:0/796394434 wait complete. 2023-10-16T14:38:14.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.136+0000 7f8647786b80 20 client.26107 trim_cache size 0 max 16384 2023-10-16T14:38:14.141 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:38:14.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.138+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 msgr2=0x7f86180a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 0x7f86180a3a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f861c0098e0 tx=0x7f861c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 msgr2=0x7f862c1a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f862800d980 tx=0x7f862800dc90 comp rx=0 tx=0).stop 2023-10-16T14:38:14.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 shutdown_connections 2023-10-16T14:38:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f86180a1580 0x7f86180a3a40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f862c1ab210 0x7f862c1ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f862c10a820 0x7f862c1a5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 --2- 172.21.15.178:0/1845582287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f862c10d1f0 0x7f862c1a61c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 >> 172.21.15.178:0/1845582287 conn(0x7f862c06dd30 msgr2=0x7f862c10c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 shutdown_connections 2023-10-16T14:38:14.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.139+0000 7f8633dff700 1 -- 172.21.15.178:0/1845582287 wait complete. 2023-10-16T14:38:14.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:13 smithi187 ceph-mon[135297]: pgmap v474: 129 pgs: 129 active+clean; 880 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.2 MiB/s wr, 466 op/s 2023-10-16T14:38:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:13 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1842620850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:38:14.203 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:38:14.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 -- 172.21.15.178:0/284174147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 msgr2=0x7f8f60073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 --2- 172.21.15.178:0/284174147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f60073d90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009990 tx=0x7f8f50009ca0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 -- 172.21.15.178:0/284174147 shutdown_connections 2023-10-16T14:38:14.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 --2- 172.21.15.178:0/284174147 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 0x7f8f6010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 --2- 172.21.15.178:0/284174147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 0x7f8f60073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 --2- 172.21.15.178:0/284174147 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f60073d90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 -- 172.21.15.178:0/284174147 >> 172.21.15.178:0/284174147 conn(0x7f8f600fc400 msgr2=0x7f8f600fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 -- 172.21.15.178:0/284174147 shutdown_connections 2023-10-16T14:38:14.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.466+0000 7f8f677ff700 1 -- 172.21.15.178:0/284174147 wait complete. 2023-10-16T14:38:14.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.467+0000 7f8f677ff700 1 Processor -- start 2023-10-16T14:38:14.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 -- start start 2023-10-16T14:38:14.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 0x7f8f6019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 0x7f8f601a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a4db0 con 0x7f8f60074ef0 2023-10-16T14:38:14.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a4ef0 con 0x7f8f6010d4a0 2023-10-16T14:38:14.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f677ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a5080 con 0x7f8f60073910 2023-10-16T14:38:14.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f64d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 0x7f8f6019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:35000/0 (socket says 172.21.15.178:35000) 2023-10-16T14:38:14.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f6559b700 1 -- 172.21.15.178:0/2464942757 learned_addr learned my addr 172.21.15.178:0/2464942757 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.468+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 0x7f8f601a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 -- 172.21.15.178:0/2464942757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 msgr2=0x7f8f601a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 0x7f8f601a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 -- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 msgr2=0x7f8f6019d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 0x7f8f6019d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 -- 172.21.15.178:0/2464942757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c009520 con 0x7f8f60073910 2023-10-16T14:38:14.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00eb30 tx=0x7f8f5c00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f567fc700 1 -- 172.21.15.178:0/2464942757 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c009880 con 0x7f8f60073910 2023-10-16T14:38:14.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f567fc700 1 -- 172.21.15.178:0/2464942757 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5c0044e0 con 0x7f8f60073910 2023-10-16T14:38:14.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f567fc700 1 -- 172.21.15.178:0/2464942757 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c003d80 con 0x7f8f60073910 2023-10-16T14:38:14.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.469+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f500095f0 con 0x7f8f60073910 2023-10-16T14:38:14.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.470+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601a5690 con 0x7f8f60073910 2023-10-16T14:38:14.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.476+0000 7f8f567fc700 1 -- 172.21.15.178:0/2464942757 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8f5c003f20 con 0x7f8f60073910 2023-10-16T14:38:14.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.478+0000 7f8f567fc700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 0x7f8f4c0a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.478+0000 7f8f567fc700 1 -- 172.21.15.178:0/2464942757 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8f5c014070 con 0x7f8f60073910 2023-10-16T14:38:14.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.479+0000 7f8f7b030b80 1 Processor -- start 2023-10-16T14:38:14.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.481+0000 7f8f7b030b80 1 -- start start 2023-10-16T14:38:14.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.482+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.482+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.482+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.483+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.483+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 0x7f8f4c0a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.483+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:41192/0 (socket says 172.21.15.178:41192) 2023-10-16T14:38:14.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.484+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 0x7f8f4c0a3a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000b2e0 tx=0x7f8f50005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.484+0000 7f8f64d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.484+0000 7f8f64d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:35010/0 (socket says 172.21.15.178:35010) 2023-10-16T14:38:14.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.485+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f633c70 con 0x55a51f6c0620 2023-10-16T14:38:14.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.485+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f618c60 con 0x55a51f6bee20 2023-10-16T14:38:14.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.485+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f616580 con 0x55a51f6bf780 2023-10-16T14:38:14.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.486+0000 7f8f65d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.486+0000 7f8f65d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35264/0 (socket says 172.21.15.178:35264) 2023-10-16T14:38:14.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.486+0000 7f8f65d9c700 1 -- 172.21.15.178:0/35884819 learned_addr learned my addr 172.21.15.178:0/35884819 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.487+0000 7f8f65d9c700 1 -- 172.21.15.178:0/35884819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 msgr2=0x55a51f6c0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.487+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.487+0000 7f8f65d9c700 1 -- 172.21.15.178:0/35884819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 msgr2=0x55a51f6bf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.487+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.487+0000 7f8f65d9c700 1 -- 172.21.15.178:0/35884819 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a51f5cf3f0 con 0x55a51f6c0620 2023-10-16T14:38:14.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.488+0000 7f8f6559b700 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-16T14:38:14.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.488+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-16T14:38:14.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.489+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f58011680 tx=0x7f8f58009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f738e1cf921b65c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.491+0000 7f8f4b7fe700 1 -- 172.21.15.178:0/35884819 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800e040 con 0x55a51f6c0620 2023-10-16T14:38:14.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.491+0000 7f8f4b7fe700 1 -- 172.21.15.178:0/35884819 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5800f040 con 0x55a51f6c0620 2023-10-16T14:38:14.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.491+0000 7f8f4b7fe700 1 -- 172.21.15.178:0/35884819 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580048c0 con 0x55a51f6c0620 2023-10-16T14:38:14.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.493+0000 7f8f7b030b80 1 -- 172.21.15.178:0/35884819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 msgr2=0x55a51f6c0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.493+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8f58011680 tx=0x7f8f58009920 comp rx=0 tx=0).stop 2023-10-16T14:38:14.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 -- 172.21.15.178:0/35884819 shutdown_connections 2023-10-16T14:38:14.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6c0aa0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6bf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/35884819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bf780 0x55a51f6c0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 -- 172.21.15.178:0/35884819 >> 172.21.15.178:0/35884819 conn(0x55a51f5ecdd0 msgr2=0x55a51f6b9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.500 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 -- 172.21.15.178:0/35884819 shutdown_connections 2023-10-16T14:38:14.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.495+0000 7f8f7b030b80 1 -- 172.21.15.178:0/35884819 wait complete. 2023-10-16T14:38:14.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.496+0000 7f8f7b030b80 1 -- start start 2023-10-16T14:38:14.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.500+0000 7f8f7b030b80 1 Processor -- start 2023-10-16T14:38:14.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6c0620 0x55a51f6d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f611e70 con 0x55a51f6bf780 2023-10-16T14:38:14.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f5ae710 con 0x55a51f6bee20 2023-10-16T14:38:14.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f5ae3b0 con 0x55a51f6c0620 2023-10-16T14:38:14.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.505+0000 7f8f7b030b80 10 client.0 _ll_register_callbacks cb 0x55a51f40b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:14.512 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.506+0000 7f8f64d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.506+0000 7f8f64d9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35266/0 (socket says 172.21.15.178:35266) 2023-10-16T14:38:14.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.506+0000 7f8f64d9a700 1 -- 172.21.15.178:0/2901032406 learned_addr learned my addr 172.21.15.178:0/2901032406 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.514 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.506+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.515 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.506+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6c0620 0x55a51f6d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.507+0000 7f8f64d9a700 1 -- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6c0620 msgr2=0x55a51f6d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.507+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6c0620 0x55a51f6d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.507+0000 7f8f64d9a700 1 -- 172.21.15.178:0/2901032406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 msgr2=0x55a51f6c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.507+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.517 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.507+0000 7f8f64d9a700 1 -- 172.21.15.178:0/2901032406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f58011330 con 0x55a51f6bf780 2023-10-16T14:38:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.508+0000 7f8f64d9a700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8f50015000 tx=0x7f8f50009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646668d56d40fbcc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.509+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:38:14.518 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.509+0000 7f8f48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:38:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.510+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5000f460 con 0x55a51f6bf780 2023-10-16T14:38:14.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.510+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8f5000f5c0 con 0x55a51f6bf780 2023-10-16T14:38:14.521 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.518+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a51f616ef0 con 0x55a51f6bf780 2023-10-16T14:38:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.518+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f5000f780 con 0x55a51f6bf780 2023-10-16T14:38:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.520+0000 7f8f7b030b80 10 client.16068 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:14.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.520+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a51f43cde0 con 0x55a51f6bf780 2023-10-16T14:38:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.523+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a51f616ef0 con 0x55a51f6bf780 2023-10-16T14:38:14.526 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.523+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8f500249f0 con 0x55a51f6bf780 2023-10-16T14:38:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 10 client.16068 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 10 client.16068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:38:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 20 client.16068 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:14.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f6e5a20 0x55a51f6e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 4 client.16068 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:38:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.527+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a51f68ec10 con 0x55a51f6e5a20 2023-10-16T14:38:14.533 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.529+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f6e5a20 0x55a51f6e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.544+0000 7f8f48ff9700 10 client.16068 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:38:14.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.544+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f6e5a20 0x55a51f6e7e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3640b366caa9465 server_cookie=681a1318ef510341 in_seq=0 out_seq=0 2023-10-16T14:38:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.544+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2901032406 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a51f68ec10 con 0x55a51f6e5a20 2023-10-16T14:38:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.544+0000 7f8f48ff9700 10 client.16068 handle_command_reply: tid=0 2023-10-16T14:38:14.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.544+0000 7f8f7b030b80 1 client.16068 shutdown 2023-10-16T14:38:14.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.545+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 msgr2=0x55a51f6c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.545+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8f50015000 tx=0x7f8f50009ca0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 shutdown_connections 2023-10-16T14:38:14.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f6e5a20 0x55a51f6e7e40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6c0620 0x55a51f6d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2901032406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6bf780 0x55a51f6c9190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 >> 172.21.15.178:0/2901032406 conn(0x55a51f5ecdd0 msgr2=0x55a51f6b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.546+0000 7f8f48ff9700 0 client.16068 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:38:14.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.548+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 shutdown_connections 2023-10-16T14:38:14.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.549+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2901032406 wait complete. 2023-10-16T14:38:14.593 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.585+0000 7f8f7b030b80 20 client.16068 trim_cache size 0 max 16384 2023-10-16T14:38:14.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.806+0000 7f8f7b030b80 1 Processor -- start 2023-10-16T14:38:14.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 -- start start 2023-10-16T14:38:14.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6b87a0 0x55a51f6e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f492920 con 0x55a51f6b87a0 2023-10-16T14:38:14.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.813+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f6c9e40 con 0x55a51f6c0620 2023-10-16T14:38:14.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.814+0000 7f8f7b030b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f6c04a0 con 0x55a51f6bee20 2023-10-16T14:38:14.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.814+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.814+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:35022/0 (socket says 172.21.15.178:35022) 2023-10-16T14:38:14.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.814+0000 7f8f6559b700 1 -- 172.21.15.178:0/980819220 learned_addr learned my addr 172.21.15.178:0/980819220 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.815+0000 7f8f6559b700 1 -- 172.21.15.178:0/980819220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6c0620 msgr2=0x55a51f6f5620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:38:14.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.815+0000 7f8f6559b700 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.815+0000 7f8f6559b700 1 -- 172.21.15.178:0/980819220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6b87a0 msgr2=0x55a51f6e7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.815+0000 7f8f6559b700 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6b87a0 0x55a51f6e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.823 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.815+0000 7f8f6559b700 1 -- 172.21.15.178:0/980819220 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a51f425c60 con 0x55a51f6bee20 2023-10-16T14:38:14.824 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.816+0000 7f8f6559b700 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c5b0 tx=0x7f8f5c003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96dacbb6917fac89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.823+0000 7f8f4bfff700 1 -- 172.21.15.178:0/980819220 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c09a4e0 con 0x55a51f6bee20 2023-10-16T14:38:14.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.823+0000 7f8f4bfff700 1 -- 172.21.15.178:0/980819220 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5c09e630 con 0x55a51f6bee20 2023-10-16T14:38:14.846 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.823+0000 7f8f4bfff700 1 -- 172.21.15.178:0/980819220 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0caef0 con 0x55a51f6bee20 2023-10-16T14:38:14.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.846+0000 7f8f7b030b80 1 -- 172.21.15.178:0/980819220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 msgr2=0x55a51f6c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.846+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c5b0 tx=0x7f8f5c003820 comp rx=0 tx=0).stop 2023-10-16T14:38:14.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.849+0000 7f8f7b030b80 1 -- 172.21.15.178:0/980819220 shutdown_connections 2023-10-16T14:38:14.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.849+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6b87a0 0x55a51f6e7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.857 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.849+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f6bee20 0x55a51f6c8c10 secure :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00c5b0 tx=0x7f8f5c003820 comp rx=0 tx=0).stop 2023-10-16T14:38:14.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.849+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/980819220 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.849+0000 7f8f7b030b80 1 -- 172.21.15.178:0/980819220 >> 172.21.15.178:0/980819220 conn(0x55a51f5ecdd0 msgr2=0x55a51f6c7950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.877 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.874+0000 7f8f7b030b80 1 -- 172.21.15.178:0/980819220 shutdown_connections 2023-10-16T14:38:14.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.874+0000 7f8f7b030b80 1 -- 172.21.15.178:0/980819220 wait complete. 2023-10-16T14:38:14.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.874+0000 7f8f7b030b80 1 -- start start 2023-10-16T14:38:14.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.876+0000 7f8f7b030b80 1 Processor -- start 2023-10-16T14:38:14.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.885+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 0x55a51f739710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.885+0000 7f8f7b030b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.885+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 0x55a51f739710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.885+0000 7f8f6559b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 0x55a51f739710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:41214/0 (socket says 172.21.15.178:41214) 2023-10-16T14:38:14.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f6559b700 1 -- 172.21.15.178:0/2597487238 learned_addr learned my addr 172.21.15.178:0/2597487238 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 0x55a51f7381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f739c90 con 0x55a51f6c0620 2023-10-16T14:38:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f739e00 con 0x55a51f6b87a0 2023-10-16T14:38:14.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a51f738760 con 0x55a51f737d20 2023-10-16T14:38:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.887+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 0x55a51f7381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f7b030b80 10 client.0 _ll_register_callbacks cb 0x7f8f50005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:14.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 -- 172.21.15.178:0/2597487238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 msgr2=0x55a51f739710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 0x55a51f739710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 -- 172.21.15.178:0/2597487238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 msgr2=0x55a51f6f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601a5690 con 0x55a51f737d20 2023-10-16T14:38:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f65d9c700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 0x55a51f7381a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b010 tx=0x7f8f58015c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf2f5eee817d630 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.888+0000 7f8f48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:38:14.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.889+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800e040 con 0x55a51f737d20 2023-10-16T14:38:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.889+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f5800f040 con 0x55a51f737d20 2023-10-16T14:38:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.890+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8f580093a0 con 0x55a51f737d20 2023-10-16T14:38:14.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.890+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f58003c10 con 0x55a51f737d20 2023-10-16T14:38:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.898+0000 7f8f7b030b80 10 client.26122 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:14.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.898+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55a51f463250 con 0x55a51f737d20 2023-10-16T14:38:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5800f040 con 0x55a51f737d20 2023-10-16T14:38:14.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f8f5801e500 con 0x55a51f737d20 2023-10-16T14:38:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 10 client.26122 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 10 client.26122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:38:14.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 20 client.26122 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f74e260 0x55a51f750650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:14.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 4 client.26122 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:38:14.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a51f60ae50 con 0x55a51f74e260 2023-10-16T14:38:14.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.903+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f74e260 0x55a51f750650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:14.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.906+0000 7f8f6559b700 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f74e260 0x55a51f750650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=744a39f819206937 server_cookie=ca421c794bb185ee in_seq=0 out_seq=0 2023-10-16T14:38:14.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.906+0000 7f8f48ff9700 10 client.26122 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:38:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.910+0000 7f8f48ff9700 1 -- 172.21.15.178:0/2597487238 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a51f60ae50 con 0x55a51f74e260 2023-10-16T14:38:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.910+0000 7f8f48ff9700 10 client.26122 handle_command_reply: tid=0 2023-10-16T14:38:14.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.910+0000 7f8f7b030b80 1 client.26122 shutdown 2023-10-16T14:38:14.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.918+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 msgr2=0x55a51f7381a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.918+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 0x55a51f7381a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800b010 tx=0x7f8f58015c80 comp rx=0 tx=0).stop 2023-10-16T14:38:14.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 shutdown_connections 2023-10-16T14:38:14.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55a51f74e260 0x55a51f750650 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51f737d20 0x55a51f7381a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55a51f6b87a0 0x55a51f739710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 --2- 172.21.15.178:0/2597487238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55a51f6c0620 0x55a51f6f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.919+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 >> 172.21.15.178:0/2597487238 conn(0x55a51f5ecdd0 msgr2=0x55a51f6c8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.920+0000 7f8f48ff9700 0 client.26122 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:38:14.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.933+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 shutdown_connections 2023-10-16T14:38:14.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.939+0000 7f8f7b030b80 1 -- 172.21.15.178:0/2597487238 wait complete. 2023-10-16T14:38:14.947 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:38:14.948 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.942+0000 7f8f7b030b80 20 client.26122 trim_cache size 0 max 16384 2023-10-16T14:38:14.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 msgr2=0x7f8f4c0a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 0x7f8f4c0a3a40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000b2e0 tx=0x7f8f50005fb0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 msgr2=0x7f8f6019d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:14.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00eb30 tx=0x7f8f5c00ee40 comp rx=0 tx=0).stop 2023-10-16T14:38:14.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 shutdown_connections 2023-10-16T14:38:14.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8f4c0a1580 0x7f8f4c0a3a40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.978 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8f6010d4a0 0x7f8f601a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f60073910 0x7f8f6019d230 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00eb30 tx=0x7f8f5c00ee40 comp rx=0 tx=0).stop 2023-10-16T14:38:14.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 --2- 172.21.15.178:0/2464942757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f60074ef0 0x7f8f6019d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:14.979 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 >> 172.21.15.178:0/2464942757 conn(0x7f8f600fc400 msgr2=0x7f8f600fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:14.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.957+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 shutdown_connections 2023-10-16T14:38:14.980 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:14.958+0000 7f8f677ff700 1 -- 172.21.15.178:0/2464942757 wait complete. 2023-10-16T14:38:15.125 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:38:16.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.007+0000 7f36669ff700 1 -- 172.21.15.178:0/3795173572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 msgr2=0x7f366010cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.011 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.007+0000 7f36669ff700 1 --2- 172.21.15.178:0/3795173572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f366010cf10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3658009f90 tx=0x7f365800b330 comp rx=0 tx=0).stop 2023-10-16T14:38:16.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.013+0000 7f36669ff700 1 -- 172.21.15.178:0/3795173572 shutdown_connections 2023-10-16T14:38:16.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.013+0000 7f36669ff700 1 --2- 172.21.15.178:0/3795173572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f366010d5a0 0x7f366010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.013+0000 7f36669ff700 1 --2- 172.21.15.178:0/3795173572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3660108120 0x7f366010a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.013+0000 7f36669ff700 1 --2- 172.21.15.178:0/3795173572 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f366010cf10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.013+0000 7f36669ff700 1 -- 172.21.15.178:0/3795173572 >> 172.21.15.178:0/3795173572 conn(0x7f366006dd30 msgr2=0x7f36600701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.020+0000 7f36669ff700 1 -- 172.21.15.178:0/3795173572 shutdown_connections 2023-10-16T14:38:16.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.020+0000 7f36669ff700 1 -- 172.21.15.178:0/3795173572 wait complete. 2023-10-16T14:38:16.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.023+0000 7f36669ff700 1 Processor -- start 2023-10-16T14:38:16.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.024+0000 7f36669ff700 1 -- start start 2023-10-16T14:38:16.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3660108120 0x7f36601a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 0x7f36601a8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a9420 con 0x7f366010d5a0 2023-10-16T14:38:16.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a9590 con 0x7f366010aae0 2023-10-16T14:38:16.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f36669ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a9750 con 0x7f3660108120 2023-10-16T14:38:16.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f365ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3660108120 0x7f36601a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.033+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.034+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:41238/0 (socket says 172.21.15.178:41238) 2023-10-16T14:38:16.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.034+0000 7f365f7fe700 1 -- 172.21.15.178:0/3462371909 learned_addr learned my addr 172.21.15.178:0/3462371909 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:16.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.034+0000 7f365f7fe700 1 -- 172.21.15.178:0/3462371909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3660108120 msgr2=0x7f36601a1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.034+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3660108120 0x7f36601a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.035+0000 7f365f7fe700 1 -- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 msgr2=0x7f36601a8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.036+0000 7f3664f9c700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 0x7f36601a8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.036+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 0x7f36601a8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.036+0000 7f365f7fe700 1 -- 172.21.15.178:0/3462371909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009c40 con 0x7f366010aae0 2023-10-16T14:38:16.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.036+0000 7f3664f9c700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 0x7f36601a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:38:16.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.037+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3658004d50 tx=0x7f3658004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.040+0000 7f365d7fa700 1 -- 172.21.15.178:0/3462371909 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800e040 con 0x7f366010aae0 2023-10-16T14:38:16.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.040+0000 7f365d7fa700 1 -- 172.21.15.178:0/3462371909 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3658007a10 con 0x7f366010aae0 2023-10-16T14:38:16.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.040+0000 7f365d7fa700 1 -- 172.21.15.178:0/3462371909 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365801b910 con 0x7f366010aae0 2023-10-16T14:38:16.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.040+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36601a99d0 con 0x7f366010aae0 2023-10-16T14:38:16.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.040+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36601a9e00 con 0x7f366010aae0 2023-10-16T14:38:16.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:15 smithi178 ceph-mon[98495]: pgmap v475: 129 pgs: 129 active+clean; 920 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 644 op/s 2023-10-16T14:38:16.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.050+0000 7f365d7fa700 1 -- 172.21.15.178:0/3462371909 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3658007c00 con 0x7f366010aae0 2023-10-16T14:38:16.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.054+0000 7f367a3e2b80 1 Processor -- start 2023-10-16T14:38:16.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.055+0000 7f367a3e2b80 1 -- start start 2023-10-16T14:38:16.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.057+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.057+0000 7f365ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.057+0000 7f365ffff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:35042/0 (socket says 172.21.15.178:35042) 2023-10-16T14:38:16.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.058+0000 7f365ffff700 1 -- 172.21.15.178:0/1364705367 learned_addr learned my addr 172.21.15.178:0/1364705367 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:16.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.058+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a9634d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.057+0000 7f365d7fa700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 0x7f36480a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634d620 0x560a9634daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.063 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a962c0c70 con 0x560a9634d620 2023-10-16T14:38:16.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a962a5c60 con 0x560a9634c780 2023-10-16T14:38:16.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a962a3580 con 0x560a9634be20 2023-10-16T14:38:16.064 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f365f7fe700 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a9634d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.059+0000 7f3664f9c700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 0x7f36480a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365d7fa700 1 -- 172.21.15.178:0/3462371909 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f36580ce2e0 con 0x7f366010aae0 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 -- 172.21.15.178:0/1364705367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 msgr2=0x560a9634d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a9634d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 -- 172.21.15.178:0/1364705367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634d620 msgr2=0x560a9634daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.065 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634d620 0x560a9634daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 -- 172.21.15.178:0/1364705367 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a9625c3f0 con 0x560a9634be20 2023-10-16T14:38:16.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.060+0000 7f365ffff700 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3650009c40 tx=0x7f3650009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20b222909b58e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.066 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.063+0000 7f36467fc700 1 -- 172.21.15.178:0/1364705367 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365000ebb0 con 0x560a9634be20 2023-10-16T14:38:16.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.063+0000 7f36467fc700 1 -- 172.21.15.178:0/1364705367 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3650004cf0 con 0x560a9634be20 2023-10-16T14:38:16.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.063+0000 7f36467fc700 1 -- 172.21.15.178:0/1364705367 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650005720 con 0x560a9634be20 2023-10-16T14:38:16.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.064+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 msgr2=0x560a9634c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.067 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.064+0000 7f3664f9c700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 0x7f36480a3ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3654005d50 tx=0x7f365400c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.068 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.064+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3650009c40 tx=0x7f3650009f50 comp rx=0 tx=0).stop 2023-10-16T14:38:16.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 shutdown_connections 2023-10-16T14:38:16.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634d620 0x560a9634daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.069 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634be20 0x560a9634c240 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1364705367 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a9634d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 >> 172.21.15.178:0/1364705367 conn(0x560a96279dd0 msgr2=0x560a96346fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.065+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 shutdown_connections 2023-10-16T14:38:16.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.067+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1364705367 wait complete. 2023-10-16T14:38:16.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.068+0000 7f367a3e2b80 1 -- start start 2023-10-16T14:38:16.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.069+0000 7f367a3e2b80 1 Processor -- start 2023-10-16T14:38:16.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.071+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a96355cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a96356210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a9635d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9629ee70 con 0x560a9634be20 2023-10-16T14:38:16.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:15 smithi186 ceph-mon[134892]: pgmap v475: 129 pgs: 129 active+clean; 920 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 644 op/s 2023-10-16T14:38:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9623b710 con 0x560a9634c780 2023-10-16T14:38:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a96356210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f365ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a96355cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.072+0000 7f365f7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a96356210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:41246/0 (socket says 172.21.15.178:41246) 2023-10-16T14:38:16.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.073+0000 7f365f7fe700 1 -- 172.21.15.178:0/524176744 learned_addr learned my addr 172.21.15.178:0/524176744 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:16.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.073+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9623b3b0 con 0x560a9634d620 2023-10-16T14:38:16.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.073+0000 7f367a3e2b80 10 client.0 _ll_register_callbacks cb 0x560a96098800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:16.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.073+0000 7f3664f9c700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a9635d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.074+0000 7f3664f9c700 1 -- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 msgr2=0x560a96356210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.074+0000 7f3664f9c700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a96356210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.074+0000 7f3664f9c700 1 -- 172.21.15.178:0/524176744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 msgr2=0x560a96355cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.075+0000 7f3664f9c700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a96355cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.075+0000 7f3664f9c700 1 -- 172.21.15.178:0/524176744 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36500098a0 con 0x560a9634d620 2023-10-16T14:38:16.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.075+0000 7f365ffff700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a96355cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:38:16.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.075+0000 7f3664f9c700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a9635d250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x560a9634ce00 tx=0x7f3654004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26634f6fea90c4b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.075+0000 7f363f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:38:16.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.076+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365401e070 con 0x560a9634d620 2023-10-16T14:38:16.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.077+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f36540049d0 con 0x560a9634d620 2023-10-16T14:38:16.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.077+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a962a3ef0 con 0x560a9634d620 2023-10-16T14:38:16.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.078+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3654007e90 con 0x560a9634d620 2023-10-16T14:38:16.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.078+0000 7f367a3e2b80 10 client.26131 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:16.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.078+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560a960c9de0 con 0x560a9634d620 2023-10-16T14:38:16.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.079+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560a962a3ef0 con 0x560a9634d620 2023-10-16T14:38:16.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.079+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f3654023420 con 0x560a9634d620 2023-10-16T14:38:16.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.079+0000 7f367a3e2b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:16.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.079+0000 7f367a3e2b80 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:16.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.079+0000 7f367a3e2b80 20 client.26131 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:16.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.080+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a96363f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.080+0000 7f367a3e2b80 4 client.26131 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:38:16.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.080+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a9631bc10 con 0x560a96361b00 2023-10-16T14:38:16.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.080+0000 7f365ffff700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a96363f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.087+0000 7f365ffff700 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a96363f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f021483071999f9 server_cookie=53c319730c6c90c2 in_seq=0 out_seq=0 2023-10-16T14:38:16.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.087+0000 7f363f7fe700 10 client.26131 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:16.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.094+0000 7f363f7fe700 1 -- 172.21.15.178:0/524176744 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a9631bc10 con 0x560a96361b00 2023-10-16T14:38:16.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.094+0000 7f363f7fe700 10 client.26131 handle_command_reply: tid=0 2023-10-16T14:38:16.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.094+0000 7f367a3e2b80 1 client.26131 shutdown 2023-10-16T14:38:16.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.095+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 msgr2=0x560a9635d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.095+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a9635d250 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x560a9634ce00 tx=0x7f3654004960 comp rx=0 tx=0).stop 2023-10-16T14:38:16.100 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 shutdown_connections 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a96363f20 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a9635d250 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a96355cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/524176744 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634c780 0x560a96356210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f363f7fe700 0 client.26131 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:16.101 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.097+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 >> 172.21.15.178:0/524176744 conn(0x560a96279dd0 msgr2=0x560a96346bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.101+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 shutdown_connections 2023-10-16T14:38:16.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.101+0000 7f367a3e2b80 1 -- 172.21.15.178:0/524176744 wait complete. 2023-10-16T14:38:16.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.108+0000 7f367a3e2b80 20 client.26131 trim_cache size 0 max 16384 2023-10-16T14:38:16.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:15 smithi187 ceph-mon[135297]: pgmap v475: 129 pgs: 129 active+clean; 920 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 644 op/s 2023-10-16T14:38:16.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.169+0000 7f367a3e2b80 1 Processor -- start 2023-10-16T14:38:16.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.173+0000 7f367a3e2b80 1 -- start start 2023-10-16T14:38:16.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634be20 0x560a96355a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 0x560a9635d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a96355fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9611f920 con 0x560a9634c780 2023-10-16T14:38:16.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a963542e0 con 0x560a9634be20 2023-10-16T14:38:16.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.177+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a96355180 con 0x560a9634d620 2023-10-16T14:38:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.178+0000 7f3664f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a96355fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.178+0000 7f3664f9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a96355fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:35058/0 (socket says 172.21.15.178:35058) 2023-10-16T14:38:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.178+0000 7f3664f9c700 1 -- 172.21.15.178:0/3919323640 learned_addr learned my addr 172.21.15.178:0/3919323640 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:16.183 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.178+0000 7f365ffff700 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634be20 0x560a96355a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.179+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 0x560a9635d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.180+0000 7f365f7fe700 1 -- 172.21.15.178:0/3919323640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 msgr2=0x560a96355fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.180+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a96355fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.180+0000 7f365f7fe700 1 -- 172.21.15.178:0/3919323640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634be20 msgr2=0x560a96355a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.180+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634be20 0x560a96355a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.180+0000 7f365f7fe700 1 -- 172.21.15.178:0/3919323640 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a96343580 con 0x560a9634c780 2023-10-16T14:38:16.187 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.184+0000 7f365f7fe700 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 0x560a9635d170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3658006070 tx=0x7f365806d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457fa5e768cc77a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.200+0000 7f3646ffd700 1 -- 172.21.15.178:0/3919323640 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658012b50 con 0x560a9634c780 2023-10-16T14:38:16.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.200+0000 7f3646ffd700 1 -- 172.21.15.178:0/3919323640 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3658066c60 con 0x560a9634c780 2023-10-16T14:38:16.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.200+0000 7f3646ffd700 1 -- 172.21.15.178:0/3919323640 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580a36b0 con 0x560a9634c780 2023-10-16T14:38:16.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.202+0000 7f367a3e2b80 1 -- 172.21.15.178:0/3919323640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 msgr2=0x560a9635d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.202+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 0x560a9635d170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3658006070 tx=0x7f365806d2b0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.203+0000 7f367a3e2b80 1 -- 172.21.15.178:0/3919323640 shutdown_connections 2023-10-16T14:38:16.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.203+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634d620 0x560a96355fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.203+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634be20 0x560a96355a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.203+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/3919323640 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634c780 0x560a9635d170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.204+0000 7f367a3e2b80 1 -- 172.21.15.178:0/3919323640 >> 172.21.15.178:0/3919323640 conn(0x560a96279dd0 msgr2=0x560a96350590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.206+0000 7f367a3e2b80 1 -- 172.21.15.178:0/3919323640 shutdown_connections 2023-10-16T14:38:16.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.206+0000 7f367a3e2b80 1 -- 172.21.15.178:0/3919323640 wait complete. 2023-10-16T14:38:16.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.207+0000 7f367a3e2b80 1 -- start start 2023-10-16T14:38:16.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.209+0000 7f367a3e2b80 1 Processor -- start 2023-10-16T14:38:16.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.211+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.211+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 0x560a9635b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f367a3e2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 0x560a9635c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9635cce0 con 0x560a9634be20 2023-10-16T14:38:16.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9635ce50 con 0x560a9634d620 2023-10-16T14:38:16.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f367a3e2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a9635d010 con 0x560a9634c780 2023-10-16T14:38:16.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f365ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f365ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:35312/0 (socket says 172.21.15.178:35312) 2023-10-16T14:38:16.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.212+0000 7f365ffff700 1 -- 172.21.15.178:0/1549509434 learned_addr learned my addr 172.21.15.178:0/1549509434 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:16.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f367a3e2b80 10 client.0 _ll_register_callbacks cb 0x7f3654007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:16.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365f7fe700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 0x560a9635b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f3664f9c700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 0x560a9635c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365ffff700 1 -- 172.21.15.178:0/1549509434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 msgr2=0x560a9635b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365ffff700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 0x560a9635b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365ffff700 1 -- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 msgr2=0x560a9635c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:16.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365ffff700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 0x560a9635c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.213+0000 7f365ffff700 1 -- 172.21.15.178:0/1549509434 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36601a9e00 con 0x560a9634be20 2023-10-16T14:38:16.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.215+0000 7f3664f9c700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 0x560a9635c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:38:16.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.215+0000 7f365f7fe700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 0x560a9635b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:38:16.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.217+0000 7f365ffff700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f365000e680 tx=0x7f3650009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fa8cb45e6dbf37 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:16.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.217+0000 7f363f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:38:16.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.219+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650004f60 con 0x560a9634be20 2023-10-16T14:38:16.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.220+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3650004b20 con 0x560a9634be20 2023-10-16T14:38:16.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.221+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3650004cf0 con 0x560a9634be20 2023-10-16T14:38:16.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.221+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36500279f0 con 0x560a9634be20 2023-10-16T14:38:16.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.221+0000 7f367a3e2b80 10 client.16077 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:16.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.221+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x560a960b2c60 con 0x560a9634be20 2023-10-16T14:38:16.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.222+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3650004cf0 con 0x560a9634be20 2023-10-16T14:38:16.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.223+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f365000c070 con 0x560a9634be20 2023-10-16T14:38:16.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:16.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 10 client.16077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:16.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 20 client.16077 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:16.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a963b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:16.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 4 client.16077 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:38:16.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a96297e50 con 0x560a96361b00 2023-10-16T14:38:16.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.224+0000 7f365f7fe700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a963b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:16.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.225+0000 7f365f7fe700 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a963b8c00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dafad159a5bc178 server_cookie=e768b548790c3ff6 in_seq=0 out_seq=0 2023-10-16T14:38:16.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.225+0000 7f363f7fe700 10 client.16077 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:16.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.225+0000 7f363f7fe700 1 -- 172.21.15.178:0/1549509434 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a96297e50 con 0x560a96361b00 2023-10-16T14:38:16.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.225+0000 7f363f7fe700 10 client.16077 handle_command_reply: tid=0 2023-10-16T14:38:16.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.225+0000 7f367a3e2b80 1 client.16077 shutdown 2023-10-16T14:38:16.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 msgr2=0x560a9635b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f365000e680 tx=0x7f3650009780 comp rx=0 tx=0).stop 2023-10-16T14:38:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 shutdown_connections 2023-10-16T14:38:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x560a96361b00 0x560a963b8c00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x560a9634d620 0x560a9635c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x560a9634be20 0x560a9635b2c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 --2- 172.21.15.178:0/1549509434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560a9634c780 0x560a9635b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f363f7fe700 0 client.16077 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:16.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 >> 172.21.15.178:0/1549509434 conn(0x560a96279dd0 msgr2=0x560a96355a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 shutdown_connections 2023-10-16T14:38:16.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.229+0000 7f367a3e2b80 1 -- 172.21.15.178:0/1549509434 wait complete. 2023-10-16T14:38:16.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.230+0000 7f367a3e2b80 20 client.16077 trim_cache size 0 max 16384 2023-10-16T14:38:16.235 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:38:16.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.237+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 msgr2=0x7f36480a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.237+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 0x7f36480a3ac0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3654005d50 tx=0x7f365400c040 comp rx=0 tx=0).stop 2023-10-16T14:38:16.241 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 msgr2=0x7f36601a1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:16.242 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3658004d50 tx=0x7f3658004e30 comp rx=0 tx=0).stop 2023-10-16T14:38:16.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 shutdown_connections 2023-10-16T14:38:16.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f36480a1600 0x7f36480a3ac0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366010d5a0 0x7f36601a8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3660108120 0x7f36601a1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.243 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 --2- 172.21.15.178:0/3462371909 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f366010aae0 0x7f36601a1da0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:16.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.238+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 >> 172.21.15.178:0/3462371909 conn(0x7f366006dd30 msgr2=0x7f366010f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:16.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.239+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 shutdown_connections 2023-10-16T14:38:16.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:16.239+0000 7f36669ff700 1 -- 172.21.15.178:0/3462371909 wait complete. 2023-10-16T14:38:17.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:38:17.322 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:38:18.067 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:17 smithi178 ceph-mon[98495]: pgmap v476: 129 pgs: 129 active+clean; 970 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 859 op/s 2023-10-16T14:38:18.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:17 smithi186 ceph-mon[134892]: pgmap v476: 129 pgs: 129 active+clean; 970 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 859 op/s 2023-10-16T14:38:18.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:17 smithi187 ceph-mon[135297]: pgmap v476: 129 pgs: 129 active+clean; 970 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 859 op/s 2023-10-16T14:38:19.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:18 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:38:19.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:18 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:38:19.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:18 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:38:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:19 smithi187 ceph-mon[135297]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 801 op/s 2023-10-16T14:38:20.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:19 smithi178 ceph-mon[98495]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 801 op/s 2023-10-16T14:38:20.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:19 smithi186 ceph-mon[134892]: pgmap v477: 129 pgs: 129 active+clean; 981 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 801 op/s 2023-10-16T14:38:21.039 INFO:tasks.workunit.client.0.smithi178.stdout: 238 453978 41 320081 171 219960 686 2023-10-16T14:38:21.547 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.527+0000 7f62cc1ff700 1 -- 172.21.15.178:0/2227225709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c410d1f0 msgr2=0x7f62c410d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.527+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/2227225709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c410d1f0 0x7f62c410d6a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009e00 tx=0x7f62bc00b330 comp rx=0 tx=0).stop 2023-10-16T14:38:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.544+0000 7f62cc1ff700 1 -- 172.21.15.178:0/2227225709 shutdown_connections 2023-10-16T14:38:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.544+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/2227225709 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c410d1f0 0x7f62c410d6a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.548 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.544+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/2227225709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c4107eb0 0x7f62c410a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.544+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/2227225709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410a820 0x7f62c410ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.544+0000 7f62cc1ff700 1 -- 172.21.15.178:0/2227225709 >> 172.21.15.178:0/2227225709 conn(0x7f62c406dd30 msgr2=0x7f62c40701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:21.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.594+0000 7f62cc1ff700 1 -- 172.21.15.178:0/2227225709 shutdown_connections 2023-10-16T14:38:21.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.598+0000 7f62cc1ff700 1 -- 172.21.15.178:0/2227225709 wait complete. 2023-10-16T14:38:21.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.599+0000 7f62cc1ff700 1 Processor -- start 2023-10-16T14:38:21.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 -- start start 2023-10-16T14:38:21.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4107eb0 0x7f62c419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410d1f0 0x7f62c41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c41a4f10 con 0x7f62c410a820 2023-10-16T14:38:21.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c41a5050 con 0x7f62c4107eb0 2023-10-16T14:38:21.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.607+0000 7f62cc1ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c41a51e0 con 0x7f62c410d1f0 2023-10-16T14:38:21.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.608+0000 7f62c979a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.608+0000 7f62c979a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40706/0 (socket says 172.21.15.178:40706) 2023-10-16T14:38:21.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.608+0000 7f62c979a700 1 -- 172.21.15.178:0/297099583 learned_addr learned my addr 172.21.15.178:0/297099583 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:21.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.609+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4107eb0 0x7f62c419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.609+0000 7f62ca79c700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410d1f0 0x7f62c41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.610+0000 7f62c979a700 1 -- 172.21.15.178:0/297099583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410d1f0 msgr2=0x7f62c41a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:21.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.610+0000 7f62c979a700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410d1f0 0x7f62c41a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.610+0000 7f62c979a700 1 -- 172.21.15.178:0/297099583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4107eb0 msgr2=0x7f62c419d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:21.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.610+0000 7f62c979a700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4107eb0 0x7f62c419d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.610+0000 7f62c979a700 1 -- 172.21.15.178:0/297099583 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc005e70 con 0x7f62c410a820 2023-10-16T14:38:21.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.611+0000 7f62c979a700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b550 tx=0x7f62b400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:21.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.617+0000 7f62baffd700 1 -- 172.21.15.178:0/297099583 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4033640 con 0x7f62c410a820 2023-10-16T14:38:21.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.621+0000 7f62baffd700 1 -- 172.21.15.178:0/297099583 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62b4033c60 con 0x7f62c410a820 2023-10-16T14:38:21.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.621+0000 7f62baffd700 1 -- 172.21.15.178:0/297099583 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4032d10 con 0x7f62c410a820 2023-10-16T14:38:21.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.622+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c41a54c0 con 0x7f62c410a820 2023-10-16T14:38:21.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.622+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c41a5a00 con 0x7f62c410a820 2023-10-16T14:38:21.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:21 smithi187 ceph-mon[135297]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 993 op/s 2023-10-16T14:38:21.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.691+0000 7f62dfa5ab80 1 Processor -- start 2023-10-16T14:38:21.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.691+0000 7f62baffd700 1 -- 172.21.15.178:0/297099583 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f62b403c400 con 0x7f62c410a820 2023-10-16T14:38:21.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.693+0000 7f62baffd700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 0x7f62b00a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.698 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.694+0000 7f62baffd700 1 -- 172.21.15.178:0/297099583 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f62b4033dd0 con 0x7f62c410a820 2023-10-16T14:38:21.699 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.695+0000 7f62ca79c700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 0x7f62b00a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.706 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.701+0000 7f62dfa5ab80 1 -- start start 2023-10-16T14:38:21.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.704+0000 7f62ca79c700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 0x7f62b00a3b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009fd0 tx=0x7f62bc00b730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:21.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac50ffb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac50ffc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac50ffcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50f6fcf0 con 0x55ac50ffaea0 2023-10-16T14:38:21.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50f54ce0 con 0x55ac50ffc6a0 2023-10-16T14:38:21.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.707+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50f52600 con 0x55ac50ffb800 2023-10-16T14:38:21.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.708+0000 7f62c979a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac50ffc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.708+0000 7f62c979a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac50ffc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:48046/0 (socket says 172.21.15.178:48046) 2023-10-16T14:38:21.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.708+0000 7f62c979a700 1 -- 172.21.15.178:0/1004763869 learned_addr learned my addr 172.21.15.178:0/1004763869 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:21.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac50ffb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 -- 172.21.15.178:0/1004763869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 msgr2=0x55ac50ffc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:21.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac50ffc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 -- 172.21.15.178:0/1004763869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 msgr2=0x55ac50ffcb20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:21.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac50ffcb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.709+0000 7f62c9f9b700 1 -- 172.21.15.178:0/1004763869 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac50f0b470 con 0x55ac50ffaea0 2023-10-16T14:38:21.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.710+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac50ffb2c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f62c001ced0 tx=0x7f62c001e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ad1a649a8cbc03 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:21.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.713+0000 7f62affff700 1 -- 172.21.15.178:0/1004763869 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c001ed10 con 0x55ac50ffaea0 2023-10-16T14:38:21.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.713+0000 7f62affff700 1 -- 172.21.15.178:0/1004763869 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62c001ee70 con 0x55ac50ffaea0 2023-10-16T14:38:21.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.713+0000 7f62affff700 1 -- 172.21.15.178:0/1004763869 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c000d420 con 0x55ac50ffaea0 2023-10-16T14:38:21.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.715+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/1004763869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 msgr2=0x55ac50ffb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:21.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.715+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac50ffb2c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f62c001ced0 tx=0x7f62c001e630 comp rx=0 tx=0).stop 2023-10-16T14:38:21.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.752+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/1004763869 shutdown_connections 2023-10-16T14:38:21.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.752+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac50ffcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.752+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac50ffb2c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.752+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/1004763869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac50ffc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.752+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/1004763869 >> 172.21.15.178:0/1004763869 conn(0x55ac50f28e50 msgr2=0x55ac50ff6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:21.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.782+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/1004763869 shutdown_connections 2023-10-16T14:38:21.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.782+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/1004763869 wait complete. 2023-10-16T14:38:21.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.784+0000 7f62dfa5ab80 1 -- start start 2023-10-16T14:38:21.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.784+0000 7f62dfa5ab80 1 Processor -- start 2023-10-16T14:38:21.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac51004d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac51005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50f4def0 con 0x55ac50ffaea0 2023-10-16T14:38:21.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50eea790 con 0x55ac50ffc6a0 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.804+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50eea430 con 0x55ac50ffb800 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.805+0000 7f62ca79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.805+0000 7f62ca79c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:51816/0 (socket says 172.21.15.178:51816) 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.805+0000 7f62ca79c700 1 -- 172.21.15.178:0/2598600903 learned_addr learned my addr 172.21.15.178:0/2598600903 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.805+0000 7f62c979a700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac51005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62dfa5ab80 10 client.0 _ll_register_callbacks cb 0x55ac50d47880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:21.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62ca79c700 1 -- 172.21.15.178:0/2598600903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 msgr2=0x55ac51005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:21.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac51005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62ca79c700 1 -- 172.21.15.178:0/2598600903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 msgr2=0x55ac51004d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:21.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac51004d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:21.817 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62ca79c700 1 -- 172.21.15.178:0/2598600903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c001c8d0 con 0x55ac50ffc6a0 2023-10-16T14:38:21.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.806+0000 7f62c979a700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac51005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:38:21.818 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.807+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62bc015000 tx=0x7f62bc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821f610a2bbc7be8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:21.818 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:21 smithi178 ceph-mon[98495]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 993 op/s 2023-10-16T14:38:21.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:21 smithi186 ceph-mon[134892]: pgmap v478: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 16 MiB/s wr, 993 op/s 2023-10-16T14:38:21.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.828+0000 7f62ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:38:21.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.828+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc01b840 con 0x55ac50ffc6a0 2023-10-16T14:38:21.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.828+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f62bc022050 con 0x55ac50ffc6a0 2023-10-16T14:38:21.865 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.854+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac50f52f70 con 0x55ac50ffc6a0 2023-10-16T14:38:21.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.881+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62bc01b9a0 con 0x55ac50ffc6a0 2023-10-16T14:38:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.901+0000 7f62dfa5ab80 10 client.26168 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:21.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.901+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ac50d78e60 con 0x55ac50ffc6a0 2023-10-16T14:38:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.909+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac50f52f70 con 0x55ac50ffc6a0 2023-10-16T14:38:21.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.909+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f62bc027af0 con 0x55ac50ffc6a0 2023-10-16T14:38:21.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 10 client.26168 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:21.927 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:21.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 20 client.26168 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:21.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51010bb0 0x55ac51012ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:21.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 4 client.26168 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:38:21.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.914+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac50fcac90 con 0x55ac51010bb0 2023-10-16T14:38:21.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.929+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51010bb0 0x55ac51012ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:21.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.931+0000 7f62ad7fa700 10 client.26168 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:21.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.931+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51010bb0 0x55ac51012ff0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca901d104cc4eb7 server_cookie=8b2eaf459f3194e6 in_seq=0 out_seq=0 2023-10-16T14:38:21.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.936+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2598600903 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ac50fcac90 con 0x55ac51010bb0 2023-10-16T14:38:21.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.936+0000 7f62ad7fa700 10 client.26168 handle_command_reply: tid=0 2023-10-16T14:38:21.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.944+0000 7f62dfa5ab80 1 client.26168 shutdown 2023-10-16T14:38:21.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.985+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 msgr2=0x55ac5100c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:21.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.985+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f62bc015000 tx=0x7f62bc009920 comp rx=0 tx=0).stop 2023-10-16T14:38:21.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 shutdown_connections 2023-10-16T14:38:22.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51010bb0 0x55ac51012ff0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100c2d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffaea0 0x55ac51004d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2598600903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac51005290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.000 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 >> 172.21.15.178:0/2598600903 conn(0x55ac50f28e50 msgr2=0x55ac50ff5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:22.001 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:21.996+0000 7f62ad7fa700 0 client.26168 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:22.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.019+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 shutdown_connections 2023-10-16T14:38:22.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.020+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2598600903 wait complete. 2023-10-16T14:38:22.090 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.041+0000 7f62dfa5ab80 20 client.26168 trim_cache size 0 max 16384 2023-10-16T14:38:22.524 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.518+0000 7f62dfa5ab80 1 Processor -- start 2023-10-16T14:38:22.525 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.520+0000 7f62dfa5ab80 1 -- start start 2023-10-16T14:38:22.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.523+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffaea0 0x55ac51004d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.529+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac5100c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.529+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.533+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50dce9a0 con 0x55ac50ffc6a0 2023-10-16T14:38:22.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.533+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50fcac90 con 0x55ac50ffaea0 2023-10-16T14:38:22.538 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.533+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac50f52f70 con 0x55ac50ffb800 2023-10-16T14:38:22.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.534+0000 7f62ca79c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:22.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.534+0000 7f62ca79c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40730/0 (socket says 172.21.15.178:40730) 2023-10-16T14:38:22.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.534+0000 7f62ca79c700 1 -- 172.21.15.178:0/2124172232 learned_addr learned my addr 172.21.15.178:0/2124172232 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:22.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.535+0000 7f62ca79c700 1 -- 172.21.15.178:0/2124172232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 msgr2=0x55ac5100c290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:22.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.535+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac5100c290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.535+0000 7f62ca79c700 1 -- 172.21.15.178:0/2124172232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffaea0 msgr2=0x55ac51004d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:22.542 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.535+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffaea0 0x55ac51004d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.536+0000 7f62ca79c700 1 -- 172.21.15.178:0/2124172232 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac51017c60 con 0x55ac50ffc6a0 2023-10-16T14:38:22.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.536+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f62bc029000 tx=0x7f62bc007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf826a0e583642f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:22.545 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.541+0000 7f62b8ff9700 1 -- 172.21.15.178:0/2124172232 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc027620 con 0x55ac50ffc6a0 2023-10-16T14:38:22.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.541+0000 7f62b8ff9700 1 -- 172.21.15.178:0/2124172232 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62bc027780 con 0x55ac50ffc6a0 2023-10-16T14:38:22.546 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.541+0000 7f62b8ff9700 1 -- 172.21.15.178:0/2124172232 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc030810 con 0x55ac50ffc6a0 2023-10-16T14:38:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.548+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2124172232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 msgr2=0x55ac51005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:22.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.548+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f62bc029000 tx=0x7f62bc007ca0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.609+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2124172232 shutdown_connections 2023-10-16T14:38:22.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.609+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffc6a0 0x55ac51005250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.609+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffaea0 0x55ac51004d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.609+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2124172232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffb800 0x55ac5100c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.609+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2124172232 >> 172.21.15.178:0/2124172232 conn(0x55ac50f28e50 msgr2=0x55ac50fff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:22.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.635+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2124172232 shutdown_connections 2023-10-16T14:38:22.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.639+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2124172232 wait complete. 2023-10-16T14:38:22.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.642+0000 7f62dfa5ab80 1 -- start start 2023-10-16T14:38:22.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.643+0000 7f62dfa5ab80 1 Processor -- start 2023-10-16T14:38:22.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.662+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 0x55ac5100aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.662+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 0x55ac5100b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62dfa5ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac51064a30 con 0x55ac50ffb800 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac51064b70 con 0x55ac50ffc6a0 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62dfa5ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac51064cb0 con 0x55ac50ffaea0 2023-10-16T14:38:22.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62dfa5ab80 10 client.0 _ll_register_callbacks cb 0x7f62bc007730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:22.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.663+0000 7f62c979a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 0x55ac5100b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:22.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c979a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 0x55ac5100b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40746/0 (socket says 172.21.15.178:40746) 2023-10-16T14:38:22.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c979a700 1 -- 172.21.15.178:0/2677680677 learned_addr learned my addr 172.21.15.178:0/2677680677 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:22.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 0x55ac5100aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:22.675 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62ca79c700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:22.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 -- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 msgr2=0x55ac5100b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:22.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 -- 172.21.15.178:0/2677680677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 msgr2=0x55ac5100b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:22.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 0x55ac5100b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.665+0000 7f62c9f9b700 1 -- 172.21.15.178:0/2677680677 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac50d78e60 con 0x55ac50ffaea0 2023-10-16T14:38:22.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.668+0000 7f62c9f9b700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 0x55ac5100aad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62c001c7f0 tx=0x7f62c0005da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1efc83e22ca5fa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:22.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.698+0000 7f62ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:38:22.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.698+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c0005590 con 0x55ac50ffaea0 2023-10-16T14:38:22.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.698+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f62c0003760 con 0x55ac50ffaea0 2023-10-16T14:38:22.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.729+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62c001ee70 con 0x55ac50ffaea0 2023-10-16T14:38:22.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.737+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62c00056f0 con 0x55ac50ffaea0 2023-10-16T14:38:22.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.765+0000 7f62dfa5ab80 10 client.26149 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:22.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.765+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55ac50d61ce0 con 0x55ac50ffaea0 2023-10-16T14:38:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62c001ee70 con 0x55ac50ffaea0 2023-10-16T14:38:22.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f62c000b040 con 0x55ac50ffaea0 2023-10-16T14:38:22.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 10 client.26149 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:22.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 10 client.26149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:22.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 20 client.26149 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:22.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51066770 0x55ac51068b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:22.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 4 client.26149 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:38:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.773+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55ac50f46ed0 con 0x55ac51066770 2023-10-16T14:38:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.774+0000 7f62c979a700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51066770 0x55ac51068b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:22.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.776+0000 7f62c979a700 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51066770 0x55ac51068b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d504baf17f0b094d server_cookie=2eb15d08c1e0f126 in_seq=0 out_seq=0 2023-10-16T14:38:22.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.803+0000 7f62ad7fa700 10 client.26149 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:22.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.803+0000 7f62ad7fa700 1 -- 172.21.15.178:0/2677680677 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ac50f46ed0 con 0x55ac51066770 2023-10-16T14:38:22.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.803+0000 7f62ad7fa700 10 client.26149 handle_command_reply: tid=0 2023-10-16T14:38:22.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.807+0000 7f62dfa5ab80 1 client.26149 shutdown 2023-10-16T14:38:22.882 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.869+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 msgr2=0x55ac5100aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:22.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.869+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 0x55ac5100aad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f62c001c7f0 tx=0x7f62c0005da0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 shutdown_connections 2023-10-16T14:38:22.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55ac51066770 0x55ac51068b90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ac50ffc6a0 0x55ac5100b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ac50ffaea0 0x55ac5100aad0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 --2- 172.21.15.178:0/2677680677 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55ac50ffb800 0x55ac5100b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:22.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 >> 172.21.15.178:0/2677680677 conn(0x55ac50f28e50 msgr2=0x55ac50fff610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:22.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.891+0000 7f62ad7fa700 0 client.26149 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:22.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.911+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 shutdown_connections 2023-10-16T14:38:22.934 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.924+0000 7f62dfa5ab80 1 -- 172.21.15.178:0/2677680677 wait complete. 2023-10-16T14:38:22.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:22.956+0000 7f62dfa5ab80 20 client.26149 trim_cache size 0 max 16384 2023-10-16T14:38:22.984 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:38:22.985 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:38:22.985 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "faf138f9-ada7-4b09-95d1-94c09def0776", 2023-10-16T14:38:22.985 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:38:22.985 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:38:23.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.010+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 msgr2=0x7f62b00a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:23.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.010+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 0x7f62b00a3b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009fd0 tx=0x7f62bc00b730 comp rx=0 tx=0).stop 2023-10-16T14:38:23.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.010+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 msgr2=0x7f62c419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:23.014 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.010+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b550 tx=0x7f62b400b860 comp rx=0 tx=0).stop 2023-10-16T14:38:23.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 shutdown_connections 2023-10-16T14:38:23.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f62b00a1680 0x7f62b00a3b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:23.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62c410d1f0 0x7f62c41a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:23.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f62c4107eb0 0x7f62c419d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:23.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 --2- 172.21.15.178:0/297099583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62c410a820 0x7f62c419d8d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:23.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.021+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 >> 172.21.15.178:0/297099583 conn(0x7f62c406dd30 msgr2=0x7f62c4108890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:23.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.025+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 shutdown_connections 2023-10-16T14:38:23.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:23.025+0000 7f62cc1ff700 1 -- 172.21.15.178:0/297099583 wait complete. 2023-10-16T14:38:23.801 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:38:24.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 13 MiB/s wr, 814 op/s 2023-10-16T14:38:24.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:38:24.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:24.060 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:38:24.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:38:24.061 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:23 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:38:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 13 MiB/s wr, 814 op/s 2023-10-16T14:38:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:38:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:38:24.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:38:24.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:23 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:38:24.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: pgmap v479: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 13 MiB/s wr, 814 op/s 2023-10-16T14:38:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:38:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:38:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:38:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:23 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:38:26.075 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:25 smithi178 ceph-mon[98495]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 842 op/s 2023-10-16T14:38:26.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:25 smithi186 ceph-mon[134892]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 842 op/s 2023-10-16T14:38:26.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:25 smithi187 ceph-mon[135297]: pgmap v480: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 842 op/s 2023-10-16T14:38:28.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:27 smithi186 ceph-mon[134892]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 753 op/s 2023-10-16T14:38:28.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:27 smithi187 ceph-mon[135297]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 753 op/s 2023-10-16T14:38:28.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:27 smithi178 ceph-mon[98495]: pgmap v481: 129 pgs: 129 active+clean; 1.0 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 753 op/s 2023-10-16T14:38:30.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:29 smithi186 ceph-mon[134892]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2023-10-16T14:38:30.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:29 smithi187 ceph-mon[135297]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2023-10-16T14:38:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:29 smithi178 ceph-mon[98495]: pgmap v482: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 534 op/s 2023-10-16T14:38:31.041 INFO:tasks.workunit.client.0.smithi178.stdout: 248 558340 705 397020 633 263817 886 2023-10-16T14:38:31.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.475+0000 7f510f5ff700 1 -- 172.21.15.178:0/826872739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 msgr2=0x7f510810d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:31.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.475+0000 7f510f5ff700 1 --2- 172.21.15.178:0/826872739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f510810d6a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f510000b1d0 tx=0x7f510000b4e0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.482+0000 7f510f5ff700 1 -- 172.21.15.178:0/826872739 shutdown_connections 2023-10-16T14:38:31.485 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.482+0000 7f510f5ff700 1 --2- 172.21.15.178:0/826872739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f510810d6a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.482+0000 7f510f5ff700 1 --2- 172.21.15.178:0/826872739 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108107eb0 0x7f510810a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.482+0000 7f510f5ff700 1 --2- 172.21.15.178:0/826872739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f510810a820 0x7f510810ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.482+0000 7f510f5ff700 1 -- 172.21.15.178:0/826872739 >> 172.21.15.178:0/826872739 conn(0x7f510806dd30 msgr2=0x7f51080701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:31.522 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.508+0000 7f510f5ff700 1 -- 172.21.15.178:0/826872739 shutdown_connections 2023-10-16T14:38:31.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.509+0000 7f510f5ff700 1 -- 172.21.15.178:0/826872739 wait complete. 2023-10-16T14:38:31.523 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.514+0000 7f510f5ff700 1 Processor -- start 2023-10-16T14:38:31.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.527+0000 7f510f5ff700 1 -- start start 2023-10-16T14:38:31.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.530+0000 7f510f5ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.551 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.530+0000 7f510f5ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5108114a30 0x7f51081b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510d39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510d39b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43438/0 (socket says 172.21.15.178:43438) 2023-10-16T14:38:31.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510f5ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108114eb0 0x7f51081b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510d39b700 1 -- 172.21.15.178:0/3839507726 learned_addr learned my addr 172.21.15.178:0/3839507726 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:31.552 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b6780 con 0x7f510810d1f0 2023-10-16T14:38:31.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b68c0 con 0x7f5108114eb0 2023-10-16T14:38:31.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.531+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081b6a00 con 0x7f5108114a30 2023-10-16T14:38:31.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5108114a30 0x7f51081b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510db9c700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108114eb0 0x7f51081b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.553 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510d39b700 1 -- 172.21.15.178:0/3839507726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5108114a30 msgr2=0x7f51081b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:31.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510d39b700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5108114a30 0x7f51081b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510d39b700 1 -- 172.21.15.178:0/3839507726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108114eb0 msgr2=0x7f51081b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:31.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510d39b700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108114eb0 0x7f51081b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.532+0000 7f510d39b700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100005e70 con 0x7f510810d1f0 2023-10-16T14:38:31.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.533+0000 7f510d39b700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:31.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.551+0000 7f50fe7fc700 1 -- 172.21.15.178:0/3839507726 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8011630 con 0x7f510810d1f0 2023-10-16T14:38:31.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.551+0000 7f50fe7fc700 1 -- 172.21.15.178:0/3839507726 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f8011c50 con 0x7f510810d1f0 2023-10-16T14:38:31.555 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.551+0000 7f50fe7fc700 1 -- 172.21.15.178:0/3839507726 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8010c10 con 0x7f510810d1f0 2023-10-16T14:38:31.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.559+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51081b6ce0 con 0x7f510810d1f0 2023-10-16T14:38:31.571 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.559+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081b7170 con 0x7f510810d1f0 2023-10-16T14:38:31.589 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.578+0000 7f50fe7fc700 1 -- 172.21.15.178:0/3839507726 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f50f8010d70 con 0x7f510810d1f0 2023-10-16T14:38:31.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.589+0000 7f50fe7fc700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 0x7f50f40a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.593+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 0x7f50f40a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.605+0000 7f50fe7fc700 1 -- 172.21.15.178:0/3839507726 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f50f80cc400 con 0x7f510810d1f0 2023-10-16T14:38:31.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.608+0000 7f5122f03b80 1 Processor -- start 2023-10-16T14:38:31.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.613+0000 7f5122f03b80 1 -- start start 2023-10-16T14:38:31.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a869240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a86a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7ddc70 con 0x55be4a868e20 2023-10-16T14:38:31.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7c2c60 con 0x55be4a869780 2023-10-16T14:38:31.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.614+0000 7f5122f03b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7c0580 con 0x55be4a86a620 2023-10-16T14:38:31.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.620+0000 7f510db9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.620+0000 7f510db9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57278/0 (socket says 172.21.15.178:57278) 2023-10-16T14:38:31.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.620+0000 7f510db9c700 1 -- 172.21.15.178:0/1722010425 learned_addr learned my addr 172.21.15.178:0/1722010425 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:31.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 0x7f50f40a3b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5108072320 tx=0x7f5104009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:31.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510cb9a700 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a86a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510db9c700 1 -- 172.21.15.178:0/1722010425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 msgr2=0x55be4a86a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:31.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510db9c700 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a86a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510db9c700 1 -- 172.21.15.178:0/1722010425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 msgr2=0x55be4a869240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:38:31.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.628+0000 7f510db9c700 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a869240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.629+0000 7f510db9c700 1 -- 172.21.15.178:0/1722010425 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be4a7793f0 con 0x55be4a86a620 2023-10-16T14:38:31.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.632+0000 7f510db9c700 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5100007d40 tx=0x7f5100007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a795f1a5ac02436e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:31.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.644+0000 7f50f37fe700 1 -- 172.21.15.178:0/1722010425 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000e040 con 0x55be4a86a620 2023-10-16T14:38:31.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.644+0000 7f50f37fe700 1 -- 172.21.15.178:0/1722010425 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51000076f0 con 0x55be4a86a620 2023-10-16T14:38:31.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.644+0000 7f50f37fe700 1 -- 172.21.15.178:0/1722010425 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510003dd70 con 0x55be4a86a620 2023-10-16T14:38:31.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.651+0000 7f5122f03b80 1 -- 172.21.15.178:0/1722010425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 msgr2=0x55be4a86aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:31.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.651+0000 7f5122f03b80 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5100007d40 tx=0x7f5100007d70 comp rx=0 tx=0).stop 2023-10-16T14:38:31.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:31 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:31 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:31 smithi187 ceph-mon[135297]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-10-16T14:38:31.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.682+0000 7f5122f03b80 1 -- 172.21.15.178:0/1722010425 shutdown_connections 2023-10-16T14:38:31.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.682+0000 7f5122f03b80 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a86aaa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.682+0000 7f5122f03b80 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a869240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.682+0000 7f5122f03b80 1 --2- 172.21.15.178:0/1722010425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a86a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.682+0000 7f5122f03b80 1 -- 172.21.15.178:0/1722010425 >> 172.21.15.178:0/1722010425 conn(0x55be4a796dd0 msgr2=0x55be4a863fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:31.742 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.735+0000 7f5122f03b80 1 -- 172.21.15.178:0/1722010425 shutdown_connections 2023-10-16T14:38:31.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.741+0000 7f5122f03b80 1 -- 172.21.15.178:0/1722010425 wait complete. 2023-10-16T14:38:31.768 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.742+0000 7f5122f03b80 1 -- start start 2023-10-16T14:38:31.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.743+0000 7f5122f03b80 1 Processor -- start 2023-10-16T14:38:31.769 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a868e20 0x55be4a872d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7bbe70 con 0x55be4a86a620 2023-10-16T14:38:31.770 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a758710 con 0x55be4a869780 2023-10-16T14:38:31.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7583b0 con 0x55be4a868e20 2023-10-16T14:38:31.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.752+0000 7f5122f03b80 10 client.0 _ll_register_callbacks cb 0x55be4a5b5800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:31.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.755+0000 7f510db9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.758+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.771 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.758+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:39622/0 (socket says 172.21.15.178:39622) 2023-10-16T14:38:31.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.758+0000 7f510cb9a700 1 -- 172.21.15.178:0/3064151199 learned_addr learned my addr 172.21.15.178:0/3064151199 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:31.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.758+0000 7f510d39b700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a868e20 0x55be4a872d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.772 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.757+0000 7f510db9c700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43448/0 (socket says 172.21.15.178:43448) 2023-10-16T14:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.760+0000 7f510cb9a700 1 -- 172.21.15.178:0/3064151199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a868e20 msgr2=0x55be4a872d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.760+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a868e20 0x55be4a872d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.760+0000 7f510cb9a700 1 -- 172.21.15.178:0/3064151199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 msgr2=0x55be4a87a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.762+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.773 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.762+0000 7f510cb9a700 1 -- 172.21.15.178:0/3064151199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510810e240 con 0x55be4a869780 2023-10-16T14:38:31.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.762+0000 7f510db9c700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:38:31.774 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.762+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5104015730 tx=0x7f51040047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9baf4b49b46cb26 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:31.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:31 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:31 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:31 smithi178 ceph-mon[98495]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-10-16T14:38:31.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:31 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:31 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:31 smithi186 ceph-mon[134892]: pgmap v483: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 563 op/s 2023-10-16T14:38:31.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.795+0000 7f50f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:38:31.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.796+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510400ee40 con 0x55be4a869780 2023-10-16T14:38:31.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.796+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5104015d50 con 0x55be4a869780 2023-10-16T14:38:31.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.804+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be4a7c0ef0 con 0x55be4a869780 2023-10-16T14:38:31.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.835+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5104021470 con 0x55be4a869780 2023-10-16T14:38:31.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.852+0000 7f5122f03b80 10 client.26180 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:31.856 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.852+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55be4a5e6de0 con 0x55be4a869780 2023-10-16T14:38:31.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.853+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be4a7c0ef0 con 0x55be4a869780 2023-10-16T14:38:31.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.853+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5104013700 con 0x55be4a869780 2023-10-16T14:38:31.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 10 client.26180 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:31.858 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 10 client.26180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:31.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 20 client.26180 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:31.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a87eab0 0x55be4a880ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:31.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 4 client.26180 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:38:31.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be4a838c10 con 0x55be4a87eab0 2023-10-16T14:38:31.859 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.854+0000 7f510d39b700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a87eab0 0x55be4a880ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:31.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.855+0000 7f510d39b700 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a87eab0 0x55be4a880ed0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6023e641a935be4b server_cookie=33ecc5742b7c2817 in_seq=0 out_seq=0 2023-10-16T14:38:31.862 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.855+0000 7f50f0ff9700 10 client.26180 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.877+0000 7f50f0ff9700 1 -- 172.21.15.178:0/3064151199 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55be4a838c10 con 0x55be4a87eab0 2023-10-16T14:38:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.877+0000 7f50f0ff9700 10 client.26180 handle_command_reply: tid=0 2023-10-16T14:38:31.881 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.877+0000 7f5122f03b80 1 client.26180 shutdown 2023-10-16T14:38:31.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.901+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 msgr2=0x55be4a873260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:31.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.901+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5104015730 tx=0x7f51040047b0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 shutdown_connections 2023-10-16T14:38:31.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a87eab0 0x55be4a880ed0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a86a620 0x55be4a87a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a868e20 0x55be4a872d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3064151199 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a873260 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:31.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.959+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 >> 172.21.15.178:0/3064151199 conn(0x55be4a796dd0 msgr2=0x55be4a863bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:31.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.972+0000 7f50f0ff9700 0 client.26180 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.976+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 shutdown_connections 2023-10-16T14:38:31.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.978+0000 7f5122f03b80 1 -- 172.21.15.178:0/3064151199 wait complete. 2023-10-16T14:38:31.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:31.981+0000 7f5122f03b80 20 client.26180 trim_cache size 0 max 16384 2023-10-16T14:38:32.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.598+0000 7f5122f03b80 1 Processor -- start 2023-10-16T14:38:32.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.598+0000 7f5122f03b80 1 -- start start 2023-10-16T14:38:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.599+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 0x55be4a871850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.599+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.599+0000 7f510d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 0x55be4a871850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.599+0000 7f510d39b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 0x55be4a871850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:39634/0 (socket says 172.21.15.178:39634) 2023-10-16T14:38:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.600+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.600+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:43450/0 (socket says 172.21.15.178:43450) 2023-10-16T14:38:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.600+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a863b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.603+0000 7f5122f03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a63c920 con 0x55be4a869780 2023-10-16T14:38:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.603+0000 7f5122f03b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a838c10 con 0x55be4a868e20 2023-10-16T14:38:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.603+0000 7f5122f03b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a7c0ef0 con 0x55be4a86a620 2023-10-16T14:38:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.603+0000 7f510d39b700 1 -- 172.21.15.178:0/3094653749 learned_addr learned my addr 172.21.15.178:0/3094653749 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 -- 172.21.15.178:0/3094653749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 msgr2=0x55be4a863b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:32.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a863b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 -- 172.21.15.178:0/3094653749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 msgr2=0x55be4a871850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 0x55be4a871850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 -- 172.21.15.178:0/3094653749 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be4a5cfc60 con 0x55be4a869780 2023-10-16T14:38:32.645 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.604+0000 7f510cb9a700 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5104015700 tx=0x7f5104006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27e23bc64c33264 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:32.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.641+0000 7f50f3fff700 1 -- 172.21.15.178:0/3094653749 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104013550 con 0x55be4a869780 2023-10-16T14:38:32.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.641+0000 7f50f3fff700 1 -- 172.21.15.178:0/3094653749 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51040136b0 con 0x55be4a869780 2023-10-16T14:38:32.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.641+0000 7f50f3fff700 1 -- 172.21.15.178:0/3094653749 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51040216d0 con 0x55be4a869780 2023-10-16T14:38:32.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.647+0000 7f5122f03b80 1 -- 172.21.15.178:0/3094653749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 msgr2=0x55be4a87a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:32.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.647+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5104015700 tx=0x7f5104006d80 comp rx=0 tx=0).stop 2023-10-16T14:38:32.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.675+0000 7f5122f03b80 1 -- 172.21.15.178:0/3094653749 shutdown_connections 2023-10-16T14:38:32.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.675+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a863b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.675+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a868e20 0x55be4a871850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.675+0000 7f5122f03b80 1 --2- 172.21.15.178:0/3094653749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a869780 0x55be4a87a260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.675+0000 7f5122f03b80 1 -- 172.21.15.178:0/3094653749 >> 172.21.15.178:0/3094653749 conn(0x55be4a796dd0 msgr2=0x55be4a880ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:32.704 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.701+0000 7f5122f03b80 1 -- 172.21.15.178:0/3094653749 shutdown_connections 2023-10-16T14:38:32.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.708+0000 7f5122f03b80 1 -- 172.21.15.178:0/3094653749 wait complete. 2023-10-16T14:38:32.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.709+0000 7f5122f03b80 1 -- start start 2023-10-16T14:38:32.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.711+0000 7f5122f03b80 1 Processor -- start 2023-10-16T14:38:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a877f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.751 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a8794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a8799e0 con 0x55be4a868e20 2023-10-16T14:38:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a879b50 con 0x55be4a869780 2023-10-16T14:38:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be4a879d10 con 0x55be4a86a620 2023-10-16T14:38:32.752 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.730+0000 7f5122f03b80 10 client.0 _ll_register_callbacks cb 0x7f510400ea90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:38:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:39642/0 (socket says 172.21.15.178:39642) 2023-10-16T14:38:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 -- 172.21.15.178:0/2159353514 learned_addr learned my addr 172.21.15.178:0/2159353514 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:38:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 -- 172.21.15.178:0/2159353514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 msgr2=0x55be4a8794a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:38:32.753 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a8794a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 -- 172.21.15.178:0/2159353514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 msgr2=0x55be4a877f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:38:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a877f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 -- 172.21.15.178:0/2159353514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be4a5e6de0 con 0x55be4a869780 2023-10-16T14:38:32.754 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.733+0000 7f510cb9a700 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5104014fb0 tx=0x7f5104014b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54662d9baee16044 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:38:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.756+0000 7f50f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:38:32.758 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.756+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104004040 con 0x55be4a869780 2023-10-16T14:38:32.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.756+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f51040041a0 con 0x55be4a869780 2023-10-16T14:38:32.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.777+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51000076f0 con 0x55be4a869780 2023-10-16T14:38:32.808 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.805+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5104004360 con 0x55be4a869780 2023-10-16T14:38:32.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.826+0000 7f5122f03b80 10 client.26186 fetch_fsmap learned FSMap version 31 2023-10-16T14:38:32.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.826+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55be4a60d250 con 0x55be4a869780 2023-10-16T14:38:32.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.828+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51000076f0 con 0x55be4a869780 2023-10-16T14:38:32.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.828+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 31) v1 ==== 7898+0+0 (secure 0 0 0) 0x7f5104044470 con 0x55be4a869780 2023-10-16T14:38:32.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.829+0000 7f5122f03b80 10 client.26186 fetch_fsmap finished waiting for FSMap version 31 2023-10-16T14:38:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.829+0000 7f5122f03b80 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:38:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.829+0000 7f5122f03b80 20 client.26186 populate_metadata read hostname 'smithi178' 2023-10-16T14:38:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.830+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a8d34b0 0x55be4a8d58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:38:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.830+0000 7f5122f03b80 4 client.26186 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:38:32.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.830+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55be4a7b4e50 con 0x55be4a8d34b0 2023-10-16T14:38:32.842 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:38:32.843 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-16T14:38:32.843 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:38:32.843 INFO:teuthology.orchestra.run.smithi178.stdout: "faf138f9-ada7-4b09-95d1-94c09def0776": { 2023-10-16T14:38:32.843 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:38:32.843 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "faf138f9-ada7-4b09-95d1-94c09def0776", 2023-10-16T14:38:32.844 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:38:32.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:38:32.844 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:38:32.844 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:38:32.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.832+0000 7f510d39b700 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a8d34b0 0x55be4a8d58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.836+0000 7f50f0ff9700 10 client.26186 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.836+0000 7f510d39b700 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a8d34b0 0x55be4a8d58d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c6cd7e64a083ba server_cookie=c63d12df0e52ed4b in_seq=0 out_seq=0 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.837+0000 7f50f0ff9700 1 -- 172.21.15.178:0/2159353514 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55be4a7b4e50 con 0x55be4a8d34b0 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.837+0000 7f50f0ff9700 10 client.26186 handle_command_reply: tid=0 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.837+0000 7f5122f03b80 1 client.26186 shutdown 2023-10-16T14:38:32.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.837+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 msgr2=0x55be4a878490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.837+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5104014fb0 tx=0x7f5104014b40 comp rx=0 tx=0).stop 2023-10-16T14:38:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 shutdown_connections 2023-10-16T14:38:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55be4a8d34b0 0x55be4a8d58d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55be4a86a620 0x55be4a8794a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55be4a868e20 0x55be4a877f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 --2- 172.21.15.178:0/2159353514 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55be4a869780 0x55be4a878490 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5104014fb0 tx=0x7f5104014b40 comp rx=0 tx=0).stop 2023-10-16T14:38:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 >> 172.21.15.178:0/2159353514 conn(0x55be4a796dd0 msgr2=0x55be4a87aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f50f0ff9700 0 client.26186 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:38:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 shutdown_connections 2023-10-16T14:38:32.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.838+0000 7f5122f03b80 1 -- 172.21.15.178:0/2159353514 wait complete. 2023-10-16T14:38:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.840+0000 7f5122f03b80 20 client.26186 trim_cache size 0 max 16384 2023-10-16T14:38:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.843+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 msgr2=0x7f50f40a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.843+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 0x7f50f40a3b40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5108072320 tx=0x7f5104009110 comp rx=0 tx=0).stop 2023-10-16T14:38:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.843+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 msgr2=0x7f51081144f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:38:32.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.843+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006fd0 tx=0x7f50f800b730 comp rx=0 tx=0).stop 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 shutdown_connections 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f50f40a1680 0x7f50f40a3b40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5108114eb0 0x7f51081b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f510810d1f0 0x7f51081144f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 --2- 172.21.15.178:0/3839507726 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5108114a30 0x7f51081b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:38:32.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 >> 172.21.15.178:0/3839507726 conn(0x7f510806dd30 msgr2=0x7f510810c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:38:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 shutdown_connections 2023-10-16T14:38:32.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:38:32.844+0000 7f510f5ff700 1 -- 172.21.15.178:0/3839507726 wait complete. 2023-10-16T14:38:33.034 INFO:tasks.cephfs.filesystem:scrub status for tag:faf138f9-ada7-4b09-95d1-94c09def0776 - {'path': '/', 'tag': 'faf138f9-ada7-4b09-95d1-94c09def0776', 'options': 'recursive,force'} 2023-10-16T14:38:34.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:33 smithi186 ceph-mon[134892]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2023-10-16T14:38:34.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:33 smithi187 ceph-mon[135297]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2023-10-16T14:38:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:33 smithi178 ceph-mon[98495]: pgmap v484: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 319 op/s 2023-10-16T14:38:35.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:35.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:35.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:38:36.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:35 smithi186 ceph-mon[134892]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2023-10-16T14:38:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:35 smithi187 ceph-mon[135297]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2023-10-16T14:38:36.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:35 smithi178 ceph-mon[98495]: pgmap v485: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 335 op/s 2023-10-16T14:38:37.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:36 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:38:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:36 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:38:37.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:36 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:38:37.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:38:37.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:38:37.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:36 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:38:37.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:36 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:38:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:36 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:38:37.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:36 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:38:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:37 smithi187 ceph-mon[135297]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2023-10-16T14:38:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:37 smithi186 ceph-mon[134892]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2023-10-16T14:38:38.342 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:37 smithi178 ceph-mon[98495]: pgmap v486: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 304 op/s 2023-10-16T14:38:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:39 smithi187 ceph-mon[135297]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 241 op/s 2023-10-16T14:38:40.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:39 smithi186 ceph-mon[134892]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 241 op/s 2023-10-16T14:38:40.327 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:39 smithi178 ceph-mon[98495]: pgmap v487: 129 pgs: 129 active+clean; 1.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 241 op/s 2023-10-16T14:38:41.041 INFO:tasks.workunit.client.0.smithi178.stdout: 279 702789 1523 496436 1422 321146 1934 2023-10-16T14:38:42.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:41 smithi187 ceph-mon[135297]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-16T14:38:42.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:41 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:42.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:41 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:38:42.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:41 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:38:42.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:41 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:38:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:41 smithi186 ceph-mon[134892]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-16T14:38:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:41 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:41 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:38:42.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:41 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:38:42.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:41 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:38:42.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:41 smithi178 ceph-mon[98495]: pgmap v488: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 306 op/s 2023-10-16T14:38:42.345 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:41 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:42.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:41 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:38:42.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:41 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:38:42.346 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:41 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:38:44.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:43 smithi187 ceph-mon[135297]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-10-16T14:38:44.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:43 smithi178 ceph-mon[98495]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-10-16T14:38:44.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:43 smithi186 ceph-mon[134892]: pgmap v489: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 227 op/s 2023-10-16T14:38:46.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:45 smithi187 ceph-mon[135297]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2023-10-16T14:38:46.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:45 smithi178 ceph-mon[98495]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2023-10-16T14:38:46.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:45 smithi186 ceph-mon[134892]: pgmap v490: 129 pgs: 129 active+clean; 1.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 276 op/s 2023-10-16T14:38:47.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:46 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:46 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:47.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:46 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:38:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:47 smithi186 ceph-mon[134892]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 383 op/s 2023-10-16T14:38:48.327 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:47 smithi178 ceph-mon[98495]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 383 op/s 2023-10-16T14:38:48.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:47 smithi187 ceph-mon[135297]: pgmap v491: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 383 op/s 2023-10-16T14:38:50.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:49 smithi178 ceph-mon[98495]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-10-16T14:38:50.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:49 smithi186 ceph-mon[134892]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-10-16T14:38:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:49 smithi187 ceph-mon[135297]: pgmap v492: 129 pgs: 129 active+clean; 1.1 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 432 op/s 2023-10-16T14:38:51.041 INFO:tasks.workunit.client.0.smithi178.stdout: 291 541722 365 381815 499 257521 1520 2023-10-16T14:38:51.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:51 smithi187 ceph-mon[135297]: pgmap v493: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-10-16T14:38:51.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:51 smithi178 ceph-mon[98495]: pgmap v493: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-10-16T14:38:51.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:51 smithi186 ceph-mon[134892]: pgmap v493: 129 pgs: 129 active+clean; 1.2 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 12 MiB/s wr, 698 op/s 2023-10-16T14:38:54.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:53 smithi187 ceph-mon[135297]: pgmap v494: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-16T14:38:54.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:53 smithi178 ceph-mon[98495]: pgmap v494: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-16T14:38:54.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:53 smithi186 ceph-mon[134892]: pgmap v494: 129 pgs: 129 active+clean; 1.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 629 op/s 2023-10-16T14:38:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:55 smithi187 ceph-mon[135297]: pgmap v495: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 750 op/s 2023-10-16T14:38:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:55 smithi178 ceph-mon[98495]: pgmap v495: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 750 op/s 2023-10-16T14:38:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:55 smithi186 ceph-mon[134892]: pgmap v495: 129 pgs: 129 active+clean; 1.2 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 750 op/s 2023-10-16T14:38:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:57 smithi187 ceph-mon[135297]: pgmap v496: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 978 op/s 2023-10-16T14:38:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:57 smithi186 ceph-mon[134892]: pgmap v496: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 978 op/s 2023-10-16T14:38:58.343 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:57 smithi178 ceph-mon[98495]: pgmap v496: 129 pgs: 129 active+clean; 1.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 978 op/s 2023-10-16T14:39:00.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:38:59 smithi187 ceph-mon[135297]: pgmap v497: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 15 MiB/s wr, 862 op/s 2023-10-16T14:39:00.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:38:59 smithi178 ceph-mon[98495]: pgmap v497: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 15 MiB/s wr, 862 op/s 2023-10-16T14:39:00.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:38:59 smithi186 ceph-mon[134892]: pgmap v497: 129 pgs: 129 active+clean; 1.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 15 MiB/s wr, 862 op/s 2023-10-16T14:39:01.041 INFO:tasks.workunit.client.0.smithi178.stdout: 298 434494 569 305997 301 216792 2012 2023-10-16T14:39:01.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:01 smithi187 ceph-mon[135297]: pgmap v498: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-10-16T14:39:01.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:01 smithi186 ceph-mon[134892]: pgmap v498: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-10-16T14:39:01.883 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:01 smithi178 ceph-mon[98495]: pgmap v498: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 968 op/s 2023-10-16T14:39:03.036 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:39:04.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:03 smithi186 ceph-mon[134892]: pgmap v499: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-10-16T14:39:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:03 smithi187 ceph-mon[135297]: pgmap v499: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-10-16T14:39:04.341 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:03 smithi178 ceph-mon[98495]: pgmap v499: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 10 MiB/s wr, 702 op/s 2023-10-16T14:39:06.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:05 smithi187 ceph-mon[135297]: pgmap v500: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 743 op/s 2023-10-16T14:39:06.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:05 smithi178 ceph-mon[98495]: pgmap v500: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 743 op/s 2023-10-16T14:39:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:05 smithi186 ceph-mon[134892]: pgmap v500: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 743 op/s 2023-10-16T14:39:08.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:07 smithi187 ceph-mon[135297]: pgmap v501: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 759 op/s 2023-10-16T14:39:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:07 smithi186 ceph-mon[134892]: pgmap v501: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 759 op/s 2023-10-16T14:39:08.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:07 smithi178 ceph-mon[98495]: pgmap v501: 129 pgs: 129 active+clean; 1.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 9.0 MiB/s wr, 759 op/s 2023-10-16T14:39:09.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.913+0000 7fafbabff700 1 -- 172.21.15.178:0/3366506627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410a820 msgr2=0x7fafb410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:09.920 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.913+0000 7fafbabff700 1 --2- 172.21.15.178:0/3366506627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410a820 0x7fafb410ccb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fafa801ac00 tx=0x7fafa801af10 comp rx=0 tx=0).stop 2023-10-16T14:39:09.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.918+0000 7fafbabff700 1 -- 172.21.15.178:0/3366506627 shutdown_connections 2023-10-16T14:39:09.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.918+0000 7fafbabff700 1 --2- 172.21.15.178:0/3366506627 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410d1f0 0x7fafb410d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:09.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.918+0000 7fafbabff700 1 --2- 172.21.15.178:0/3366506627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb410a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:09.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.918+0000 7fafbabff700 1 --2- 172.21.15.178:0/3366506627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410a820 0x7fafb410ccb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:09.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.918+0000 7fafbabff700 1 -- 172.21.15.178:0/3366506627 >> 172.21.15.178:0/3366506627 conn(0x7fafb406dd30 msgr2=0x7fafb40701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:09.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.926+0000 7fafbabff700 1 -- 172.21.15.178:0/3366506627 shutdown_connections 2023-10-16T14:39:09.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.926+0000 7fafbabff700 1 -- 172.21.15.178:0/3366506627 wait complete. 2023-10-16T14:39:09.940 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.937+0000 7fafbabff700 1 Processor -- start 2023-10-16T14:39:09.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.939+0000 7fafbabff700 1 -- start start 2023-10-16T14:39:09.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.939+0000 7fafbabff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb419d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:09.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.951+0000 7fafb899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb419d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:09.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.951+0000 7fafb899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb419d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36748/0 (socket says 172.21.15.178:36748) 2023-10-16T14:39:09.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.951+0000 7fafbabff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 0x7fafb419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:09.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.951+0000 7fafb899b700 1 -- 172.21.15.178:0/1432756750 learned_addr learned my addr 172.21.15.178:0/1432756750 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:09.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.951+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 0x7fafb41a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:09.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.952+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 0x7fafb419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:09.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.953+0000 7fafb919c700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 0x7fafb41a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:09.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.953+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41a4f90 con 0x7fafb410d1f0 2023-10-16T14:39:09.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.953+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41a50d0 con 0x7fafb410a820 2023-10-16T14:39:09.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.953+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb41a5260 con 0x7fafb4107eb0 2023-10-16T14:39:09.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.957+0000 7fafb3fff700 1 -- 172.21.15.178:0/1432756750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 msgr2=0x7fafb419d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:09.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.957+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb419d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.957+0000 7fafb3fff700 1 -- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 msgr2=0x7fafb41a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.957+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 0x7fafb41a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:09.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.957+0000 7fafb3fff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa801a600 con 0x7fafb410a820 2023-10-16T14:39:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.958+0000 7fafb919c700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 0x7fafb41a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:09.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:09.958+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 0x7fafb419d950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000c00 tx=0x7fafa8005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:10.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.005+0000 7fafb1ffb700 1 -- 172.21.15.178:0/1432756750 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa802d070 con 0x7fafb410a820 2023-10-16T14:39:10.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.005+0000 7fafb1ffb700 1 -- 172.21.15.178:0/1432756750 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafa8031410 con 0x7fafb410a820 2023-10-16T14:39:10.018 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.005+0000 7fafb1ffb700 1 -- 172.21.15.178:0/1432756750 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa801f3f0 con 0x7fafb410a820 2023-10-16T14:39:10.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.053+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb41a54e0 con 0x7fafb410a820 2023-10-16T14:39:10.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.053+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb41a5930 con 0x7fafb410a820 2023-10-16T14:39:10.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.109+0000 7fafb1ffb700 1 -- 172.21.15.178:0/1432756750 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fafa801cb40 con 0x7fafb410a820 2023-10-16T14:39:10.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.113+0000 7fafb1ffb700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 0x7faf9c0a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.120+0000 7fafb1ffb700 1 -- 172.21.15.178:0/1432756750 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fafa80deab0 con 0x7fafb410a820 2023-10-16T14:39:10.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.121+0000 7fafb899b700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 0x7faf9c0a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.137+0000 7fafb899b700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 0x7faf9c0a3b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009a90 tx=0x7fafa4009210 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:10.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:09 smithi187 ceph-mon[135297]: pgmap v502: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 497 op/s 2023-10-16T14:39:10.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.225+0000 7fafce5d0b80 1 Processor -- start 2023-10-16T14:39:10.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 -- start start 2023-10-16T14:39:10.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d73e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d73f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d6b2c70 con 0x56227d73de20 2023-10-16T14:39:10.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.239+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d697c60 con 0x56227d73e780 2023-10-16T14:39:10.248 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.240+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d695580 con 0x56227d73f620 2023-10-16T14:39:10.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.240+0000 7fafb919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.240+0000 7fafb919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36754/0 (socket says 172.21.15.178:36754) 2023-10-16T14:39:10.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.240+0000 7fafb919c700 1 -- 172.21.15.178:0/2635514447 learned_addr learned my addr 172.21.15.178:0/2635514447 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:10.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.243+0000 7fafb919c700 1 -- 172.21.15.178:0/2635514447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 msgr2=0x56227d73f040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:10.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.243+0000 7fafb919c700 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d73f040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.243+0000 7fafb919c700 1 -- 172.21.15.178:0/2635514447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 msgr2=0x56227d73e240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:10.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.243+0000 7fafb919c700 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d73e240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.243+0000 7fafb919c700 1 -- 172.21.15.178:0/2635514447 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56227d64e3f0 con 0x56227d73f620 2023-10-16T14:39:10.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.244+0000 7fafb919c700 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafac007ea0 tx=0x7fafac00d480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4d4b98e6087b1e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:10.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:09 smithi178 ceph-mon[98495]: pgmap v502: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 497 op/s 2023-10-16T14:39:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:09 smithi186 ceph-mon[134892]: pgmap v502: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 497 op/s 2023-10-16T14:39:10.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.291+0000 7faf9affd700 1 -- 172.21.15.178:0/2635514447 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac012440 con 0x56227d73f620 2023-10-16T14:39:10.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.291+0000 7faf9affd700 1 -- 172.21.15.178:0/2635514447 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafac012a60 con 0x56227d73f620 2023-10-16T14:39:10.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.291+0000 7faf9affd700 1 -- 172.21.15.178:0/2635514447 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac011920 con 0x56227d73f620 2023-10-16T14:39:10.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.296+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2635514447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 msgr2=0x56227d73faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:10.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.296+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fafac007ea0 tx=0x7fafac00d480 comp rx=0 tx=0).stop 2023-10-16T14:39:10.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.322+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2635514447 shutdown_connections 2023-10-16T14:39:10.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.322+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d73faa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.322+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d73e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.323+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2635514447 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d73f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.323+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2635514447 >> 172.21.15.178:0/2635514447 conn(0x56227d66bdd0 msgr2=0x56227d738fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:10.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.351+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2635514447 shutdown_connections 2023-10-16T14:39:10.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.354+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2635514447 wait complete. 2023-10-16T14:39:10.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.355+0000 7fafce5d0b80 1 -- start start 2023-10-16T14:39:10.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.356+0000 7fafce5d0b80 1 Processor -- start 2023-10-16T14:39:10.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.364+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.364+0000 7fafb899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.365+0000 7fafb899b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36770/0 (socket says 172.21.15.178:36770) 2023-10-16T14:39:10.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d7482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 0x56227d74f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d690e70 con 0x56227d73f620 2023-10-16T14:39:10.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d62d710 con 0x56227d73e780 2023-10-16T14:39:10.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d62d3b0 con 0x56227d73de20 2023-10-16T14:39:10.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.368+0000 7fafce5d0b80 10 client.0 _ll_register_callbacks cb 0x56227d48a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:10.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.370+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d7482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.371+0000 7fafb919c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 0x56227d74f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.371+0000 7fafb919c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 0x56227d74f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55308/0 (socket says 172.21.15.178:55308) 2023-10-16T14:39:10.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.371+0000 7fafb919c700 1 -- 172.21.15.178:0/1306279647 learned_addr learned my addr 172.21.15.178:0/1306279647 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:10.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.372+0000 7fafb899b700 1 -- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 msgr2=0x56227d7482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:10.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.372+0000 7fafb899b700 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d7482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.372+0000 7fafb899b700 1 -- 172.21.15.178:0/1306279647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 msgr2=0x56227d74f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:10.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.372+0000 7fafb899b700 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 0x56227d74f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.372+0000 7fafb899b700 1 -- 172.21.15.178:0/1306279647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac007b20 con 0x56227d73de20 2023-10-16T14:39:10.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.373+0000 7fafb899b700 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafa4015700 tx=0x7fafa4004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13db9aff1dfd1ef server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:10.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.400+0000 7faf93fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:39:10.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.400+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4015ca0 con 0x56227d73de20 2023-10-16T14:39:10.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.400+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fafa4015e00 con 0x56227d73de20 2023-10-16T14:39:10.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.411+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56227d695ef0 con 0x56227d73de20 2023-10-16T14:39:10.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.440+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafa4013450 con 0x56227d73de20 2023-10-16T14:39:10.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.499+0000 7fafce5d0b80 10 client.26167 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:10.503 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.499+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56227d4bbde0 con 0x56227d73de20 2023-10-16T14:39:10.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.499+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56227d695ef0 con 0x56227d73de20 2023-10-16T14:39:10.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.500+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fafa401f6c0 con 0x56227d73de20 2023-10-16T14:39:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 10 client.26167 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 10 client.26167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:10.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 20 client.26167 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d753b70 0x56227d755f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 4 client.26167 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.520+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56227d70dc10 con 0x56227d753b70 2023-10-16T14:39:10.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.526+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d753b70 0x56227d755f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:10.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.536+0000 7fafb3fff700 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d753b70 0x56227d755f90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca732b7d74a36d3 server_cookie=a93c786ddd1dd495 in_seq=0 out_seq=0 2023-10-16T14:39:10.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.539+0000 7faf93fff700 10 client.26167 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:10.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.539+0000 7faf93fff700 1 -- 172.21.15.178:0/1306279647 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56227d70dc10 con 0x56227d753b70 2023-10-16T14:39:10.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.539+0000 7faf93fff700 10 client.26167 handle_command_reply: tid=0 2023-10-16T14:39:10.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.556+0000 7fafce5d0b80 1 client.26167 shutdown 2023-10-16T14:39:10.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.599+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 msgr2=0x56227d747d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:10.603 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.599+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fafa4015700 tx=0x7fafa4004800 comp rx=0 tx=0).stop 2023-10-16T14:39:10.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 shutdown_connections 2023-10-16T14:39:10.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d753b70 0x56227d755f90 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73f620 0x56227d74f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.605 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73de20 0x56227d747d90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/1306279647 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73e780 0x56227d7482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:10.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 >> 172.21.15.178:0/1306279647 conn(0x56227d66bdd0 msgr2=0x56227d738b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:10.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.601+0000 7faf93fff700 0 client.26167 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:10.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.606+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 shutdown_connections 2023-10-16T14:39:10.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.607+0000 7fafce5d0b80 1 -- 172.21.15.178:0/1306279647 wait complete. 2023-10-16T14:39:10.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:10.641+0000 7fafce5d0b80 20 client.26167 trim_cache size 0 max 16384 2023-10-16T14:39:11.054 INFO:tasks.workunit.client.0.smithi178.stdout: 316 605102 1133 422514 945 293772 2391 2023-10-16T14:39:11.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.144+0000 7fafce5d0b80 1 Processor -- start 2023-10-16T14:39:11.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.156+0000 7fafce5d0b80 1 -- start start 2023-10-16T14:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.156+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d747b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.156+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 0x56227d74f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.156+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 0x56227d748080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.161+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d511920 con 0x56227d73de20 2023-10-16T14:39:11.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.161+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d73edd0 con 0x56227d73f620 2023-10-16T14:39:11.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.161+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d73eb60 con 0x56227d73e780 2023-10-16T14:39:11.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.177+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 0x56227d74f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:11.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.177+0000 7fafb3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 0x56227d74f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36786/0 (socket says 172.21.15.178:36786) 2023-10-16T14:39:11.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.177+0000 7fafb3fff700 1 -- 172.21.15.178:0/2607409015 learned_addr learned my addr 172.21.15.178:0/2607409015 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:11.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.177+0000 7fafb919c700 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 0x56227d748080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:11.184 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.179+0000 7fafb919c700 1 -- 172.21.15.178:0/2607409015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 msgr2=0x56227d74f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:11.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.179+0000 7fafb919c700 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 0x56227d74f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.179+0000 7fafb919c700 1 -- 172.21.15.178:0/2607409015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 msgr2=0x56227d747b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:11.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.179+0000 7fafb919c700 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d747b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.185 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.179+0000 7fafb919c700 1 -- 172.21.15.178:0/2607409015 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56227d4a4c60 con 0x56227d73f620 2023-10-16T14:39:11.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.181+0000 7fafb919c700 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 0x56227d748080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x56227d733da0 tx=0x7fafac007360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=459d50c19fcdaa70 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:11.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.192+0000 7faf9b7fe700 1 -- 172.21.15.178:0/2607409015 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac021070 con 0x56227d73f620 2023-10-16T14:39:11.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.192+0000 7faf9b7fe700 1 -- 172.21.15.178:0/2607409015 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafac00de20 con 0x56227d73f620 2023-10-16T14:39:11.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.192+0000 7faf9b7fe700 1 -- 172.21.15.178:0/2607409015 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac01c960 con 0x56227d73f620 2023-10-16T14:39:11.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.252+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2607409015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 msgr2=0x56227d748080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:11.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.252+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 0x56227d748080 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x56227d733da0 tx=0x7fafac007360 comp rx=0 tx=0).stop 2023-10-16T14:39:11.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.266+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2607409015 shutdown_connections 2023-10-16T14:39:11.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.266+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73f620 0x56227d748080 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.270 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.266+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73de20 0x56227d747b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.266+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/2607409015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73e780 0x56227d74f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.266+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2607409015 >> 172.21.15.178:0/2607409015 conn(0x56227d66bdd0 msgr2=0x56227d742600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:11.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.362+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2607409015 shutdown_connections 2023-10-16T14:39:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.387+0000 7fafce5d0b80 1 -- 172.21.15.178:0/2607409015 wait complete. 2023-10-16T14:39:11.396 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.388+0000 7fafce5d0b80 1 -- start start 2023-10-16T14:39:11.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.388+0000 7fafce5d0b80 1 Processor -- start 2023-10-16T14:39:11.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73de20 0x56227d74f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73e780 0x56227d74d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d74e0b0 con 0x56227d73e780 2023-10-16T14:39:11.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d74e220 con 0x56227d73de20 2023-10-16T14:39:11.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56227d74e3e0 con 0x56227d73f620 2023-10-16T14:39:11.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.445+0000 7fafce5d0b80 10 client.0 _ll_register_callbacks cb 0x7fafa400eb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:11.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.446+0000 7fafb919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:11.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.446+0000 7fafb919c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36794/0 (socket says 172.21.15.178:36794) 2023-10-16T14:39:11.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.446+0000 7fafb919c700 1 -- 172.21.15.178:0/3282497211 learned_addr learned my addr 172.21.15.178:0/3282497211 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:11.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb3fff700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73e780 0x56227d74d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:11.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb919c700 1 -- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73de20 msgr2=0x56227d74f020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:11.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb919c700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73de20 0x56227d74f020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb919c700 1 -- 172.21.15.178:0/3282497211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73e780 msgr2=0x56227d74d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:11.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb919c700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73e780 0x56227d74d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.448+0000 7fafb919c700 1 -- 172.21.15.178:0/3282497211 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56227d64e3f0 con 0x56227d73f620 2023-10-16T14:39:11.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.459+0000 7fafb919c700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafac00d4c0 tx=0x7fafac012720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f80b7c0193e181e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:11.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.463+0000 7faf93fff700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:39:11.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.529+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafac021040 con 0x56227d73f620 2023-10-16T14:39:11.534 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.529+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fafac01caf0 con 0x56227d73f620 2023-10-16T14:39:11.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.557+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafac012a60 con 0x56227d73f620 2023-10-16T14:39:11.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.586+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafac01ccb0 con 0x56227d73f620 2023-10-16T14:39:11.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.659+0000 7fafce5d0b80 10 client.26173 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:11.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.659+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x56227d4e2250 con 0x56227d73f620 2023-10-16T14:39:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fafac012a60 con 0x56227d73f620 2023-10-16T14:39:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fafac034450 con 0x56227d73f620 2023-10-16T14:39:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 10 client.26173 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:11.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 20 client.26173 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d7a8e60 0x56227d7ab280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 4 client.26173 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:39:11.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.665+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56227d689e50 con 0x56227d7a8e60 2023-10-16T14:39:11.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:11 smithi187 ceph-mon[135297]: pgmap v503: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-10-16T14:39:11.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.686+0000 7fafb899b700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d7a8e60 0x56227d7ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:11.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.688+0000 7faf93fff700 10 client.26173 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:11.692 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.688+0000 7fafb899b700 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d7a8e60 0x56227d7ab280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822c67aa59e58e8 server_cookie=ee8ce3ed8d6031fb in_seq=0 out_seq=0 2023-10-16T14:39:11.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.691+0000 7faf93fff700 1 -- 172.21.15.178:0/3282497211 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56227d689e50 con 0x56227d7a8e60 2023-10-16T14:39:11.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.691+0000 7faf93fff700 10 client.26173 handle_command_reply: tid=0 2023-10-16T14:39:11.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.691+0000 7fafce5d0b80 1 client.26173 shutdown 2023-10-16T14:39:11.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.732+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 msgr2=0x56227d74db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:11.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.732+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fafac00d4c0 tx=0x7fafac012720 comp rx=0 tx=0).stop 2023-10-16T14:39:11.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 shutdown_connections 2023-10-16T14:39:11.737 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x56227d7a8e60 0x56227d7ab280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d73f620 0x56227d74db70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56227d73de20 0x56227d74f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 --2- 172.21.15.178:0/3282497211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56227d73e780 0x56227d74d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 >> 172.21.15.178:0/3282497211 conn(0x56227d66bdd0 msgr2=0x56227d747b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:11.738 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7faf93fff700 0 client.26173 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.733+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 shutdown_connections 2023-10-16T14:39:11.739 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.734+0000 7fafce5d0b80 1 -- 172.21.15.178:0/3282497211 wait complete. 2023-10-16T14:39:11.739 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:39:11.739 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:39:11.740 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:39:11.740 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:39:11.740 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.734+0000 7fafce5d0b80 20 client.26173 trim_cache size 0 max 16384 2023-10-16T14:39:11.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 msgr2=0x7faf9c0a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:11.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 0x7faf9c0a3b10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009a90 tx=0x7fafa4009210 comp rx=0 tx=0).stop 2023-10-16T14:39:11.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 msgr2=0x7fafb419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 0x7fafb419d950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fafa8000c00 tx=0x7fafa8005750 comp rx=0 tx=0).stop 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 shutdown_connections 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7faf9c0a1650 0x7faf9c0a3b10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fafb410d1f0 0x7fafb41a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafb4107eb0 0x7fafb419d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.744 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 --2- 172.21.15.178:0/1432756750 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fafb410a820 0x7fafb419d950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.736+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 >> 172.21.15.178:0/1432756750 conn(0x7fafb406dd30 msgr2=0x7fafb410b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.737+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 shutdown_connections 2023-10-16T14:39:11.745 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:11.737+0000 7fafbabff700 1 -- 172.21.15.178:0/1432756750 wait complete. 2023-10-16T14:39:11.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:11 smithi178 ceph-mon[98495]: pgmap v503: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-10-16T14:39:11.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:11 smithi186 ceph-mon[134892]: pgmap v503: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-10-16T14:39:11.952 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:39:11.953 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:39:13.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.888+0000 7ff3799ff700 1 -- 172.21.15.178:0/3829783629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410a820 msgr2=0x7ff37410ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:13.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.888+0000 7ff3799ff700 1 --2- 172.21.15.178:0/3829783629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410a820 0x7ff37410ccb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff3680098f0 tx=0x7ff368009c00 comp rx=0 tx=0).stop 2023-10-16T14:39:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.896+0000 7ff3799ff700 1 -- 172.21.15.178:0/3829783629 shutdown_connections 2023-10-16T14:39:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.898+0000 7ff3799ff700 1 --2- 172.21.15.178:0/3829783629 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff37410d1f0 0x7ff37410d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:13.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.898+0000 7ff3799ff700 1 --2- 172.21.15.178:0/3829783629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff374107eb0 0x7ff37410a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:13.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.899+0000 7ff3799ff700 1 --2- 172.21.15.178:0/3829783629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410a820 0x7ff37410ccb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:13.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:13.900+0000 7ff3799ff700 1 -- 172.21.15.178:0/3829783629 >> 172.21.15.178:0/3829783629 conn(0x7ff37406dd20 msgr2=0x7ff3740701a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:14.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.050+0000 7ff3799ff700 1 -- 172.21.15.178:0/3829783629 shutdown_connections 2023-10-16T14:39:14.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.050+0000 7ff3799ff700 1 -- 172.21.15.178:0/3829783629 wait complete. 2023-10-16T14:39:14.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.073+0000 7ff3799ff700 1 Processor -- start 2023-10-16T14:39:14.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.094+0000 7ff3799ff700 1 -- start start 2023-10-16T14:39:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.094+0000 7ff3799ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff374107eb0 0x7ff3741a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.101+0000 7ff3799ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.103+0000 7ff3799ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410d1f0 0x7ff374079110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.105+0000 7ff3799ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374079710 con 0x7ff37410a820 2023-10-16T14:39:14.113 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.105+0000 7ff3799ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff374079880 con 0x7ff374107eb0 2023-10-16T14:39:14.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.105+0000 7ff3799ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740799f0 con 0x7ff37410d1f0 2023-10-16T14:39:14.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.111+0000 7ff3727fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:14.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.111+0000 7ff3727fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55350/0 (socket says 172.21.15.178:55350) 2023-10-16T14:39:14.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.111+0000 7ff3727fc700 1 -- 172.21.15.178:0/1896155097 learned_addr learned my addr 172.21.15.178:0/1896155097 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:14.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.112+0000 7ff3737fe700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410d1f0 0x7ff374079110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:14.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.112+0000 7ff372ffd700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff374107eb0 0x7ff3741a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:14.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.129+0000 7ff3727fc700 1 -- 172.21.15.178:0/1896155097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410d1f0 msgr2=0x7ff374079110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:14.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.129+0000 7ff3727fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410d1f0 0x7ff374079110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:14.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.129+0000 7ff3727fc700 1 -- 172.21.15.178:0/1896155097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff374107eb0 msgr2=0x7ff3741a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:14.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.129+0000 7ff3727fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff374107eb0 0x7ff3741a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:14.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.129+0000 7ff3727fc700 1 -- 172.21.15.178:0/1896155097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368009520 con 0x7ff37410a820 2023-10-16T14:39:14.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.133+0000 7ff3727fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff36800fb90 tx=0x7ff36800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:14.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.138+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36801c070 con 0x7ff37410a820 2023-10-16T14:39:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.138+0000 7ff3799ff700 1 -- 172.21.15.178:0/1896155097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff374079c70 con 0x7ff37410a820 2023-10-16T14:39:14.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.139+0000 7ff3799ff700 1 -- 172.21.15.178:0/1896155097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37407a070 con 0x7ff37410a820 2023-10-16T14:39:14.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.141+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff368005700 con 0x7ff37410a820 2023-10-16T14:39:14.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.142+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff368004a90 con 0x7ff37410a820 2023-10-16T14:39:14.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.144+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37411b5a0 con 0x7ff37410a820 2023-10-16T14:39:14.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.148+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ff368004bf0 con 0x7ff37410a820 2023-10-16T14:39:14.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.150+0000 7ff3789fd700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 0x7ff35c0a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:14.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.151+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff3680cde00 con 0x7ff37410a820 2023-10-16T14:39:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:13 smithi187 ceph-mon[135297]: pgmap v504: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2023-10-16T14:39:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.172+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ff3680971a0 con 0x7ff37410a820 2023-10-16T14:39:14.247 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.216+0000 7ff372ffd700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 0x7ff35c0a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:14.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:14.252+0000 7ff372ffd700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 0x7ff35c0a3af0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff364005e50 tx=0x7ff364005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:14.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:13 smithi178 ceph-mon[98495]: pgmap v504: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2023-10-16T14:39:14.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:13 smithi186 ceph-mon[134892]: pgmap v504: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 440 op/s 2023-10-16T14:39:16.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:15 smithi187 ceph-mon[135297]: pgmap v505: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-10-16T14:39:16.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:15 smithi178 ceph-mon[98495]: pgmap v505: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-10-16T14:39:16.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:15 smithi186 ceph-mon[134892]: pgmap v505: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 460 op/s 2023-10-16T14:39:18.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:17 smithi187 ceph-mon[135297]: pgmap v506: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2023-10-16T14:39:18.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:17 smithi186 ceph-mon[134892]: pgmap v506: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2023-10-16T14:39:18.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:17 smithi178 ceph-mon[98495]: pgmap v506: 129 pgs: 129 active+clean; 1.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 587 op/s 2023-10-16T14:39:20.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:19 smithi187 ceph-mon[135297]: pgmap v507: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 529 op/s 2023-10-16T14:39:20.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:19 smithi178 ceph-mon[98495]: pgmap v507: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 529 op/s 2023-10-16T14:39:20.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:19 smithi186 ceph-mon[134892]: pgmap v507: 129 pgs: 129 active+clean; 1.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 529 op/s 2023-10-16T14:39:21.054 INFO:tasks.workunit.client.0.smithi178.stdout: 324 511087 424 359185 406 249056 1430 2023-10-16T14:39:21.675 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:21 smithi187 ceph-mon[135297]: pgmap v508: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 784 op/s 2023-10-16T14:39:21.812 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:21 smithi178 ceph-mon[98495]: pgmap v508: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 784 op/s 2023-10-16T14:39:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:21 smithi186 ceph-mon[134892]: pgmap v508: 129 pgs: 129 active+clean; 1.3 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 784 op/s 2023-10-16T14:39:23.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.545+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff354005180 con 0x7ff37410a820 2023-10-16T14:39:23.554 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.551+0000 7ff3789fd700 1 -- 172.21.15.178:0/1896155097 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+11346 (secure 0 0 0) 0x7ff36809a7c0 con 0x7ff37410a820 2023-10-16T14:39:23.591 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:39:23.592 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:38:45.286354+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":13,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:39:23.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.632+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 msgr2=0x7ff35c0a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:23.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.632+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 0x7ff35c0a3af0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff364005e50 tx=0x7ff364005da0 comp rx=0 tx=0).stop 2023-10-16T14:39:23.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.632+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 msgr2=0x7ff3741a5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:23.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.632+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff36800fb90 tx=0x7ff36800bc60 comp rx=0 tx=0).stop 2023-10-16T14:39:23.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 shutdown_connections 2023-10-16T14:39:23.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ff35c0a1630 0x7ff35c0a3af0 secure :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff364005e50 tx=0x7ff364005da0 comp rx=0 tx=0).stop 2023-10-16T14:39:23.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37410d1f0 0x7ff374079110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:23.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ff374107eb0 0x7ff3741a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:23.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 --2- 172.21.15.178:0/1896155097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff37410a820 0x7ff3741a5df0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:23.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.645+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 >> 172.21.15.178:0/1896155097 conn(0x7ff37406dd20 msgr2=0x7ff374109600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.682+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 shutdown_connections 2023-10-16T14:39:23.686 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:23.683+0000 7ff35a7fc700 1 -- 172.21.15.178:0/1896155097 wait complete. 2023-10-16T14:39:23.698 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 33 2023-10-16T14:39:24.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:39:24.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:23 smithi187 ceph-mon[135297]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 744 op/s 2023-10-16T14:39:24.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:23 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1896155097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:39:24.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:23 smithi178 ceph-mon[98495]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 744 op/s 2023-10-16T14:39:24.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:23 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1896155097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:39:24.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:23 smithi186 ceph-mon[134892]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 744 op/s 2023-10-16T14:39:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:23 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1896155097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:39:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 -- 172.21.15.178:0/4224774542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c810a820 msgr2=0x7fe0c810ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:25.887 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/4224774542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c810a820 0x7fe0c810ccb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009990 tx=0x7fe0bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 -- 172.21.15.178:0/4224774542 shutdown_connections 2023-10-16T14:39:25.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/4224774542 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c810d1f0 0x7fe0c810d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/4224774542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0c8107eb0 0x7fe0c810a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/4224774542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c810a820 0x7fe0c810ccb0 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009990 tx=0x7fe0bc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.882+0000 7fe0d07ff700 1 -- 172.21.15.178:0/4224774542 >> 172.21.15.178:0/4224774542 conn(0x7fe0c806dd30 msgr2=0x7fe0c80701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:25.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.883+0000 7fe0d07ff700 1 -- 172.21.15.178:0/4224774542 shutdown_connections 2023-10-16T14:39:25.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.884+0000 7fe0d07ff700 1 -- 172.21.15.178:0/4224774542 wait complete. 2023-10-16T14:39:25.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.885+0000 7fe0d07ff700 1 Processor -- start 2023-10-16T14:39:25.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 -- start start 2023-10-16T14:39:25.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8107eb0 0x7fe0c81a5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0c810d1f0 0x7fe0c81a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81adb50 con 0x7fe0c810d1f0 2023-10-16T14:39:25.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81adcc0 con 0x7fe0c81ab190 2023-10-16T14:39:25.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.888+0000 7fe0d07ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c81ade80 con 0x7fe0c8107eb0 2023-10-16T14:39:25.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:57840/0 (socket says 172.21.15.178:57840) 2023-10-16T14:39:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 -- 172.21.15.178:0/1594437835 learned_addr learned my addr 172.21.15.178:0/1594437835 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8107eb0 0x7fe0c81a5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 -- 172.21.15.178:0/1594437835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8107eb0 msgr2=0x7fe0c81a5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:25.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8107eb0 0x7fe0c81a5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 -- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0c810d1f0 msgr2=0x7fe0c81a6170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:39:25.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0c810d1f0 0x7fe0c81a6170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.889+0000 7fe0ced9c700 1 -- 172.21.15.178:0/1594437835 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0095f0 con 0x7fe0c81ab190 2023-10-16T14:39:25.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.890+0000 7fe0ced9c700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007ee0 tx=0x7fe0c000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:25.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.890+0000 7fe0bb7fe700 1 -- 172.21.15.178:0/1594437835 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000ddd0 con 0x7fe0c81ab190 2023-10-16T14:39:25.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.890+0000 7fe0bb7fe700 1 -- 172.21.15.178:0/1594437835 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c000f040 con 0x7fe0c81ab190 2023-10-16T14:39:25.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.890+0000 7fe0bb7fe700 1 -- 172.21.15.178:0/1594437835 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c00043a0 con 0x7fe0c81ab190 2023-10-16T14:39:25.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.895+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c81ae160 con 0x7fe0c81ab190 2023-10-16T14:39:25.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.895+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c81ae5f0 con 0x7fe0c81ab190 2023-10-16T14:39:25.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.920+0000 7fe0bb7fe700 1 -- 172.21.15.178:0/1594437835 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe0c0015400 con 0x7fe0c81ab190 2023-10-16T14:39:25.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.924+0000 7fe0e41b0b80 1 Processor -- start 2023-10-16T14:39:25.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.926+0000 7fe0e41b0b80 1 -- start start 2023-10-16T14:39:25.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 0x5607fe5a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a9620 0x5607fe5a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe51cc70 con 0x5607fe5a8780 2023-10-16T14:39:25.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe501c60 con 0x5607fe5a7e20 2023-10-16T14:39:25.932 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.927+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe4ff580 con 0x5607fe5a9620 2023-10-16T14:39:25.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.928+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 0x5607fe5a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.928+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 0x5607fe5a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41606/0 (socket says 172.21.15.178:41606) 2023-10-16T14:39:25.933 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.928+0000 7fe0ce59b700 1 -- 172.21.15.178:0/1859884352 learned_addr learned my addr 172.21.15.178:0/1859884352 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:25.935 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.928+0000 7fe0bb7fe700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 0x7fe0b40a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.929+0000 7fe0bb7fe700 1 -- 172.21.15.178:0/1594437835 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe0c00ccda0 con 0x7fe0c81ab190 2023-10-16T14:39:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a9620 0x5607fe5a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1859884352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a9620 msgr2=0x5607fe5a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:25.936 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a9620 0x5607fe5a9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1859884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 msgr2=0x5607fe5a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 0x5607fe5a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.931+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1859884352 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607fe4b83f0 con 0x5607fe5a7e20 2023-10-16T14:39:25.937 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.932+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5a8240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8072320 tx=0x7fe0bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59cb1db4e4eb9806 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.932+0000 7fe0ced9c700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 0x7fe0b40a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.932+0000 7fe0b8ff9700 1 -- 172.21.15.178:0/1859884352 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00baa0 con 0x5607fe5a7e20 2023-10-16T14:39:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.932+0000 7fe0b8ff9700 1 -- 172.21.15.178:0/1859884352 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0bc005590 con 0x5607fe5a7e20 2023-10-16T14:39:25.938 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.932+0000 7fe0b8ff9700 1 -- 172.21.15.178:0/1859884352 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00f460 con 0x5607fe5a7e20 2023-10-16T14:39:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.936+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1859884352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 msgr2=0x5607fe5a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:25.939 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.936+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5a8240 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8072320 tx=0x7fe0bc000bc0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1859884352 shutdown_connections 2023-10-16T14:39:25.943 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a9620 0x5607fe5a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5a8240 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1859884352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a8780 0x5607fe5a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1859884352 >> 172.21.15.178:0/1859884352 conn(0x5607fe4d5dd0 msgr2=0x5607fe5a2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:25.944 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0ced9c700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 0x7fe0b40a3a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0099c80 tx=0x7fe0c000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1859884352 shutdown_connections 2023-10-16T14:39:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.940+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1859884352 wait complete. 2023-10-16T14:39:25.945 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.941+0000 7fe0e41b0b80 1 -- start start 2023-10-16T14:39:25.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.943+0000 7fe0e41b0b80 1 Processor -- start 2023-10-16T14:39:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a9620 0x5607fe5b91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe4fae70 con 0x5607fe5a9620 2023-10-16T14:39:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe497710 con 0x5607fe5a7e20 2023-10-16T14:39:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe4973b0 con 0x5607fe5a8780 2023-10-16T14:39:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.946+0000 7fe0e41b0b80 10 client.0 _ll_register_callbacks cb 0x5607fe2f4800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.947+0000 7fe0cdd9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a9620 0x5607fe5b91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.947+0000 7fe0cdd9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.947+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.947+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:55552/0 (socket says 172.21.15.178:55552) 2023-10-16T14:39:25.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.947+0000 7fe0ce59b700 1 -- 172.21.15.178:0/1132260642 learned_addr learned my addr 172.21.15.178:0/1132260642 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:25.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1132260642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 msgr2=0x5607fe5b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:25.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1132260642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a9620 msgr2=0x5607fe5b91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:25.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a9620 0x5607fe5b91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/1132260642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c810e240 con 0x5607fe5a7e20 2023-10-16T14:39:25.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5b1c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005fd0 tx=0x7fe0bc005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3651709ce02b9f66 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:25.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.951+0000 7fe0b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:39:25.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.952+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0052d0 con 0x5607fe5a7e20 2023-10-16T14:39:25.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.952+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe0bc00bab0 con 0x5607fe5a7e20 2023-10-16T14:39:25.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.953+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607fe4ffef0 con 0x5607fe5a7e20 2023-10-16T14:39:25.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.953+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0bc005430 con 0x5607fe5a7e20 2023-10-16T14:39:25.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.954+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:39:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.955+0000 7fe0e41b0b80 10 client.26210 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:25.959 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.955+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5607fe325de0 con 0x5607fe5a7e20 2023-10-16T14:39:25.960 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.957+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5607fe4ffef0 con 0x5607fe5a7e20 2023-10-16T14:39:25.961 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.957+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe0bc003d20 con 0x5607fe5a7e20 2023-10-16T14:39:25.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 10 client.26210 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 10 client.26210 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:39:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 20 client.26210 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe5bd4e0 0x5607fe5bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:25.963 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 4 client.26210 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:25.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.958+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607fe577c10 con 0x5607fe5bd4e0 2023-10-16T14:39:25.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.959+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe5bd4e0 0x5607fe5bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:25.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.959+0000 7fe0b1ffb700 10 client.26210 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:39:25.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.960+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe5bd4e0 0x5607fe5bf900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41454dd9d4e016eb server_cookie=a0724098d81932f in_seq=0 out_seq=0 2023-10-16T14:39:25.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.960+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/1132260642 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607fe577c10 con 0x5607fe5bd4e0 2023-10-16T14:39:25.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.960+0000 7fe0b1ffb700 10 client.26210 handle_command_reply: tid=0 2023-10-16T14:39:25.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.961+0000 7fe0e41b0b80 1 client.26210 shutdown 2023-10-16T14:39:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.967+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 msgr2=0x5607fe5b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:25.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.967+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5b1c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005fd0 tx=0x7fe0bc005760 comp rx=0 tx=0).stop 2023-10-16T14:39:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 shutdown_connections 2023-10-16T14:39:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe5bd4e0 0x5607fe5bf900 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a9620 0x5607fe5b91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a7e20 0x5607fe5b1c30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/1132260642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 >> 172.21.15.178:0/1132260642 conn(0x5607fe4d5dd0 msgr2=0x5607fe5a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0b1ffb700 0 client.26210 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:39:25.976 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 shutdown_connections 2023-10-16T14:39:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.971+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/1132260642 wait complete. 2023-10-16T14:39:25.977 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:25.974+0000 7fe0e41b0b80 20 client.26210 trim_cache size 0 max 16384 2023-10-16T14:39:26.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.040+0000 7fe0e41b0b80 1 Processor -- start 2023-10-16T14:39:26.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.042+0000 7fe0e41b0b80 1 -- start start 2023-10-16T14:39:26.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.042+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.043+0000 7fe0cdd9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.043+0000 7fe0cdd9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41626/0 (socket says 172.21.15.178:41626) 2023-10-16T14:39:26.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0cdd9a700 1 -- 172.21.15.178:0/2019384549 learned_addr learned my addr 172.21.15.178:0/2019384549 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:26.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe37b920 con 0x5607fe5a7e20 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe577c10 con 0x5607fe5a9620 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe4ffef0 con 0x5607fe5a8780 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0ce59b700 1 -- 172.21.15.178:0/2019384549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 msgr2=0x5607fe5acb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.044+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5acb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.045+0000 7fe0ce59b700 1 -- 172.21.15.178:0/2019384549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 msgr2=0x5607fe5ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.045+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.045+0000 7fe0ce59b700 1 -- 172.21.15.178:0/2019384549 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607fe30ec60 con 0x5607fe5a8780 2023-10-16T14:39:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.045+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:39:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.045+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b9170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5607fe5bfee0 tx=0x7fe0c401bbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c128d2a8399c332f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:26.052 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.046+0000 7fe0b97fa700 1 -- 172.21.15.178:0/2019384549 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4016040 con 0x5607fe5a8780 2023-10-16T14:39:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.046+0000 7fe0b97fa700 1 -- 172.21.15.178:0/2019384549 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0c401b470 con 0x5607fe5a8780 2023-10-16T14:39:26.053 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.046+0000 7fe0b97fa700 1 -- 172.21.15.178:0/2019384549 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4005260 con 0x5607fe5a8780 2023-10-16T14:39:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.047+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 msgr2=0x5607fe5b9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.047+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b9170 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5607fe5bfee0 tx=0x7fe0c401bbe0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.055 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.048+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 shutdown_connections 2023-10-16T14:39:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.048+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.048+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.048+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/2019384549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b9170 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.048+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 >> 172.21.15.178:0/2019384549 conn(0x5607fe4d5dd0 msgr2=0x5607fe5b9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:26.056 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.049+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 shutdown_connections 2023-10-16T14:39:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.049+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/2019384549 wait complete. 2023-10-16T14:39:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.050+0000 7fe0e41b0b80 1 -- start start 2023-10-16T14:39:26.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.050+0000 7fe0e41b0b80 1 Processor -- start 2023-10-16T14:39:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.052+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5b8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.058 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe5b7f10 con 0x5607fe5a7e20 2023-10-16T14:39:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe5b8080 con 0x5607fe5a9620 2023-10-16T14:39:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607fe5b8240 con 0x5607fe5a8780 2023-10-16T14:39:26.059 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0e41b0b80 10 client.0 _ll_register_callbacks cb 0x7fe0bc00b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0ce59b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:55568/0 (socket says 172.21.15.178:55568) 2023-10-16T14:39:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0ce59b700 1 -- 172.21.15.178:0/248843737 learned_addr learned my addr 172.21.15.178:0/248843737 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:26.060 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.053+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5b8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 -- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 msgr2=0x5607fe5b79d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5b79d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 -- 172.21.15.178:0/248843737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 msgr2=0x5607fe5b8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:26.061 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5b8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 -- 172.21.15.178:0/248843737 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607fe325de0 con 0x5607fe5a8780 2023-10-16T14:39:26.062 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.056+0000 7fe0ce59b700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4008320 tx=0x7fe0c4004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bcb2c4d2067ec12 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:26.070 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.060+0000 7fe0b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.060+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4016040 con 0x5607fe5a8780 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.060+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe0c4004f20 con 0x5607fe5a8780 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.061+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0bc005590 con 0x5607fe5a8780 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.061+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0c4005120 con 0x5607fe5a8780 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.067+0000 7fe0e41b0b80 10 client.26197 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.067+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fe0c4008af0 con 0x5607fe5a8780 2023-10-16T14:39:26.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.067+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0bc005590 con 0x5607fe5a8780 2023-10-16T14:39:26.074 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fe0c403c020 con 0x5607fe5a8780 2023-10-16T14:39:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 10 client.26197 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 10 client.26197 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:39:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 20 client.26197 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe6129b0 0x5607fe614dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.075 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 4 client.26197 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:39:26.076 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.071+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607fe4f3e50 con 0x5607fe6129b0 2023-10-16T14:39:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.077+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe6129b0 0x5607fe614dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.080+0000 7fe0b1ffb700 10 client.26197 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:39:26.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.080+0000 7fe0cdd9a700 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe6129b0 0x5607fe614dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c11f40b55128a9e7 server_cookie=56fe033f21178d0a in_seq=0 out_seq=0 2023-10-16T14:39:26.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0b1ffb700 1 -- 172.21.15.178:0/248843737 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607fe4f3e50 con 0x5607fe6129b0 2023-10-16T14:39:26.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0b1ffb700 10 client.26197 handle_command_reply: tid=0 2023-10-16T14:39:26.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 client.26197 shutdown 2023-10-16T14:39:26.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 msgr2=0x5607fe5b7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4008320 tx=0x7fe0c4004a70 comp rx=0 tx=0).stop 2023-10-16T14:39:26.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 shutdown_connections 2023-10-16T14:39:26.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x5607fe6129b0 0x5607fe614dd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5607fe5a9620 0x5607fe5b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5607fe5a7e20 0x5607fe5b8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 --2- 172.21.15.178:0/248843737 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607fe5a8780 0x5607fe5b7490 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0b1ffb700 0 client.26197 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:39:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.081+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 >> 172.21.15.178:0/248843737 conn(0x5607fe4d5dd0 msgr2=0x5607fe5ac5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.082+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 shutdown_connections 2023-10-16T14:39:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.082+0000 7fe0e41b0b80 1 -- 172.21.15.178:0/248843737 wait complete. 2023-10-16T14:39:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.082+0000 7fe0e41b0b80 20 client.26197 trim_cache size 0 max 16384 2023-10-16T14:39:26.090 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:39:26.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.089+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 msgr2=0x7fe0b40a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.089+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 0x7fe0b40a3a70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0099c80 tx=0x7fe0c000b040 comp rx=0 tx=0).stop 2023-10-16T14:39:26.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.089+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 msgr2=0x7fe0c81ad580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.093 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.089+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0007ee0 tx=0x7fe0c000d3b0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 shutdown_connections 2023-10-16T14:39:26.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe0b40a15b0 0x7fe0b40a3a70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe0c81ab190 0x7fe0c81ad580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0c8107eb0 0x7fe0c81a5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.098 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 --2- 172.21.15.178:0/1594437835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0c810d1f0 0x7fe0c81a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 >> 172.21.15.178:0/1594437835 conn(0x7fe0c806dd30 msgr2=0x7fe0c8109790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:26.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.094+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 shutdown_connections 2023-10-16T14:39:26.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.095+0000 7fe0d07ff700 1 -- 172.21.15.178:0/1594437835 wait complete. 2023-10-16T14:39:26.173 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:25 smithi178 ceph-mon[98495]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-10-16T14:39:26.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:25 smithi187 ceph-mon[135297]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-10-16T14:39:26.178 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:39:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:25 smithi186 ceph-mon[134892]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 803 op/s 2023-10-16T14:39:26.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.855+0000 7f38ea19d700 1 -- 172.21.15.178:0/1283372854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4072b60 msgr2=0x7f38e4072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.855+0000 7f38ea19d700 1 --2- 172.21.15.178:0/1283372854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4072b60 0x7f38e4072f80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009990 tx=0x7f38d4009ca0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.856+0000 7f38ea19d700 1 -- 172.21.15.178:0/1283372854 shutdown_connections 2023-10-16T14:39:26.891 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.856+0000 7f38ea19d700 1 --2- 172.21.15.178:0/1283372854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e407acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.856+0000 7f38ea19d700 1 --2- 172.21.15.178:0/1283372854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4072b60 0x7f38e4072f80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.856+0000 7f38ea19d700 1 --2- 172.21.15.178:0/1283372854 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e4075e90 0x7f38e4078320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.856+0000 7f38ea19d700 1 -- 172.21.15.178:0/1283372854 >> 172.21.15.178:0/1283372854 conn(0x7f38e406dc20 msgr2=0x7f38e4070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:26.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.866+0000 7f38ea19d700 1 -- 172.21.15.178:0/1283372854 shutdown_connections 2023-10-16T14:39:26.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.866+0000 7f38ea19d700 1 -- 172.21.15.178:0/1283372854 wait complete. 2023-10-16T14:39:26.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.876+0000 7f38ea19d700 1 Processor -- start 2023-10-16T14:39:26.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.882+0000 7f38ea19d700 1 -- start start 2023-10-16T14:39:26.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e4072b60 0x7f38e41b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4075e90 0x7f38e41b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41b5bc0 con 0x7f38e4078860 2023-10-16T14:39:26.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41b5d30 con 0x7f38e4072b60 2023-10-16T14:39:26.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.883+0000 7f38ea19d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41b5ef0 con 0x7f38e4075e90 2023-10-16T14:39:26.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.884+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.884+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41632/0 (socket says 172.21.15.178:41632) 2023-10-16T14:39:26.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.884+0000 7f38e999c700 1 -- 172.21.15.178:0/2715610822 learned_addr learned my addr 172.21.15.178:0/2715610822 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:26.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.886+0000 7f38e999c700 1 -- 172.21.15.178:0/2715610822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4075e90 msgr2=0x7f38e41b3130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:39:26.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.886+0000 7f38e999c700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4075e90 0x7f38e41b3130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.886+0000 7f38e999c700 1 -- 172.21.15.178:0/2715610822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e4072b60 msgr2=0x7f38e41b0be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T14:39:26.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.886+0000 7f38e999c700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e4072b60 0x7f38e41b0be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.886+0000 7f38e999c700 1 -- 172.21.15.178:0/2715610822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40095f0 con 0x7f38e4078860 2023-10-16T14:39:26.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.887+0000 7f38e999c700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b780 tx=0x7f38e000bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:26.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.899+0000 7f38da7fc700 1 -- 172.21.15.178:0/2715610822 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e000f640 con 0x7f38e4078860 2023-10-16T14:39:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.899+0000 7f38da7fc700 1 -- 172.21.15.178:0/2715610822 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e000fc60 con 0x7f38e4078860 2023-10-16T14:39:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.899+0000 7f38da7fc700 1 -- 172.21.15.178:0/2715610822 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e0017e00 con 0x7f38e4078860 2023-10-16T14:39:26.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.899+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e41b61d0 con 0x7f38e4078860 2023-10-16T14:39:26.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.904+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41b6710 con 0x7f38e4078860 2023-10-16T14:39:26.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.913+0000 7f38da7fc700 1 -- 172.21.15.178:0/2715610822 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f38e00213f0 con 0x7f38e4078860 2023-10-16T14:39:26.946 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.943+0000 7f38fedc2b80 1 Processor -- start 2023-10-16T14:39:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.951+0000 7f38fedc2b80 1 -- start start 2023-10-16T14:39:26.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.951+0000 7f38da7fc700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 0x7f38d00a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.952+0000 7f38da7fc700 1 -- 172.21.15.178:0/2715610822 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f38e000fdd0 con 0x7f38e4078860 2023-10-16T14:39:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef047c70 con 0x55f6ef0d2fb0 2023-10-16T14:39:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef02cc60 con 0x55f6ef0d4800 2023-10-16T14:39:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.955+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef02a580 con 0x55f6ef0d3910 2023-10-16T14:39:26.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.959+0000 7f38e999c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.959+0000 7f38e999c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:57880/0 (socket says 172.21.15.178:57880) 2023-10-16T14:39:26.966 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.959+0000 7f38e999c700 1 -- 172.21.15.178:0/1916946011 learned_addr learned my addr 172.21.15.178:0/1916946011 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:26.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.961+0000 7f38e899a700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.962+0000 7f38e999c700 1 -- 172.21.15.178:0/1916946011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 msgr2=0x55f6ef0d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:26.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.963+0000 7f38e919b700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.969+0000 7f38e999c700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.969+0000 7f38e999c700 1 -- 172.21.15.178:0/1916946011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 msgr2=0x55f6ef0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.969+0000 7f38e999c700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:26.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.970+0000 7f38e999c700 1 -- 172.21.15.178:0/1916946011 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6eefe33f0 con 0x55f6ef0d4800 2023-10-16T14:39:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.971+0000 7f38e899a700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:39:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.971+0000 7f38e899a700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 0x7f38d00a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:26.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.971+0000 7f38e999c700 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38e00a2650 tx=0x7f38e009a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49adc3c7379656f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:26.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.982+0000 7f38cf7fe700 1 -- 172.21.15.178:0/1916946011 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00b2070 con 0x55f6ef0d4800 2023-10-16T14:39:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.982+0000 7f38cf7fe700 1 -- 172.21.15.178:0/1916946011 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e009e940 con 0x55f6ef0d4800 2023-10-16T14:39:26.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.982+0000 7f38cf7fe700 1 -- 172.21.15.178:0/1916946011 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00970c0 con 0x55f6ef0d4800 2023-10-16T14:39:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.993+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1916946011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 msgr2=0x55f6ef0d4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:26.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:26.993+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f38e00a2650 tx=0x7f38e009a1d0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.013 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.003+0000 7f38e899a700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 0x7f38d00a3ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d4d0 tx=0x7f38dc006ce0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:27.019 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1916946011 shutdown_connections 2023-10-16T14:39:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0d4c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1916946011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1916946011 >> 172.21.15.178:0/1916946011 conn(0x55f6ef000dd0 msgr2=0x55f6ef0cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:27.020 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.010+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1916946011 shutdown_connections 2023-10-16T14:39:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.011+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1916946011 wait complete. 2023-10-16T14:39:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.011+0000 7f38fedc2b80 1 -- start start 2023-10-16T14:39:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.013+0000 7f38fedc2b80 1 Processor -- start 2023-10-16T14:39:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.014+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0dce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.021 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.015+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0dd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.015+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.016+0000 7f38e899a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0dd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.016+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef025e70 con 0x55f6ef0d4800 2023-10-16T14:39:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.016+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eefc2710 con 0x55f6ef0d3910 2023-10-16T14:39:27.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.016+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eefc23b0 con 0x55f6ef0d2fb0 2023-10-16T14:39:27.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.016+0000 7f38fedc2b80 10 client.0 _ll_register_callbacks cb 0x55f6eee1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:27.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.017+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.017+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41660/0 (socket says 172.21.15.178:41660) 2023-10-16T14:39:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.017+0000 7f38e999c700 1 -- 172.21.15.178:0/3208581313 learned_addr learned my addr 172.21.15.178:0/3208581313 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.018+0000 7f38e899a700 1 -- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 msgr2=0x55f6ef0dce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:27.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e919b700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0dce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e899a700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0dce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e899a700 1 -- 172.21.15.178:0/3208581313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 msgr2=0x55f6ef0e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:27.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e899a700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e899a700 1 -- 172.21.15.178:0/3208581313 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e41b6710 con 0x55f6ef0d3910 2023-10-16T14:39:27.040 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e999c700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:39:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e919b700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0dce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38e899a700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0dd370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d940 tx=0x7f38dc006c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ece99d8a074f22b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:39:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.019+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc013070 con 0x55f6ef0d3910 2023-10-16T14:39:27.041 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.020+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f38dc00de50 con 0x55f6ef0d3910 2023-10-16T14:39:27.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.026+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6ef02aef0 con 0x55f6ef0d3910 2023-10-16T14:39:27.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38dc01c820 con 0x55f6ef0d3910 2023-10-16T14:39:27.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 10 client.26216 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:27.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55f6eee50de0 con 0x55f6ef0d3910 2023-10-16T14:39:27.042 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6ef02aef0 con 0x55f6ef0d3910 2023-10-16T14:39:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f38dc022020 con 0x55f6ef0d3910 2023-10-16T14:39:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 10 client.26216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:39:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 20 client.26216 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:27.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef0e8cd0 0x55f6ef0eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.030+0000 7f38fedc2b80 4 client.26216 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:27.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.031+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6ef0a2c10 con 0x55f6ef0e8cd0 2023-10-16T14:39:27.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.031+0000 7f38e919b700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef0e8cd0 0x55f6ef0eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.044 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.032+0000 7f38ccff9700 10 client.26216 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:39:27.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.032+0000 7f38e919b700 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef0e8cd0 0x55f6ef0eb0f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=11ad093f27c03cfe server_cookie=d4149fcb5bbebd5b in_seq=0 out_seq=0 2023-10-16T14:39:27.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.033+0000 7f38ccff9700 1 -- 172.21.15.178:0/3208581313 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f6ef0a2c10 con 0x55f6ef0e8cd0 2023-10-16T14:39:27.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.033+0000 7f38ccff9700 10 client.26216 handle_command_reply: tid=0 2023-10-16T14:39:27.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.034+0000 7f38fedc2b80 1 client.26216 shutdown 2023-10-16T14:39:27.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.034+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 msgr2=0x55f6ef0dd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.034+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0dd370 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d940 tx=0x7f38dc006c70 comp rx=0 tx=0).stop 2023-10-16T14:39:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 shutdown_connections 2023-10-16T14:39:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef0e8cd0 0x55f6ef0eb0f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.046 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0dce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/3208581313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0dd370 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 >> 172.21.15.178:0/3208581313 conn(0x55f6ef000dd0 msgr2=0x55f6ef0cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38ccff9700 0 client.26216 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:39:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 shutdown_connections 2023-10-16T14:39:27.047 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.035+0000 7f38fedc2b80 1 -- 172.21.15.178:0/3208581313 wait complete. 2023-10-16T14:39:27.048 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.036+0000 7f38fedc2b80 20 client.26216 trim_cache size 0 max 16384 2023-10-16T14:39:27.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.217+0000 7f38fedc2b80 1 Processor -- start 2023-10-16T14:39:27.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.232+0000 7f38fedc2b80 1 -- start start 2023-10-16T14:39:27.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.233+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.239+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.239+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.239+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6eeea6920 con 0x55f6ef0d2fb0 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.239+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef0d2bf0 con 0x55f6ef0d4800 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.239+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef0ca700 con 0x55f6ef0d3910 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.242+0000 7f38e999c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.242+0000 7f38e999c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:57900/0 (socket says 172.21.15.178:57900) 2023-10-16T14:39:27.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.242+0000 7f38e999c700 1 -- 172.21.15.178:0/1521798396 learned_addr learned my addr 172.21.15.178:0/1521798396 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.243+0000 7f38e999c700 1 -- 172.21.15.178:0/1521798396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 msgr2=0x55f6ef0e4370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.243+0000 7f38e999c700 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e4370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.243+0000 7f38e999c700 1 -- 172.21.15.178:0/1521798396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 msgr2=0x55f6ef0cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.243+0000 7f38e999c700 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.258 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.243+0000 7f38e999c700 1 -- 172.21.15.178:0/1521798396 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6eee39c60 con 0x55f6ef0d4800 2023-10-16T14:39:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.244+0000 7f38e999c700 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e00ba040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e029084fcca928 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.245+0000 7f38cffff700 1 -- 172.21.15.178:0/1521798396 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00b2040 con 0x55f6ef0d4800 2023-10-16T14:39:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.245+0000 7f38cffff700 1 -- 172.21.15.178:0/1521798396 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e00ba0b0 con 0x55f6ef0d4800 2023-10-16T14:39:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.245+0000 7f38cffff700 1 -- 172.21.15.178:0/1521798396 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00850b0 con 0x55f6ef0d4800 2023-10-16T14:39:27.259 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.247+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1521798396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 msgr2=0x55f6ef0cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:27.260 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.247+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f38e0000c00 tx=0x7f38e00ba040 comp rx=0 tx=0).stop 2023-10-16T14:39:27.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.268+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1521798396 shutdown_connections 2023-10-16T14:39:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.268+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d4800 0x55f6ef0cfa80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.268+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.268+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/1521798396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.268+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1521798396 >> 172.21.15.178:0/1521798396 conn(0x55f6ef000dd0 msgr2=0x55f6ef0dba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:27.302 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.298+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1521798396 shutdown_connections 2023-10-16T14:39:27.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.298+0000 7f38fedc2b80 1 -- 172.21.15.178:0/1521798396 wait complete. 2023-10-16T14:39:27.305 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.299+0000 7f38fedc2b80 1 -- start start 2023-10-16T14:39:27.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.299+0000 7f38fedc2b80 1 Processor -- start 2023-10-16T14:39:27.306 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.301+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.302+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.307 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.302+0000 7f38fedc2b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.302+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef0e36a0 con 0x55f6ef0d4800 2023-10-16T14:39:27.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38fedc2b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef0e3810 con 0x55f6ef0d3910 2023-10-16T14:39:27.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38e999c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:41682/0 (socket says 172.21.15.178:41682) 2023-10-16T14:39:27.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38e999c700 1 -- 172.21.15.178:0/2273430987 learned_addr learned my addr 172.21.15.178:0/2273430987 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:27.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38e919b700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.312 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.303+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ef0e39d0 con 0x55f6ef0d2fb0 2023-10-16T14:39:27.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38fedc2b80 10 client.0 _ll_register_callbacks cb 0x55f6eee1f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:27.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38e999c700 1 -- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 msgr2=0x55f6ef0e1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:27.313 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38e999c700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0e1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38e999c700 1 -- 172.21.15.178:0/2273430987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 msgr2=0x55f6ef0e2150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38e999c700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e2150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.314 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.304+0000 7f38e999c700 1 -- 172.21.15.178:0/2273430987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6eefe33f0 con 0x55f6ef0d4800 2023-10-16T14:39:27.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.305+0000 7f38e999c700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38e009a730 tx=0x7f38e0017860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a840930f2dede91c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:27.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.305+0000 7f38ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:39:27.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.306+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38e009e940 con 0x55f6ef0d4800 2023-10-16T14:39:27.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.311+0000 7f38e919b700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:27.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.312+0000 7f38ccff9700 1 -- 172.21.15.178:0/2273430987 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00b2070 con 0x55f6ef0d4800 2023-10-16T14:39:27.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.312+0000 7f38ccff9700 1 -- 172.21.15.178:0/2273430987 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f38e0099ce0 con 0x55f6ef0d4800 2023-10-16T14:39:27.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.313+0000 7f38ccff9700 1 -- 172.21.15.178:0/2273430987 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38e0099ea0 con 0x55f6ef0d4800 2023-10-16T14:39:27.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.317+0000 7f38fedc2b80 10 client.16140 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:27.325 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.317+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({fsmap=33}) v3 -- 0x55f6eee77250 con 0x55f6ef0d4800 2023-10-16T14:39:27.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38ccff9700 1 -- 172.21.15.178:0/2273430987 <== mon.0 v2:172.21.15.178:3300/0 4 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f38e00b4070 con 0x55f6ef0d4800 2023-10-16T14:39:27.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:27.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:39:27.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 20 client.16140 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:27.335 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef15eea0 0x55f6ef161290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:27.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 4 client.16140 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:39:27.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.331+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f38e009e940 con 0x55f6ef15eea0 2023-10-16T14:39:27.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.332+0000 7f38e919b700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef15eea0 0x55f6ef161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:27.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.332+0000 7f38e919b700 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef15eea0 0x55f6ef161290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77a32501911a39e5 server_cookie=6f26d97682843a03 in_seq=0 out_seq=0 2023-10-16T14:39:27.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.335+0000 7f38ccff9700 10 client.16140 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:39:27.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.347+0000 7f38ccff9700 1 -- 172.21.15.178:0/2273430987 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f38e009e940 con 0x55f6ef15eea0 2023-10-16T14:39:27.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.347+0000 7f38ccff9700 10 client.16140 handle_command_reply: tid=0 2023-10-16T14:39:27.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.347+0000 7f38fedc2b80 1 client.16140 shutdown 2023-10-16T14:39:27.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.347+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 msgr2=0x55f6ef0e3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:27.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.347+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f38e009a730 tx=0x7f38e0017860 comp rx=0 tx=0).stop 2023-10-16T14:39:27.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 shutdown_connections 2023-10-16T14:39:27.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x55f6ef15eea0 0x55f6ef161290 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38ccff9700 0 client.16140 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:39:27.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55f6ef0d4800 0x55f6ef0e3160 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f6ef0d2fb0 0x55f6ef0e1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 --2- 172.21.15.178:0/2273430987 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55f6ef0d3910 0x55f6ef0e2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.348+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 >> 172.21.15.178:0/2273430987 conn(0x55f6ef000dd0 msgr2=0x55f6ef0cf540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:27.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.352+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 shutdown_connections 2023-10-16T14:39:27.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.352+0000 7f38fedc2b80 1 -- 172.21.15.178:0/2273430987 wait complete. 2023-10-16T14:39:27.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.354+0000 7f38fedc2b80 20 client.16140 trim_cache size 0 max 16384 2023-10-16T14:39:27.358 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:39:27.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.357+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 msgr2=0x7f38d00a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:27.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.357+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 0x7f38d00a3ba0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f38dc00d4d0 tx=0x7f38dc006ce0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 msgr2=0x7f38e41b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:27.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f38e000b780 tx=0x7f38e000bb40 comp rx=0 tx=0).stop 2023-10-16T14:39:27.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 shutdown_connections 2023-10-16T14:39:27.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f38d00a16e0 0x7f38d00a3ba0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e4078860 0x7f38e41b5680 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f38e4072b60 0x7f38e41b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 --2- 172.21.15.178:0/2715610822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38e4075e90 0x7f38e41b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:27.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.358+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 >> 172.21.15.178:0/2715610822 conn(0x7f38e406dc20 msgr2=0x7f38e40775d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:27.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.359+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 shutdown_connections 2023-10-16T14:39:27.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:27.359+0000 7f38ea19d700 1 -- 172.21.15.178:0/2715610822 wait complete. 2023-10-16T14:39:27.721 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:39:28.072 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:27 smithi178 ceph-mon[98495]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 925 op/s 2023-10-16T14:39:28.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:27 smithi187 ceph-mon[135297]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 925 op/s 2023-10-16T14:39:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:27 smithi186 ceph-mon[134892]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 12 MiB/s wr, 925 op/s 2023-10-16T14:39:30.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:29 smithi187 ceph-mon[135297]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 768 op/s 2023-10-16T14:39:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:29 smithi178 ceph-mon[98495]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 768 op/s 2023-10-16T14:39:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:29 smithi186 ceph-mon[134892]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 768 op/s 2023-10-16T14:39:31.063 INFO:tasks.workunit.client.0.smithi178.stdout: 325 483437 61 341024 47 246138 1411 2023-10-16T14:39:31.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:31 smithi187 ceph-mon[135297]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-16T14:39:31.820 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:31 smithi178 ceph-mon[98495]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-16T14:39:31.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:31 smithi186 ceph-mon[134892]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 805 op/s 2023-10-16T14:39:33.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.053+0000 7fdb60d9d700 1 -- 172.21.15.178:0/526467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 msgr2=0x7fdb540ab020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.057 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.054+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/526467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb540ab020 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0644a0 tx=0x7fdb5c04f5f0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.084 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.077+0000 7fdb60d9d700 1 -- 172.21.15.178:0/526467981 shutdown_connections 2023-10-16T14:39:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.077+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/526467981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb540ab020 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.077+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/526467981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a4c90 0x7fdb540a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.077+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/526467981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a5dc0 0x7fdb540a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.077+0000 7fdb60d9d700 1 -- 172.21.15.178:0/526467981 >> 172.21.15.178:0/526467981 conn(0x7fdb540a0120 msgr2=0x7fdb540a2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.096 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.087+0000 7fdb60d9d700 1 -- 172.21.15.178:0/526467981 shutdown_connections 2023-10-16T14:39:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.087+0000 7fdb60d9d700 1 -- 172.21.15.178:0/526467981 wait complete. 2023-10-16T14:39:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.089+0000 7fdb60d9d700 1 Processor -- start 2023-10-16T14:39:33.097 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.092+0000 7fdb60d9d700 1 -- start start 2023-10-16T14:39:33.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.099+0000 7fdb60d9d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a4c90 0x7fdb5413a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.103+0000 7fdb60d9d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 0x7fdb5413ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.103+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 0x7fdb5413ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.103+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 0x7fdb5413ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:55510/0 (socket says 172.21.15.178:55510) 2023-10-16T14:39:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.103+0000 7fdb60d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb54141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.104+0000 7fdb5affd700 1 -- 172.21.15.178:0/1627358642 learned_addr learned my addr 172.21.15.178:0/1627358642 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.104+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541424c0 con 0x7fdb540a6780 2023-10-16T14:39:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.104+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54142600 con 0x7fdb540a4c90 2023-10-16T14:39:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.104+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a4c90 0x7fdb5413a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.105+0000 7fdb5bfff700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb54141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.111+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb54142790 con 0x7fdb540a5dc0 2023-10-16T14:39:33.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.112+0000 7fdb5bfff700 1 -- 172.21.15.178:0/1627358642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 msgr2=0x7fdb5413ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.112+0000 7fdb5bfff700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 0x7fdb5413ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.112+0000 7fdb5bfff700 1 -- 172.21.15.178:0/1627358642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a4c90 msgr2=0x7fdb5413a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.112+0000 7fdb5bfff700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a4c90 0x7fdb5413a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.112+0000 7fdb5bfff700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5c04f460 con 0x7fdb540a6780 2023-10-16T14:39:33.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.113+0000 7fdb5bfff700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb54141ef0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c050550 tx=0x7fdb5c07cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.118+0000 7fdb58ff9700 1 -- 172.21.15.178:0/1627358642 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c083050 con 0x7fdb540a6780 2023-10-16T14:39:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.118+0000 7fdb58ff9700 1 -- 172.21.15.178:0/1627358642 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5c084040 con 0x7fdb540a6780 2023-10-16T14:39:33.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.118+0000 7fdb58ff9700 1 -- 172.21.15.178:0/1627358642 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c08c070 con 0x7fdb540a6780 2023-10-16T14:39:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.120+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb54142a10 con 0x7fdb540a6780 2023-10-16T14:39:33.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.121+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb54142ec0 con 0x7fdb540a6780 2023-10-16T14:39:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.163+0000 7fdb58ff9700 1 -- 172.21.15.178:0/1627358642 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fdb5c087ac0 con 0x7fdb540a6780 2023-10-16T14:39:33.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.165+0000 7fdb58ff9700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 0x7fdb4c0a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.168+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 0x7fdb4c0a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.173+0000 7fdb58ff9700 1 -- 172.21.15.178:0/1627358642 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdb5c087410 con 0x7fdb540a6780 2023-10-16T14:39:33.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.174+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 0x7fdb4c0a3ba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009910 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.198+0000 7fdb75920b80 1 Processor -- start 2023-10-16T14:39:33.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.205+0000 7fdb75920b80 1 -- start start 2023-10-16T14:39:33.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc70a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc70bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc67ec70 con 0x55e7fc70a910 2023-10-16T14:39:33.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc663c60 con 0x55e7fc70b800 2023-10-16T14:39:33.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb75920b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc661580 con 0x55e7fc709fb0 2023-10-16T14:39:33.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.206+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37826/0 (socket says 172.21.15.178:37826) 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 -- 172.21.15.178:0/3582656897 learned_addr learned my addr 172.21.15.178:0/3582656897 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 -- 172.21.15.178:0/3582656897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 msgr2=0x55e7fc70a3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc70a3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 -- 172.21.15.178:0/3582656897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 msgr2=0x55e7fc70bc80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc70bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.207+0000 7fdb5affd700 1 -- 172.21.15.178:0/3582656897 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7fc61a3f0 con 0x55e7fc70a910 2023-10-16T14:39:33.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.208+0000 7fdb5affd700 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007b20 tx=0x7fdb48007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e513364d0c7f822 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.222+0000 7fdb45ffb700 1 -- 172.21.15.178:0/3582656897 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4800d710 con 0x55e7fc70a910 2023-10-16T14:39:33.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.222+0000 7fdb45ffb700 1 -- 172.21.15.178:0/3582656897 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4800dd30 con 0x55e7fc70a910 2023-10-16T14:39:33.233 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.222+0000 7fdb45ffb700 1 -- 172.21.15.178:0/3582656897 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48013e40 con 0x55e7fc70a910 2023-10-16T14:39:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.235+0000 7fdb75920b80 1 -- 172.21.15.178:0/3582656897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 msgr2=0x55e7fc70b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.235+0000 7fdb75920b80 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007b20 tx=0x7fdb48007e30 comp rx=0 tx=0).stop 2023-10-16T14:39:33.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.242+0000 7fdb75920b80 1 -- 172.21.15.178:0/3582656897 shutdown_connections 2023-10-16T14:39:33.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.242+0000 7fdb75920b80 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc70bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.242+0000 7fdb75920b80 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc70a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.242+0000 7fdb75920b80 1 --2- 172.21.15.178:0/3582656897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc70b220 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.246 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.242+0000 7fdb75920b80 1 -- 172.21.15.178:0/3582656897 >> 172.21.15.178:0/3582656897 conn(0x55e7fc637dd0 msgr2=0x55e7fc706960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.268+0000 7fdb75920b80 1 -- 172.21.15.178:0/3582656897 shutdown_connections 2023-10-16T14:39:33.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.274+0000 7fdb75920b80 1 -- 172.21.15.178:0/3582656897 wait complete. 2023-10-16T14:39:33.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.275+0000 7fdb75920b80 1 -- start start 2023-10-16T14:39:33.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.276+0000 7fdb75920b80 1 Processor -- start 2023-10-16T14:39:33.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc71b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc65ce70 con 0x55e7fc709fb0 2023-10-16T14:39:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc5f9710 con 0x55e7fc70b800 2023-10-16T14:39:33.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc5f93b0 con 0x55e7fc70a910 2023-10-16T14:39:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.279+0000 7fdb75920b80 10 client.0 _ll_register_callbacks cb 0x55e7fc456800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:55514/0 (socket says 172.21.15.178:55514) 2023-10-16T14:39:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5affd700 1 -- 172.21.15.178:0/1466099360 learned_addr learned my addr 172.21.15.178:0/1466099360 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:33.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5bfff700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc71b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5affd700 1 -- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 msgr2=0x55e7fc71b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.280+0000 7fdb5affd700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc71b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.281+0000 7fdb5affd700 1 -- 172.21.15.178:0/1466099360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 msgr2=0x55e7fc713f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:33.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.281+0000 7fdb5affd700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.281+0000 7fdb5affd700 1 -- 172.21.15.178:0/1466099360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4800e040 con 0x55e7fc70a910 2023-10-16T14:39:33.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.281+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.307+0000 7fdb5affd700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007200 tx=0x7fdb480136b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5464464bb779ba server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.311+0000 7fdb3effd700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:39:33.323 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.311+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48013dd0 con 0x55e7fc70a910 2023-10-16T14:39:33.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.311+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdb4800d710 con 0x55e7fc70a910 2023-10-16T14:39:33.328 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.326+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7fc661ef0 con 0x55e7fc70a910 2023-10-16T14:39:33.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.357+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4800d8d0 con 0x55e7fc70a910 2023-10-16T14:39:33.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.363+0000 7fdb75920b80 10 client.26209 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:33.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.363+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e7fc487de0 con 0x55e7fc70a910 2023-10-16T14:39:33.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.364+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7fc661ef0 con 0x55e7fc70a910 2023-10-16T14:39:33.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.364+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdb480038e0 con 0x55e7fc70a910 2023-10-16T14:39:33.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 10 client.26209 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:33.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:33.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 20 client.26209 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:33.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc71fc70 0x55e7fc722090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 4 client.26209 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.368+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7fc6d9c10 con 0x55e7fc71fc70 2023-10-16T14:39:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.369+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc71fc70 0x55e7fc722090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.370+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc71fc70 0x55e7fc722090 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a639a6938e8b10 server_cookie=a589777d31e4381f in_seq=0 out_seq=0 2023-10-16T14:39:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.371+0000 7fdb3effd700 10 client.26209 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.371+0000 7fdb3effd700 1 -- 172.21.15.178:0/1466099360 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e7fc6d9c10 con 0x55e7fc71fc70 2023-10-16T14:39:33.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.371+0000 7fdb3effd700 10 client.26209 handle_command_reply: tid=0 2023-10-16T14:39:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.371+0000 7fdb75920b80 1 client.26209 shutdown 2023-10-16T14:39:33.396 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 msgr2=0x55e7fc714490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007200 tx=0x7fdb480136b0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 shutdown_connections 2023-10-16T14:39:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc71fc70 0x55e7fc722090 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc71b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.397 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 --2- 172.21.15.178:0/1466099360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 >> 172.21.15.178:0/1466099360 conn(0x55e7fc637dd0 msgr2=0x55e7fc704d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.398 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.379+0000 7fdb3effd700 0 client.26209 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:33.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.399+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 shutdown_connections 2023-10-16T14:39:33.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.413+0000 7fdb75920b80 1 -- 172.21.15.178:0/1466099360 wait complete. 2023-10-16T14:39:33.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.425+0000 7fdb75920b80 20 client.26209 trim_cache size 0 max 16384 2023-10-16T14:39:33.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.738+0000 7fdb75920b80 1 Processor -- start 2023-10-16T14:39:33.760 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.758+0000 7fdb75920b80 1 -- start start 2023-10-16T14:39:33.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.758+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.760+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.760+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70b800 0x55e7fc71b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.760+0000 7fdb75920b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc4dd920 con 0x55e7fc709fb0 2023-10-16T14:39:33.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.760+0000 7fdb75920b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc713400 con 0x55e7fc70a910 2023-10-16T14:39:33.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.760+0000 7fdb75920b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc701730 con 0x55e7fc70b800 2023-10-16T14:39:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.763+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.776 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.763+0000 7fdb5b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37840/0 (socket says 172.21.15.178:37840) 2023-10-16T14:39:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.763+0000 7fdb5b7fe700 1 -- 172.21.15.178:0/2508391837 learned_addr learned my addr 172.21.15.178:0/2508391837 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.764+0000 7fdb5b7fe700 1 -- 172.21.15.178:0/2508391837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70b800 msgr2=0x55e7fc71b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.764+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70b800 0x55e7fc71b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.777 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.764+0000 7fdb5b7fe700 1 -- 172.21.15.178:0/2508391837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70a910 msgr2=0x55e7fc714490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:33.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.764+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.764+0000 7fdb5b7fe700 1 -- 172.21.15.178:0/2508391837 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7fc470c60 con 0x55e7fc709fb0 2023-10-16T14:39:33.778 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.765+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000d440 tx=0x7fdb50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a869b0a6c408394 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.792 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.782+0000 7fdb467fc700 1 -- 172.21.15.178:0/2508391837 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50023e00 con 0x55e7fc709fb0 2023-10-16T14:39:33.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.782+0000 7fdb467fc700 1 -- 172.21.15.178:0/2508391837 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb5001a040 con 0x55e7fc709fb0 2023-10-16T14:39:33.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.782+0000 7fdb467fc700 1 -- 172.21.15.178:0/2508391837 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50003740 con 0x55e7fc709fb0 2023-10-16T14:39:33.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.791+0000 7fdb75920b80 1 -- 172.21.15.178:0/2508391837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 msgr2=0x55e7fc713f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.793 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.791+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000d440 tx=0x7fdb50013000 comp rx=0 tx=0).stop 2023-10-16T14:39:33.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.808+0000 7fdb75920b80 1 -- 172.21.15.178:0/2508391837 shutdown_connections 2023-10-16T14:39:33.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.808+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc70b800 0x55e7fc71b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.808+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc709fb0 0x55e7fc713f50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.808+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2508391837 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70a910 0x55e7fc714490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.808+0000 7fdb75920b80 1 -- 172.21.15.178:0/2508391837 >> 172.21.15.178:0/2508391837 conn(0x55e7fc637dd0 msgr2=0x55e7fc70ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.826+0000 7fdb75920b80 1 -- 172.21.15.178:0/2508391837 shutdown_connections 2023-10-16T14:39:33.830 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.826+0000 7fdb75920b80 1 -- 172.21.15.178:0/2508391837 wait complete. 2023-10-16T14:39:33.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.827+0000 7fdb75920b80 1 -- start start 2023-10-16T14:39:33.831 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.827+0000 7fdb75920b80 1 Processor -- start 2023-10-16T14:39:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc71ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc719350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc719890 con 0x55e7fc70a910 2023-10-16T14:39:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc719a00 con 0x55e7fc70b800 2023-10-16T14:39:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.828+0000 7fdb75920b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7fc719bc0 con 0x55e7fc709fb0 2023-10-16T14:39:33.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.829+0000 7fdb75920b80 10 client.0 _ll_register_callbacks cb 0x7fdb48007920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.829+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.829+0000 7fdb5affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:37848/0 (socket says 172.21.15.178:37848) 2023-10-16T14:39:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.829+0000 7fdb5affd700 1 -- 172.21.15.178:0/2326988332 learned_addr learned my addr 172.21.15.178:0/2326988332 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 -- 172.21.15.178:0/2326988332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 msgr2=0x55e7fc71ad40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:33.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc71ad40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 -- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 msgr2=0x55e7fc719350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc719350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 -- 172.21.15.178:0/2326988332 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5000e040 con 0x55e7fc70a910 2023-10-16T14:39:33.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.830+0000 7fdb5affd700 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007a40 tx=0x7fdb48007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dc1779f3a44764 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.856+0000 7fdb3effd700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:39:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.856+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48013890 con 0x55e7fc70a910 2023-10-16T14:39:33.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.856+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdb4800db70 con 0x55e7fc70a910 2023-10-16T14:39:33.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.877+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb4800dd30 con 0x55e7fc70a910 2023-10-16T14:39:33.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.886+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb48003a70 con 0x55e7fc70a910 2023-10-16T14:39:33.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.915+0000 7fdb75920b80 10 client.16155 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:33.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.915+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55e7fc4ae250 con 0x55e7fc70a910 2023-10-16T14:39:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.916+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4800dd30 con 0x55e7fc70a910 2023-10-16T14:39:33.921 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.916+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fdb4801c5c0 con 0x55e7fc70a910 2023-10-16T14:39:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 10 client.16155 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 20 client.16155 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc774a90 0x55e7fc776eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:33.922 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 4 client.16155 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:39:33.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.918+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7fc655e50 con 0x55e7fc774a90 2023-10-16T14:39:33.923 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.919+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc774a90 0x55e7fc776eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:33.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.921+0000 7fdb5b7fe700 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc774a90 0x55e7fc776eb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a8538f456a8f7a0 server_cookie=381c9c0b5c1fbaae in_seq=0 out_seq=0 2023-10-16T14:39:33.924 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.922+0000 7fdb3effd700 10 client.16155 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:33.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.922+0000 7fdb3effd700 1 -- 172.21.15.178:0/2326988332 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7fc655e50 con 0x55e7fc774a90 2023-10-16T14:39:33.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.922+0000 7fdb3effd700 10 client.16155 handle_command_reply: tid=0 2023-10-16T14:39:33.931 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.922+0000 7fdb75920b80 1 client.16155 shutdown 2023-10-16T14:39:33.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.940+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 msgr2=0x55e7fc71b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.940+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007a40 tx=0x7fdb48007e30 comp rx=0 tx=0).stop 2023-10-16T14:39:33.951 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.940+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 shutdown_connections 2023-10-16T14:39:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.940+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x55e7fc774a90 0x55e7fc776eb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.941+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55e7fc70b800 0x55e7fc719350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.941+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e7fc709fb0 0x55e7fc71ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.941+0000 7fdb75920b80 1 --2- 172.21.15.178:0/2326988332 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55e7fc70a910 0x55e7fc71b280 secure :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdb48007a40 tx=0x7fdb48007e30 comp rx=0 tx=0).stop 2023-10-16T14:39:33.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.941+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 >> 172.21.15.178:0/2326988332 conn(0x55e7fc637dd0 msgr2=0x55e7fc706540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.946+0000 7fdb3effd700 0 client.16155 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:33.953 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.951+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 shutdown_connections 2023-10-16T14:39:33.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.951+0000 7fdb75920b80 1 -- 172.21.15.178:0/2326988332 wait complete. 2023-10-16T14:39:33.962 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:39:33.962 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.959+0000 7fdb75920b80 20 client.16155 trim_cache size 0 max 16384 2023-10-16T14:39:33.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.977+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 msgr2=0x7fdb4c0a3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.977+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 0x7fdb4c0a3ba0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009910 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2023-10-16T14:39:33.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.977+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 msgr2=0x7fdb54141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:33.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.977+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb54141ef0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c050550 tx=0x7fdb5c07cf90 comp rx=0 tx=0).stop 2023-10-16T14:39:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 shutdown_connections 2023-10-16T14:39:33.985 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fdb4c0a16e0 0x7fdb4c0a3ba0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb540a6780 0x7fdb54141ef0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fdb540a4c90 0x7fdb5413a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 --2- 172.21.15.178:0/1627358642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb540a5dc0 0x7fdb5413ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:33.986 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.983+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 >> 172.21.15.178:0/1627358642 conn(0x7fdb540a0120 msgr2=0x7fdb540a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.986+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 shutdown_connections 2023-10-16T14:39:33.989 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:33.986+0000 7fdb60d9d700 1 -- 172.21.15.178:0/1627358642 wait complete. 2023-10-16T14:39:34.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:33 smithi187 ceph-mon[135297]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2023-10-16T14:39:34.204 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:33 smithi178 ceph-mon[98495]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2023-10-16T14:39:34.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:33 smithi186 ceph-mon[134892]: pgmap v514: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.6 MiB/s wr, 571 op/s 2023-10-16T14:39:35.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-10-16T14:39:35.223 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-10-16T14:39:36.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:35 smithi187 ceph-mon[135297]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-16T14:39:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:35 smithi178 ceph-mon[98495]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-16T14:39:36.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:35 smithi186 ceph-mon[134892]: pgmap v515: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 531 op/s 2023-10-16T14:39:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:37 smithi187 ceph-mon[135297]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2023-10-16T14:39:38.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:37 smithi178 ceph-mon[98495]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2023-10-16T14:39:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:37 smithi186 ceph-mon[134892]: pgmap v516: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 519 op/s 2023-10-16T14:39:39.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:38 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:39:39.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:38 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:39:39.336 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:38 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:39:40.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:39 smithi187 ceph-mon[135297]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2023-10-16T14:39:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:39 smithi178 ceph-mon[98495]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2023-10-16T14:39:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:39 smithi186 ceph-mon[134892]: pgmap v517: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 353 op/s 2023-10-16T14:39:41.056 INFO:tasks.workunit.client.0.smithi178.stdout: 343 637011 1254 447513 1056 326621 1991 2023-10-16T14:39:41.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:41 smithi187 ceph-mon[135297]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2023-10-16T14:39:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:41 smithi186 ceph-mon[134892]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2023-10-16T14:39:41.835 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:41 smithi178 ceph-mon[98495]: pgmap v518: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 370 op/s 2023-10-16T14:39:41.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.863+0000 7f558139d700 1 -- 172.21.15.178:0/188101155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a5dd0 msgr2=0x7f55740a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:41.869 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.863+0000 7f558139d700 1 --2- 172.21.15.178:0/188101155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a5dd0 0x7f55740a6250 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f55680099e0 tx=0x7f5568009cf0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.889+0000 7f558139d700 1 -- 172.21.15.178:0/188101155 shutdown_connections 2023-10-16T14:39:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.889+0000 7f558139d700 1 --2- 172.21.15.178:0/188101155 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a6790 0x7f55740ab030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.889+0000 7f558139d700 1 --2- 172.21.15.178:0/188101155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a4ca0 0x7f55740a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.889+0000 7f558139d700 1 --2- 172.21.15.178:0/188101155 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a5dd0 0x7f55740a6250 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.889+0000 7f558139d700 1 -- 172.21.15.178:0/188101155 >> 172.21.15.178:0/188101155 conn(0x7f55740a0130 msgr2=0x7f55740a2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:41.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.910+0000 7f558139d700 1 -- 172.21.15.178:0/188101155 shutdown_connections 2023-10-16T14:39:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.948+0000 7f558139d700 1 -- 172.21.15.178:0/188101155 wait complete. 2023-10-16T14:39:41.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.949+0000 7f558139d700 1 Processor -- start 2023-10-16T14:39:41.954 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 -- start start 2023-10-16T14:39:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 0x7f5574143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 0x7f5574143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 0x7f557414a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557414adb0 con 0x7f55740a4ca0 2023-10-16T14:39:41.955 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740ac8c0 con 0x7f55740a5dd0 2023-10-16T14:39:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.950+0000 7f558139d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740aca50 con 0x7f55740a6790 2023-10-16T14:39:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.951+0000 7f557b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 0x7f5574143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.951+0000 7f557b7fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 0x7f5574143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37522/0 (socket says 172.21.15.178:37522) 2023-10-16T14:39:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.951+0000 7f557b7fe700 1 -- 172.21.15.178:0/2257018797 learned_addr learned my addr 172.21.15.178:0/2257018797 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:41.956 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.951+0000 7f55825ff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 0x7f557414a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.951+0000 7f557bfff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 0x7f5574143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 -- 172.21.15.178:0/2257018797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 msgr2=0x7f5574143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 0x7f5574143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 -- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 msgr2=0x7f5574143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:41.957 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 0x7f5574143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 -- 172.21.15.178:0/2257018797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680095f0 con 0x7f55740a6790 2023-10-16T14:39:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f557bfff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 0x7f5574143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:39:41.958 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.952+0000 7f55825ff700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 0x7f557414a7e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f557000b3c0 tx=0x7f557000b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.955+0000 7f55797fa700 1 -- 172.21.15.178:0/2257018797 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570011620 con 0x7f55740a6790 2023-10-16T14:39:41.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.955+0000 7f55797fa700 1 -- 172.21.15.178:0/2257018797 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5570011c40 con 0x7f55740a6790 2023-10-16T14:39:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.955+0000 7f55797fa700 1 -- 172.21.15.178:0/2257018797 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570010cf0 con 0x7f55740a6790 2023-10-16T14:39:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.959+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55740acd30 con 0x7f55740a6790 2023-10-16T14:39:41.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.959+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740ad270 con 0x7f55740a6790 2023-10-16T14:39:41.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:41.991+0000 7f55797fa700 1 -- 172.21.15.178:0/2257018797 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f557001a400 con 0x7f55740a6790 2023-10-16T14:39:42.008 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.000+0000 7f559601ab80 1 Processor -- start 2023-10-16T14:39:42.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.004+0000 7f55797fa700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 0x7f556c0a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.009 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.004+0000 7f55797fa700 1 -- 172.21.15.178:0/2257018797 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f55700cc810 con 0x7f55740a6790 2023-10-16T14:39:42.022 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.018+0000 7f557b7fe700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 0x7f556c0a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.023 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.020+0000 7f557b7fe700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 0x7f556c0a3b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5568006010 tx=0x7f5568005040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:42.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.064+0000 7f559601ab80 1 -- start start 2023-10-16T14:39:42.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4faf72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fa6acf0 con 0x555e4faf7880 2023-10-16T14:39:42.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fa4fce0 con 0x555e4faf6990 2023-10-16T14:39:42.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.065+0000 7f559601ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fa4d600 con 0x555e4faf6030 2023-10-16T14:39:42.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.074+0000 7f55825ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.074+0000 7f55825ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53138/0 (socket says 172.21.15.178:53138) 2023-10-16T14:39:42.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.074+0000 7f55825ff700 1 -- 172.21.15.178:0/3948063173 learned_addr learned my addr 172.21.15.178:0/3948063173 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:42.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 -- 172.21.15.178:0/3948063173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 msgr2=0x555e4faf6450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:42.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf6450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 -- 172.21.15.178:0/3948063173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 msgr2=0x555e4faf72a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:42.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4faf72a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 -- 172.21.15.178:0/3948063173 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4fa06470 con 0x555e4faf7880 2023-10-16T14:39:42.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.075+0000 7f55825ff700 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5570097200 tx=0x7f5570099fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7add6a7ad180c72 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:42.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.078+0000 7f55667fc700 1 -- 172.21.15.178:0/3948063173 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700b90f0 con 0x555e4faf7880 2023-10-16T14:39:42.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.078+0000 7f55667fc700 1 -- 172.21.15.178:0/3948063173 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55700a1af0 con 0x555e4faf7880 2023-10-16T14:39:42.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.078+0000 7f55667fc700 1 -- 172.21.15.178:0/3948063173 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557009e5f0 con 0x555e4faf7880 2023-10-16T14:39:42.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.095+0000 7f559601ab80 1 -- 172.21.15.178:0/3948063173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 msgr2=0x555e4faf7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:42.099 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.095+0000 7f559601ab80 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5570097200 tx=0x7f5570099fc0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.104+0000 7f559601ab80 1 -- 172.21.15.178:0/3948063173 shutdown_connections 2023-10-16T14:39:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.104+0000 7f559601ab80 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faf7d00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.104+0000 7f559601ab80 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.104+0000 7f559601ab80 1 --2- 172.21.15.178:0/3948063173 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4faf72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.104+0000 7f559601ab80 1 -- 172.21.15.178:0/3948063173 >> 172.21.15.178:0/3948063173 conn(0x555e4fa23e50 msgr2=0x555e4faf29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:42.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.109+0000 7f559601ab80 1 -- 172.21.15.178:0/3948063173 shutdown_connections 2023-10-16T14:39:42.114 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.109+0000 7f559601ab80 1 -- 172.21.15.178:0/3948063173 wait complete. 2023-10-16T14:39:42.116 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.113+0000 7f559601ab80 1 -- start start 2023-10-16T14:39:42.117 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.114+0000 7f559601ab80 1 Processor -- start 2023-10-16T14:39:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f559601ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f559601ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f559601ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 0x555e4fb07630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f557bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f557bfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:39506/0 (socket says 172.21.15.178:39506) 2023-10-16T14:39:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.129+0000 7f557b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.130+0000 7f557b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53154/0 (socket says 172.21.15.178:53154) 2023-10-16T14:39:42.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.130+0000 7f557bfff700 1 -- 172.21.15.178:0/3466455743 learned_addr learned my addr 172.21.15.178:0/3466455743 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:42.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.135+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fa48ef0 con 0x555e4faf6990 2023-10-16T14:39:42.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.135+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f9e5790 con 0x555e4faf7880 2023-10-16T14:39:42.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.135+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f9e5430 con 0x555e4faf6030 2023-10-16T14:39:42.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.138+0000 7f559601ab80 10 client.0 _ll_register_callbacks cb 0x555e4f842880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:42.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.138+0000 7f55825ff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 0x555e4fb07630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.140+0000 7f55825ff700 1 -- 172.21.15.178:0/3466455743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 msgr2=0x555e4fb000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:42.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.140+0000 7f55825ff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.140+0000 7f55825ff700 1 -- 172.21.15.178:0/3466455743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 msgr2=0x555e4fb005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:42.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.140+0000 7f55825ff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.140+0000 7f55825ff700 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740ad270 con 0x555e4faf7880 2023-10-16T14:39:42.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.141+0000 7f557b7fe700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-10-16T14:39:42.145 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.141+0000 7f55825ff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 0x555e4fb07630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55700965f0 tx=0x7f5570006dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b239337549e0c478 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:42.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.151+0000 7f557bfff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:42.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.159+0000 7f555f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:39:42.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.159+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557009ad30 con 0x555e4faf7880 2023-10-16T14:39:42.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.159+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5570011780 con 0x555e4faf7880 2023-10-16T14:39:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.174+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4fa4df70 con 0x555e4faf7880 2023-10-16T14:39:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.175+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5570011980 con 0x555e4faf7880 2023-10-16T14:39:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.175+0000 7f559601ab80 10 client.26231 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:42.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.175+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555e4f873e60 con 0x555e4faf7880 2023-10-16T14:39:42.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.176+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555e4fa4df70 con 0x555e4faf7880 2023-10-16T14:39:42.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.177+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f5570005b00 con 0x555e4faf7880 2023-10-16T14:39:42.181 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 10 client.26231 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 10 client.26231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 20 client.26231 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb0ba50 0x555e4fb0de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 4 client.26231 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:42.182 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4fac5c90 con 0x555e4fb0ba50 2023-10-16T14:39:42.186 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.179+0000 7f557bfff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb0ba50 0x555e4fb0de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:42.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.197+0000 7f557bfff700 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb0ba50 0x555e4fb0de70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76246e0ccb16fee server_cookie=706e264b5d716fe5 in_seq=0 out_seq=0 2023-10-16T14:39:42.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.198+0000 7f555f7fe700 10 client.26231 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:42.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.209+0000 7f555f7fe700 1 -- 172.21.15.178:0/3466455743 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555e4fac5c90 con 0x555e4fb0ba50 2023-10-16T14:39:42.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.209+0000 7f555f7fe700 10 client.26231 handle_command_reply: tid=0 2023-10-16T14:39:42.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.212+0000 7f559601ab80 1 client.26231 shutdown 2023-10-16T14:39:42.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.243+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 msgr2=0x555e4fb07630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:42.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.243+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 0x555e4fb07630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f55700965f0 tx=0x7f5570006dc0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.252 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 shutdown_connections 2023-10-16T14:39:42.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb0ba50 0x555e4fb0de70 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf7880 0x555e4fb07630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4fb000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 --2- 172.21.15.178:0/3466455743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:42.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.244+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 >> 172.21.15.178:0/3466455743 conn(0x555e4fa23e50 msgr2=0x555e4faf0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:42.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.245+0000 7f555f7fe700 0 client.26231 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:42.263 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.261+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 shutdown_connections 2023-10-16T14:39:42.271 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.268+0000 7f559601ab80 1 -- 172.21.15.178:0/3466455743 wait complete. 2023-10-16T14:39:42.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:42.279+0000 7f559601ab80 20 client.26231 trim_cache size 0 max 16384 2023-10-16T14:39:43.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.143+0000 7f559601ab80 1 Processor -- start 2023-10-16T14:39:43.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.193+0000 7f559601ab80 1 -- start start 2023-10-16T14:39:43.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4fb07550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f8c99a0 con 0x555e4faf7880 2023-10-16T14:39:43.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fac5c90 con 0x555e4faf6990 2023-10-16T14:39:43.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.194+0000 7f559601ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fa4df70 con 0x555e4faf6030 2023-10-16T14:39:43.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.195+0000 7f55825ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:43.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.195+0000 7f55825ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:53156/0 (socket says 172.21.15.178:53156) 2023-10-16T14:39:43.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.195+0000 7f55825ff700 1 -- 172.21.15.178:0/590774996 learned_addr learned my addr 172.21.15.178:0/590774996 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:43.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 -- 172.21.15.178:0/590774996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 msgr2=0x555e4faf2cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:39:43.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 -- 172.21.15.178:0/590774996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 msgr2=0x555e4fb07550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:43.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4fb07550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 -- 172.21.15.178:0/590774996 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4f85cce0 con 0x555e4faf7880 2023-10-16T14:39:43.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.196+0000 7f55825ff700 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5570096c90 tx=0x7f557000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aedeeae09ea6592 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:43.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.199+0000 7f5566ffd700 1 -- 172.21.15.178:0/590774996 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557007d070 con 0x555e4faf7880 2023-10-16T14:39:43.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.199+0000 7f5566ffd700 1 -- 172.21.15.178:0/590774996 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5570005a20 con 0x555e4faf7880 2023-10-16T14:39:43.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.199+0000 7f5566ffd700 1 -- 172.21.15.178:0/590774996 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557009e5f0 con 0x555e4faf7880 2023-10-16T14:39:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.209+0000 7f559601ab80 1 -- 172.21.15.178:0/590774996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 msgr2=0x555e4faeaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:43.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.209+0000 7f559601ab80 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5570096c90 tx=0x7f557000bc50 comp rx=0 tx=0).stop 2023-10-16T14:39:43.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.214+0000 7f559601ab80 1 -- 172.21.15.178:0/590774996 shutdown_connections 2023-10-16T14:39:43.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.214+0000 7f559601ab80 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf7880 0x555e4faeaab0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.214+0000 7f559601ab80 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf6030 0x555e4faf2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.214+0000 7f559601ab80 1 --2- 172.21.15.178:0/590774996 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6990 0x555e4fb07550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.214+0000 7f559601ab80 1 -- 172.21.15.178:0/590774996 >> 172.21.15.178:0/590774996 conn(0x555e4fa23e50 msgr2=0x555e4faf25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.219+0000 7f559601ab80 1 -- 172.21.15.178:0/590774996 shutdown_connections 2023-10-16T14:39:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.221+0000 7f559601ab80 1 -- 172.21.15.178:0/590774996 wait complete. 2023-10-16T14:39:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.222+0000 7f559601ab80 1 -- start start 2023-10-16T14:39:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.223+0000 7f559601ab80 1 Processor -- start 2023-10-16T14:39:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.226+0000 7f559601ab80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.226+0000 7f559601ab80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f559601ab80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 0x555e4fb06ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f559601ab80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fb5f900 con 0x555e4faf6990 2023-10-16T14:39:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f559601ab80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fb5fa70 con 0x555e4faf6030 2023-10-16T14:39:43.234 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f559601ab80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4fb5fc30 con 0x555e4faf7880 2023-10-16T14:39:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f559601ab80 10 client.0 _ll_register_callbacks cb 0x7f55700c9e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f557bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f557bfff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:37542/0 (socket says 172.21.15.178:37542) 2023-10-16T14:39:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f557bfff700 1 -- 172.21.15.178:0/852339632 learned_addr learned my addr 172.21.15.178:0/852339632 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:43.235 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.227+0000 7f557b7fe700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f55825ff700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 0x555e4fb06ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 -- 172.21.15.178:0/852339632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 msgr2=0x555e4fb06ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 0x555e4fb06ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 -- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 msgr2=0x555e4fb06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:43.236 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 -- 172.21.15.178:0/852339632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e4f873e60 con 0x555e4faf6990 2023-10-16T14:39:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f557b7fe700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb06570 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x555e4fb093b0 tx=0x7f5568009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc09b050d009387c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.228+0000 7f555f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:39:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.231+0000 7f55825ff700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 0x555e4fb06ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:43.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.231+0000 7f557bfff700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:39:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.248+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556804b050 con 0x555e4faf6990 2023-10-16T14:39:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.248+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f556800f660 con 0x555e4faf6990 2023-10-16T14:39:43.253 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.250+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55700a1af0 con 0x555e4faf6990 2023-10-16T14:39:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.251+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f556800f820 con 0x555e4faf6990 2023-10-16T14:39:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.251+0000 7f559601ab80 10 client.16170 fetch_fsmap learned FSMap version 33 2023-10-16T14:39:43.254 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.251+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x555e4f89a2d0 con 0x555e4faf6990 2023-10-16T14:39:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.252+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f556804c040 con 0x555e4faf6990 2023-10-16T14:39:43.255 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.252+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 33) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f556805ac20 con 0x555e4faf6990 2023-10-16T14:39:43.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 10 client.16170 fetch_fsmap finished waiting for FSMap version 33 2023-10-16T14:39:43.256 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:43.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 20 client.16170 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:43.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb61710 0x555e4fb63b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:43.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 4 client.16170 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-10-16T14:39:43.257 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.253+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x7f55700a1af0 con 0x555e4fb61710 2023-10-16T14:39:43.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.256+0000 7f557bfff700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb61710 0x555e4fb63b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:43.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.256+0000 7f557bfff700 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb61710 0x555e4fb63b30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136386b788b52d15 server_cookie=5d0d22609e6789a in_seq=0 out_seq=0 2023-10-16T14:39:43.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.256+0000 7f555f7fe700 10 client.16170 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.258+0000 7f555f7fe700 1 -- 172.21.15.178:0/852339632 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f55700a1af0 con 0x555e4fb61710 2023-10-16T14:39:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.258+0000 7f555f7fe700 10 client.16170 handle_command_reply: tid=0 2023-10-16T14:39:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.258+0000 7f559601ab80 1 client.16170 shutdown 2023-10-16T14:39:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.258+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 msgr2=0x555e4fb06570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:43.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.258+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb06570 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x555e4fb093b0 tx=0x7f5568009310 comp rx=0 tx=0).stop 2023-10-16T14:39:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 shutdown_connections 2023-10-16T14:39:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x555e4fb61710 0x555e4fb63b30 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555e4faf7880 0x555e4fb06ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x555e4faf6030 0x555e4fb06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 --2- 172.21.15.178:0/852339632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x555e4faf6990 0x555e4fb06570 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 >> 172.21.15.178:0/852339632 conn(0x555e4fa23e50 msgr2=0x555e4fafe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:43.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.274+0000 7f555f7fe700 0 client.16170 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:43.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.320+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 shutdown_connections 2023-10-16T14:39:43.324 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.320+0000 7f559601ab80 1 -- 172.21.15.178:0/852339632 wait complete. 2023-10-16T14:39:43.326 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.324+0000 7f559601ab80 20 client.16170 trim_cache size 0 max 16384 2023-10-16T14:39:43.347 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:39:43.347 INFO:teuthology.orchestra.run.smithi178.stdout: "return_code": 0, 2023-10-16T14:39:43.347 INFO:teuthology.orchestra.run.smithi178.stdout: "scrub_tag": "fca1b280-3016-4a9c-8fd4-d0bd2508c9a7", 2023-10-16T14:39:43.347 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": "asynchronous" 2023-10-16T14:39:43.348 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:39:43.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.352+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 msgr2=0x7f556c0a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:43.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.352+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 0x7f556c0a3b40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5568006010 tx=0x7f5568005040 comp rx=0 tx=0).stop 2023-10-16T14:39:43.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.352+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 msgr2=0x7f557414a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:43.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.352+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 0x7f557414a7e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f557000b3c0 tx=0x7f557000b6d0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 shutdown_connections 2023-10-16T14:39:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f556c0a1680 0x7f556c0a3b40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55740a6790 0x7f557414a7e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55740a4ca0 0x7f5574143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 --2- 172.21.15.178:0/2257018797 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f55740a5dd0 0x7f5574143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.377+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 >> 172.21.15.178:0/2257018797 conn(0x7f55740a0130 msgr2=0x7f55740a0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:43.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.390+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 shutdown_connections 2023-10-16T14:39:43.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:43.402+0000 7f558139d700 1 -- 172.21.15.178:0/2257018797 wait complete. 2023-10-16T14:39:43.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:43.345+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000055b [...2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/ auth{0=1} fragtree_t(*^3) v190173 ap=2 f(v0 m2023-10-16T14:39:32.155543+0000 227=227+0) n(v0 rc2023-10-16T14:39:32.155543+0000 b4549162 228=227+1) (iauth sync r=1) (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock) caps={15381=pAsLsXsFsx/pAsLsXsFsx@7},l=15381 | lock=5 importingcaps=0 dirfrag=8 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55ce4c756100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x1000000055b:[<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[]","memoryvalue":"(2)0x1000000055b:[<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2261>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-16T14:39:32.155543+0000 227=227+0)","ondisk_value.rstat":"n(v0 rc2023-10-16T14:39:32.155543+0000 b4549162 228=227+1)","memory_value.dirstat":"f(v0 m2023-10-16T14:39:32.155543+0000 227=227+0)","memory_value.rstat":"n(v0 rc2023-10-16T14:39:32.155543+0000 b4549162 228=227+1)","error_str":""},"return_code":0} 2023-10-16T14:39:43.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:43.424+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000729 [...2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/ auth{0=1} fragtree_t(*^3) v189633 ap=2 f(v0 m2023-10-16T14:39:30.700568+0000 204=204+0) n(v0 rc2023-10-16T14:39:30.700568+0000 b3275891 205=204+1) (iauth sync r=1) (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock) caps={15381=pAsLsXsFsx/pAsLsXsFsx@5},l=15381 | lock=5 importingcaps=0 dirfrag=8 caps=1 dirtyrstat=0 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55ce4f92c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000000729:[<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[]","memoryvalue":"(2)0x10000000729:[<0x10000000223/blog-10 v189633>,<0x200000001fe/blogtest_in v2266>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-16T14:39:30.700568+0000 204=204+0)","ondisk_value.rstat":"n(v0 rc2023-10-16T14:39:30.700568+0000 b3275891 205=204+1)","memory_value.dirstat":"f(v0 m2023-10-16T14:39:30.700568+0000 204=204+0)","memory_value.rstat":"n(v0 rc2023-10-16T14:39:30.700568+0000 b3275891 205=204+1)","error_str":""},"return_code":0} 2023-10-16T14:39:43.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:43.426+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001a0b [...2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33/ auth{0=1,2=1} fragtree_t(*^3) v191655 ap=1 f(v6 m2023-10-16T14:39:25.344658+0000 214=214+0) n(v10 rc2023-10-16T14:39:25.344658+0000 b2974565 215=214+1) (inest mix) (ifile excl) (iversion lock) caps={15381=pAsLsXsFsx/pAsLsXsFsx@1},l=15381 | dirtyscattered=0 lock=0 importingcaps=0 dirfrag=8 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55ce4fb6e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001a0b:[<0x10000000223/blog-33 v185247>,<0x200000001fe/blogtest_in v2205>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[]","memoryvalue":"(2)0x10000001a0b:[<0x10000000223/blog-33 v191655>,<0x200000001fe/blogtest_in v2266>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-16T14:39:25.344658+0000 214=214+0)","ondisk_value.rstat":"n(v0 rc2023-10-16T14:39:25.344658+0000 b2974565 215=214+1)","memory_value.dirstat":"f(v6 m2023-10-16T14:39:25.344658+0000 214=214+0)","memory_value.rstat":"n(v10 rc2023-10-16T14:39:25.344658+0000 b2974565 215=214+1)","error_str":""},"return_code":0} 2023-10-16T14:39:43.676 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:43.441+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf4 [...2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/ auth{0=1} fragtree_t(*^3) v191197 ap=2 f(v0 m2023-10-16T14:39:34.219508+0000 204=204+0) n(v0 rc2023-10-16T14:39:34.219508+0000 b4249585 205=204+1) (iauth sync r=1) (isnap sync r=1) (inest lock w=1) (ipolicy sync r=1) (ifile excl w=1) (iversion lock) caps={15381=pAsLsXsFsx/pAsLsXsFsx@3},l=15381 | lock=5 importingcaps=0 dirfrag=8 caps=1 replicated=1 dirty=1 authpin=1 scrubqueue=0 0x55ce4d08cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(2)0x10000001cf4:[<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[]","memoryvalue":"(2)0x10000001cf4:[<0x10000000223/blog-38 v191197>,<0x200000001fe/blogtest_in v2266>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-10-16T14:39:34.219508+0000 204=204+0)","ondisk_value.rstat":"n(v0 rc2023-10-16T14:39:34.219508+0000 b4249585 205=204+1)","memory_value.dirstat":"f(v0 m2023-10-16T14:39:34.219508+0000 204=204+0)","memory_value.rstat":"n(v0 rc2023-10-16T14:39:34.219508+0000 b4249585 205=204+1)","error_str":""},"return_code":0} 2023-10-16T14:39:43.921 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:39:44.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-10-16T14:39:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:39:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:39:44.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:39:44.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000729 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:43 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cf4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-10-16T14:39:44.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:39:44.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:44.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:39:44.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:39:44.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000729 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:43 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cf4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: pgmap v519: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 274 op/s 2023-10-16T14:39:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:39:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:39:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:39:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000055b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000729 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:43 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cf4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38) see mds.i log and `damage ls` output for details 2023-10-16T14:39:44.886 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-mon[135297]: scrub queued for path: / 2023-10-16T14:39:44.887 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:44.887 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:39:44.891 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.586+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000642 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg auth v1002 ap=1 s=53233 n(v0 rc2023-10-16T14:35:33.757619+0000 b53233 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000642:[<0x1000000055b/picture-13.jpg v1002>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000642:[<0x1000000055b/picture-13.jpg v1002>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.891 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.587+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000638 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg auth v1006 ap=1 s=33062 n(v0 rc2023-10-16T14:35:33.727620+0000 b33062 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000638:[<0x1000000055b/picture-18.jpg v1006>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000638:[<0x1000000055b/picture-18.jpg v1006>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.891 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.587+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000577 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml auth v904 ap=1 s=1129 n(v0 rc2023-10-16T14:35:33.069631+0000 b1129 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000577:[<0x1000000055b/article-18.xml v904>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000577:[<0x1000000055b/article-18.xml v904>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.891 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.587+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000637 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg auth v1008 ap=1 s=39314 n(v0 rc2023-10-16T14:35:33.720620+0000 b39314 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000637:[<0x1000000055b/picture-19.jpg v1008>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000637:[<0x1000000055b/picture-19.jpg v1008>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.891 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.589+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000654 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg auth v996 ap=1 s=61844 n(v0 rc2023-10-16T14:35:33.817618+0000 b61844 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000654:[<0x1000000055b/picture-4.jpg v996>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000654:[<0x1000000055b/picture-4.jpg v996>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.892 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.589+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000061f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg auth v1016 ap=1 s=28538 n(v0 rc2023-10-16T14:35:33.624622+0000 b28538 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51124000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000061f:[<0x1000000055b/picture-35.jpg v1016>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000061f:[<0x1000000055b/picture-35.jpg v1016>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.892 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005c0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg auth v1020 ap=1 s=25364 n(v0 rc2023-10-16T14:35:33.302627+0000 b25364 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51125080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005c0:[<0x1000000055b/picture-93.jpg v1020>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005c0:[<0x1000000055b/picture-93.jpg v1020>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.892 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg auth v1022 ap=1 s=31902 n(v0 rc2023-10-16T14:35:33.382626+0000 b31902 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5145f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d8:[<0x1000000055b/picture-82.jpg v1022>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d8:[<0x1000000055b/picture-82.jpg v1022>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.892 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000658 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg auth v994 ap=1 s=58326 n(v0 rc2023-10-16T14:35:33.828618+0000 b58326 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51124580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000658:[<0x1000000055b/picture-2.jpg v994>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000658:[<0x1000000055b/picture-2.jpg v994>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.893 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005df [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg auth v1010 ap=1 s=57034 n(v0 rc2023-10-16T14:35:33.410625+0000 b57034 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51124b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005df:[<0x1000000055b/picture-78.jpg v1010>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005df:[<0x1000000055b/picture-78.jpg v1010>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.893 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000640 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg auth v1004 ap=1 s=11710 n(v0 rc2023-10-16T14:35:33.750620+0000 b11710 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51125b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000640:[<0x1000000055b/picture-14.jpg v1004>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000640:[<0x1000000055b/picture-14.jpg v1004>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.893 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000570 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml auth v906 ap=1 s=21241 n(v0 rc2023-10-16T14:35:33.048632+0000 b21241 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51125600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000570:[<0x1000000055b/article-20.xml v906>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000570:[<0x1000000055b/article-20.xml v906>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.893 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.590+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ca [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg auth v1014 ap=1 s=50434 n(v0 rc2023-10-16T14:35:33.341627+0000 b50434 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51126100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ca:[<0x1000000055b/picture-89.jpg v1014>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ca:[<0x1000000055b/picture-89.jpg v1014>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.893 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.591+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg auth v1018 ap=1 s=47700 n(v0 rc2023-10-16T14:35:33.494624+0000 b47700 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51127180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f8:[<0x1000000055b/picture-61.jpg v1018>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f8:[<0x1000000055b/picture-61.jpg v1018>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.894 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.591+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000064e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg auth v998 ap=1 s=12328 n(v0 rc2023-10-16T14:35:33.791619+0000 b12328 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce51127700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000064e:[<0x1000000055b/picture-7.jpg v998>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000064e:[<0x1000000055b/picture-7.jpg v998>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.894 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.591+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000064a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg auth v1000 ap=1 s=38333 n(v0 rc2023-10-16T14:35:33.780619+0000 b38333 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5179a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000064a:[<0x1000000055b/picture-9.jpg v1000>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000064a:[<0x1000000055b/picture-9.jpg v1000>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.894 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.592+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f4 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg auth v1020 ap=1 s=63192 n(v0 rc2023-10-16T14:35:33.480624+0000 b63192 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5179b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f4:[<0x1000000055b/picture-64.jpg v1020>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f4:[<0x1000000055b/picture-64.jpg v1020>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.894 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.592+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000648 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg auth v990 ap=1 s=33319 n(v0 rc2023-10-16T14:35:33.774619+0000 b33319 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5179c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000648:[<0x1000000055b/picture-10.jpg v990>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000648:[<0x1000000055b/picture-10.jpg v990>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.894 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.592+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000659 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg auth v984 ap=1 s=11592 n(v0 rc2023-10-16T14:35:33.830618+0000 b11592 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5179cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000659:[<0x1000000055b/picture-1.jpg v984>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000659:[<0x1000000055b/picture-1.jpg v984>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.895 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.593+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg auth v1008 ap=1 s=16872 n(v0 rc2023-10-16T14:35:33.377626+0000 b16872 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f6c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d5:[<0x1000000055b/picture-83.jpg v1008>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d5:[<0x1000000055b/picture-83.jpg v1008>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.895 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.593+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000650 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg auth v988 ap=1 s=51825 n(v0 rc2023-10-16T14:35:33.797619+0000 b51825 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f6c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000650:[<0x1000000055b/picture-6.jpg v988>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000650:[<0x1000000055b/picture-6.jpg v988>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.895 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.593+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000563 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml auth v904 ap=1 s=28801 n(v0 rc2023-10-16T14:35:33.009632+0000 b28801 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f6d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000563:[<0x1000000055b/article-24.xml v904>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000563:[<0x1000000055b/article-24.xml v904>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.895 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.594+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000062a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg auth v1024 ap=1 s=31595 n(v0 rc2023-10-16T14:35:33.671621+0000 b31595 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f6e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000062a:[<0x1000000055b/picture-28.jpg v1024>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000062a:[<0x1000000055b/picture-28.jpg v1024>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.896 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.594+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005c2 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg auth v1012 ap=1 s=18219 n(v0 rc2023-10-16T14:35:33.316627+0000 b18219 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f6ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005c2:[<0x1000000055b/picture-92.jpg v1012>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005c2:[<0x1000000055b/picture-92.jpg v1012>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.896 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.594+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000061c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg auth v1010 ap=1 s=31296 n(v0 rc2023-10-16T14:35:33.613622+0000 b31296 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f90000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000061c:[<0x1000000055b/picture-37.jpg v1010>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000061c:[<0x1000000055b/picture-37.jpg v1010>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.896 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.595+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml auth v986 ap=1 s=1101 n(v0 rc2023-10-16T14:35:33.225629+0000 b1101 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f91080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a9:[<0x1000000055b/article-3.xml v986>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005a9:[<0x1000000055b/article-3.xml v986>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.896 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.595+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000622 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg auth v1018 ap=1 s=64568 n(v0 rc2023-10-16T14:35:33.642621+0000 b64568 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f91b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000622:[<0x1000000055b/picture-33.jpg v1018>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000622:[<0x1000000055b/picture-33.jpg v1018>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.896 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.595+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000630 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg auth v994 ap=1 s=10164 n(v0 rc2023-10-16T14:35:33.692621+0000 b10164 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f91600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000630:[<0x1000000055b/picture-24.jpg v994>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000630:[<0x1000000055b/picture-24.jpg v994>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.897 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.595+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d1 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg auth v1026 ap=1 s=55875 n(v0 rc2023-10-16T14:35:33.363626+0000 b55875 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f90b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d1:[<0x1000000055b/picture-85.jpg v1026>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d1:[<0x1000000055b/picture-85.jpg v1026>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.897 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.595+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml auth v906 ap=1 s=24969 n(v0 rc2023-10-16T14:35:33.183629+0000 b24969 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f92680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059b:[<0x1000000055b/article-7.xml v906>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000059b:[<0x1000000055b/article-7.xml v906>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.897 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.596+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005fe [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg auth v1022 ap=1 s=46337 n(v0 rc2023-10-16T14:35:33.514624+0000 b46337 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f92100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005fe:[<0x1000000055b/picture-57.jpg v1022>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005fe:[<0x1000000055b/picture-57.jpg v1022>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.897 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.596+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ee [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg auth v1014 ap=1 s=49204 n(v0 rc2023-10-16T14:35:33.460625+0000 b49204 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f93180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ee:[<0x1000000055b/picture-68.jpg v1014>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ee:[<0x1000000055b/picture-68.jpg v1014>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.897 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.597+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000619 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg auth v1016 ap=1 s=23396 n(v0 rc2023-10-16T14:35:33.602622+0000 b23396 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54f93700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000619:[<0x1000000055b/picture-39.jpg v1016>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000619:[<0x1000000055b/picture-39.jpg v1016>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.898 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.597+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000625 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg auth v1010 ap=1 s=37624 n(v0 rc2023-10-16T14:35:33.654621+0000 b37624 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce50cc2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000625:[<0x1000000055b/picture-31.jpg v1010>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000625:[<0x1000000055b/picture-31.jpg v1010>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.898 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.598+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000060a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg auth v998 ap=1 s=40169 n(v0 rc2023-10-16T14:35:33.552623+0000 b40169 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce50cc3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000060a:[<0x1000000055b/picture-49.jpg v998>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000060a:[<0x1000000055b/picture-49.jpg v998>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.898 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.598+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000064c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg auth v992 ap=1 s=36123 n(v0 rc2023-10-16T14:35:33.786619+0000 b36123 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce50cc3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000064c:[<0x1000000055b/picture-8.jpg v992>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000064c:[<0x1000000055b/picture-8.jpg v992>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.898 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.598+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000598 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml auth v908 ap=1 s=3601 n(v0 rc2023-10-16T14:35:33.168630+0000 b3601 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce50cc1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000598:[<0x1000000055b/article-8.xml v908>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000598:[<0x1000000055b/article-8.xml v908>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.898 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.598+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005cc [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg auth v1002 ap=1 s=12168 n(v0 rc2023-10-16T14:35:33.346627+0000 b12168 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce52e68000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005cc:[<0x1000000055b/picture-88.jpg v1002>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005cc:[<0x1000000055b/picture-88.jpg v1002>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.899 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.599+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000062d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg auth v996 ap=1 s=17083 n(v0 rc2023-10-16T14:35:33.681621+0000 b17083 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce52e68580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000062d:[<0x1000000055b/picture-26.jpg v996>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000062d:[<0x1000000055b/picture-26.jpg v996>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.899 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.599+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg auth v1000 ap=1 s=37664 n(v0 rc2023-10-16T14:35:33.280628+0000 b37664 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce52e69b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b9:[<0x1000000055b/picture-95.jpg v1000>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005b9:[<0x1000000055b/picture-95.jpg v1000>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.899 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.600+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000056d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml auth v906 ap=1 s=1625 n(v0 rc2023-10-16T14:35:33.040632+0000 b1625 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce52e6a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000056d:[<0x1000000055b/article-21.xml v906>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000056d:[<0x1000000055b/article-21.xml v906>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.899 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.601+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000588 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml auth v904 ap=1 s=6929 n(v0 rc2023-10-16T14:35:33.121630+0000 b6929 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce531ae000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000588:[<0x1000000055b/article-13.xml v904>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000588:[<0x1000000055b/article-13.xml v904>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.900 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.602+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000059e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml auth v988 ap=1 s=25083 n(v0 rc2023-10-16T14:35:33.193629+0000 b25083 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce531af600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000059e:[<0x1000000055b/article-6.xml v988>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000059e:[<0x1000000055b/article-6.xml v988>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.900 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.602+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ac [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml auth v990 ap=1 s=8744 n(v0 rc2023-10-16T14:35:33.237628+0000 b8744 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce531afb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ac:[<0x1000000055b/article-2.xml v990>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ac:[<0x1000000055b/article-2.xml v990>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.900 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.602+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000628 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg auth v1012 ap=1 s=54720 n(v0 rc2023-10-16T14:35:33.666621+0000 b54720 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce531b1180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000628:[<0x1000000055b/picture-29.jpg v1012>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000628:[<0x1000000055b/picture-29.jpg v1012>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.900 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.602+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000603 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg auth v1014 ap=1 s=52414 n(v0 rc2023-10-16T14:35:33.528623+0000 b52414 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce531b0c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000603:[<0x1000000055b/picture-54.jpg v1014>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000603:[<0x1000000055b/picture-54.jpg v1014>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.900 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.603+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg auth v1006 ap=1 s=8710 n(v0 rc2023-10-16T14:35:33.359626+0000 b8710 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54db9080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d0:[<0x1000000055b/picture-86.jpg v1006>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d0:[<0x1000000055b/picture-86.jpg v1006>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.901 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.603+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg auth v1020 ap=1 s=13050 n(v0 rc2023-10-16T14:35:33.272628+0000 b13050 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54db9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b7:[<0x1000000055b/picture-96.jpg v1020>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005b7:[<0x1000000055b/picture-96.jpg v1020>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.901 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.603+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg auth v1004 ap=1 s=18316 n(v0 rc2023-10-16T14:35:33.390626+0000 b18316 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54dba680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d9:[<0x1000000055b/picture-81.jpg v1004>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d9:[<0x1000000055b/picture-81.jpg v1004>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.901 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.604+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005bd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg auth v1008 ap=1 s=57975 n(v0 rc2023-10-16T14:35:33.292627+0000 b57975 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54dba100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005bd:[<0x1000000055b/picture-94.jpg v1008>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005bd:[<0x1000000055b/picture-94.jpg v1008>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.901 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.604+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000652 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg auth v1000 ap=1 s=22358 n(v0 rc2023-10-16T14:35:33.803619+0000 b22358 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54dbb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000652:[<0x1000000055b/picture-5.jpg v1000>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000652:[<0x1000000055b/picture-5.jpg v1000>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.901 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.606+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000609 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg auth v1024 ap=1 s=33480 n(v0 rc2023-10-16T14:35:33.548623+0000 b33480 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54d5e000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000609:[<0x1000000055b/picture-50.jpg v1024>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000609:[<0x1000000055b/picture-50.jpg v1024>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.902 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.606+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005fd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg auth v1002 ap=1 s=16092 n(v0 rc2023-10-16T14:35:33.509624+0000 b16092 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54d5eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005fd:[<0x1000000055b/picture-58.jpg v1002>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005fd:[<0x1000000055b/picture-58.jpg v1002>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.902 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.606+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f2 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg auth v1028 ap=1 s=58214 n(v0 rc2023-10-16T14:35:33.475624+0000 b58214 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54d5fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f2:[<0x1000000055b/picture-65.jpg v1028>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f2:[<0x1000000055b/picture-65.jpg v1028>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.902 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.616+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000607 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg auth v1018 ap=1 s=44554 n(v0 rc2023-10-16T14:35:33.543623+0000 b44554 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000607:[<0x1000000055b/picture-51.jpg v1018>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000607:[<0x1000000055b/picture-51.jpg v1018>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.902 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.616+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ec [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg auth v1022 ap=1 s=20310 n(v0 rc2023-10-16T14:35:33.455625+0000 b20310 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ec:[<0x1000000055b/picture-69.jpg v1022>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ec:[<0x1000000055b/picture-69.jpg v1022>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.902 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.617+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg auth v996 ap=1 s=45705 n(v0 rc2023-10-16T14:35:33.416625+0000 b45705 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e0:[<0x1000000055b/picture-77.jpg v996>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e0:[<0x1000000055b/picture-77.jpg v996>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.903 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.617+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg auth v998 ap=1 s=13297 n(v0 rc2023-10-16T14:35:33.489624+0000 b13297 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f7:[<0x1000000055b/picture-62.jpg v998>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f7:[<0x1000000055b/picture-62.jpg v998>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.903 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.618+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000060f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg auth v1026 ap=1 s=49140 n(v0 rc2023-10-16T14:35:33.570623+0000 b49140 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000060f:[<0x1000000055b/picture-46.jpg v1026>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000060f:[<0x1000000055b/picture-46.jpg v1026>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.903 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.618+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000656 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg auth v992 ap=1 s=53194 n(v0 rc2023-10-16T14:35:33.821618+0000 b53194 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54cf9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000656:[<0x1000000055b/picture-3.jpg v992>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000656:[<0x1000000055b/picture-3.jpg v992>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.903 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.618+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000601 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg auth v1016 ap=1 s=46720 n(v0 rc2023-10-16T14:35:33.524623+0000 b46720 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c86580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000601:[<0x1000000055b/picture-55.jpg v1016>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000601:[<0x1000000055b/picture-55.jpg v1016>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.904 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.619+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000646 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg auth v996 ap=1 s=57811 n(v0 rc2023-10-16T14:35:33.768619+0000 b57811 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c87080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000646:[<0x1000000055b/picture-11.jpg v996>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000646:[<0x1000000055b/picture-11.jpg v996>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.904 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.620+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml auth v990 ap=1 s=29044 n(v0 rc2023-10-16T14:35:33.200629+0000 b29044 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c87b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a3:[<0x1000000055b/article-5.xml v990>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005a3:[<0x1000000055b/article-5.xml v990>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.904 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.620+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg auth v994 ap=1 s=12709 n(v0 rc2023-10-16T14:35:33.446625+0000 b12709 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c89180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e9:[<0x1000000055b/picture-71.jpg v994>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e9:[<0x1000000055b/picture-71.jpg v994>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.904 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.620+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005eb [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg auth v1010 ap=1 s=61151 n(v0 rc2023-10-16T14:35:33.451625+0000 b61151 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c87600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005eb:[<0x1000000055b/picture-70.jpg v1010>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005eb:[<0x1000000055b/picture-70.jpg v1010>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.904 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.620+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg auth v1014 ap=1 s=14342 n(v0 rc2023-10-16T14:35:33.485624+0000 b14342 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54c89700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f5:[<0x1000000055b/picture-63.jpg v1014>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f5:[<0x1000000055b/picture-63.jpg v1014>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.905 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.620+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000618 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg auth v1000 ap=1 s=45755 n(v0 rc2023-10-16T14:35:33.598622+0000 b45755 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bf8000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000618:[<0x1000000055b/picture-40.jpg v1000>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000618:[<0x1000000055b/picture-40.jpg v1000>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.905 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.621+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e58 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml auth v1130 ap=1 s=3223 n(v0 rc2023-10-16T14:35:40.669501+0000 b3223 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bf8b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e58:[<0x1000000055b/article-22.xml v1130>,<0x10000000223/blog-6 v178237>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e58:[<0x1000000055b/article-22.xml v1130>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.905 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.621+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ce [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg auth v998 ap=1 s=12925 n(v0 rc2023-10-16T14:35:33.351626+0000 b12925 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bf9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ce:[<0x1000000055b/picture-87.jpg v998>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ce:[<0x1000000055b/picture-87.jpg v998>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.905 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.621+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005d4 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg auth v1012 ap=1 s=26252 n(v0 rc2023-10-16T14:35:33.371626+0000 b26252 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bfa100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005d4:[<0x1000000055b/picture-84.jpg v1012>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005d4:[<0x1000000055b/picture-84.jpg v1012>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.905 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.622+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000631 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg auth v948 ap=1 s=49930 n(v0 rc2023-10-16T14:35:33.698620+0000 b49930 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bfb180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000631:[<0x1000000055b/picture-23.jpg v948>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000631:[<0x1000000055b/picture-23.jpg v948>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.906 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.622+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005dd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg auth v952 ap=1 s=56524 n(v0 rc2023-10-16T14:35:33.405626+0000 b56524 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bfb700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005dd:[<0x1000000055b/picture-79.jpg v952>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005dd:[<0x1000000055b/picture-79.jpg v952>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.906 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.622+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000610 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg auth v972 ap=1 s=14832 n(v0 rc2023-10-16T14:35:33.574623+0000 b14832 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000610:[<0x1000000055b/picture-45.jpg v972>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000610:[<0x1000000055b/picture-45.jpg v972>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.906 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.623+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000616 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg auth v974 ap=1 s=9401 n(v0 rc2023-10-16T14:35:33.594622+0000 b9401 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000616:[<0x1000000055b/picture-41.jpg v974>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000616:[<0x1000000055b/picture-41.jpg v974>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.906 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.624+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005fa [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg auth v966 ap=1 s=22187 n(v0 rc2023-10-16T14:35:33.499624+0000 b22187 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005fa:[<0x1000000055b/picture-60.jpg v966>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005fa:[<0x1000000055b/picture-60.jpg v966>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.906 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.624+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000584 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml auth v906 ap=1 s=12917 n(v0 rc2023-10-16T14:35:33.106631+0000 b12917 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000584:[<0x1000000055b/article-14.xml v906>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000584:[<0x1000000055b/article-14.xml v906>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.907 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.625+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005f1 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg auth v982 ap=1 s=64722 n(v0 rc2023-10-16T14:35:33.470624+0000 b64722 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005f1:[<0x1000000055b/picture-66.jpg v982>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005f1:[<0x1000000055b/picture-66.jpg v982>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.907 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.625+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005db [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg auth v968 ap=1 s=29855 n(v0 rc2023-10-16T14:35:33.396626+0000 b29855 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005db:[<0x1000000055b/picture-80.jpg v968>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005db:[<0x1000000055b/picture-80.jpg v968>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.907 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.625+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005c6 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg auth v950 ap=1 s=41704 n(v0 rc2023-10-16T14:35:33.327627+0000 b41704 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005c6:[<0x1000000055b/picture-91.jpg v950>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005c6:[<0x1000000055b/picture-91.jpg v950>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.907 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.625+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000633 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg auth v946 ap=1 s=60075 n(v0 rc2023-10-16T14:35:33.703620+0000 b60075 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb3180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000633:[<0x1000000055b/picture-22.jpg v946>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000633:[<0x1000000055b/picture-22.jpg v946>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.907 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.626+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000058f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml auth v914 ap=1 s=31536 n(v0 rc2023-10-16T14:35:33.139630+0000 b31536 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54bb3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000058f:[<0x1000000055b/article-11.xml v914>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000058f:[<0x1000000055b/article-11.xml v914>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.908 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.626+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b4 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg auth v954 ap=1 s=35776 n(v0 rc2023-10-16T14:35:33.257628+0000 b35776 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b80580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b4:[<0x1000000055b/picture-97.jpg v954>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005b4:[<0x1000000055b/picture-97.jpg v954>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.908 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.626+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000063e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg auth v944 ap=1 s=11611 n(v0 rc2023-10-16T14:35:33.744620+0000 b11611 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b81600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000063e:[<0x1000000055b/picture-15.jpg v944>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000063e:[<0x1000000055b/picture-15.jpg v944>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.908 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.626+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000600 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg auth v964 ap=1 s=62777 n(v0 rc2023-10-16T14:35:33.518624+0000 b62777 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b81b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000600:[<0x1000000055b/picture-56.jpg v964>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000600:[<0x1000000055b/picture-56.jpg v964>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.908 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.626+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000060d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg auth v976 ap=1 s=15645 n(v0 rc2023-10-16T14:35:33.564623+0000 b15645 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b82100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000060d:[<0x1000000055b/picture-47.jpg v976>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000060d:[<0x1000000055b/picture-47.jpg v976>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.908 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.627+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000057d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml auth v908 ap=1 s=27153 n(v0 rc2023-10-16T14:35:33.090631+0000 b27153 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b80b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000057d:[<0x1000000055b/article-16.xml v908>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000057d:[<0x1000000055b/article-16.xml v908>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.909 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.627+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000592 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml auth v912 ap=1 s=16652 n(v0 rc2023-10-16T14:35:33.151630+0000 b16652 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000592:[<0x1000000055b/article-10.xml v912>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000592:[<0x1000000055b/article-10.xml v912>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.909 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.627+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000644 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg auth v942 ap=1 s=48718 n(v0 rc2023-10-16T14:35:33.762619+0000 b48718 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b82c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000644:[<0x1000000055b/picture-12.jpg v942>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000644:[<0x1000000055b/picture-12.jpg v942>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.909 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.627+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000612 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg auth v980 ap=1 s=12822 n(v0 rc2023-10-16T14:35:33.580623+0000 b12822 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b83180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000612:[<0x1000000055b/picture-44.jpg v980>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000612:[<0x1000000055b/picture-44.jpg v980>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.909 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.628+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005a7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml auth v940 ap=1 s=29495 n(v0 rc2023-10-16T14:35:33.211629+0000 b29495 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005a7:[<0x1000000055b/article-4.xml v940>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005a7:[<0x1000000055b/article-4.xml v940>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.909 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.629+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005fb [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg auth v970 ap=1 s=63030 n(v0 rc2023-10-16T14:35:33.505624+0000 b63030 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005fb:[<0x1000000055b/picture-59.jpg v970>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005fb:[<0x1000000055b/picture-59.jpg v970>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.910 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.629+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg auth v978 ap=1 s=21660 n(v0 rc2023-10-16T14:35:33.441625+0000 b21660 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e8:[<0x1000000055b/picture-72.jpg v978>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e8:[<0x1000000055b/picture-72.jpg v978>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.910 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.629+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000055e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml auth v910 ap=1 s=11270 n(v0 rc2023-10-16T14:35:32.997633+0000 b11270 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000055e:[<0x1000000055b/article-25.xml v910>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000055e:[<0x1000000055b/article-25.xml v910>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.910 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.629+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000634 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg auth v1011 ap=1 s=24587 n(v0 rc2023-10-16T14:35:33.709620+0000 b24587 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000634:[<0x1000000055b/picture-21.jpg v1011>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000634:[<0x1000000055b/picture-21.jpg v1011>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.910 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.630+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e6 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg auth v1025 ap=1 s=26388 n(v0 rc2023-10-16T14:35:33.435625+0000 b26388 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b4e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e6:[<0x1000000055b/picture-73.jpg v1025>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e6:[<0x1000000055b/picture-73.jpg v1025>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.911 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.630+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg auth v1029 ap=1 s=30604 n(v0 rc2023-10-16T14:35:33.426625+0000 b30604 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b22000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e3:[<0x1000000055b/picture-75.jpg v1029>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e3:[<0x1000000055b/picture-75.jpg v1029>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.911 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.632+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000061e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg auth v1023 ap=1 s=31893 n(v0 rc2023-10-16T14:35:33.618622+0000 b31893 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54ab4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000061e:[<0x1000000055b/picture-36.jpg v1023>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000061e:[<0x1000000055b/picture-36.jpg v1023>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.911 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.632+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000613 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg auth v1027 ap=1 s=12488 n(v0 rc2023-10-16T14:35:33.584622+0000 b12488 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54ab4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000613:[<0x1000000055b/picture-43.jpg v1027>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000613:[<0x1000000055b/picture-43.jpg v1027>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.911 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.632+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000573 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml auth v908 ap=1 s=22785 n(v0 rc2023-10-16T14:35:33.058631+0000 b22785 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54ab5080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000573:[<0x1000000055b/article-19.xml v908>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000573:[<0x1000000055b/article-19.xml v908>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.911 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.633+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e29 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml auth v1148 ap=1 s=32256 n(v0 rc2023-10-16T14:35:40.524503+0000 b32256 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54b24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e29:[<0x1000000055b/article-15.xml v1148>,<0x10000000223/blog-6 v178237>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e29:[<0x1000000055b/article-15.xml v1148>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.912 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.633+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000636 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg auth v1009 ap=1 s=61393 n(v0 rc2023-10-16T14:35:33.714620+0000 b61393 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54ab5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000636:[<0x1000000055b/picture-20.jpg v1009>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000636:[<0x1000000055b/picture-20.jpg v1009>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.912 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.634+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000061b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg auth v1033 ap=1 s=52835 n(v0 rc2023-10-16T14:35:33.608622+0000 b52835 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54ab7700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000061b:[<0x1000000055b/picture-38.jpg v1033>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000061b:[<0x1000000055b/picture-38.jpg v1033>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.912 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.635+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000062b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg auth v1031 ap=1 s=27422 n(v0 rc2023-10-16T14:35:33.675621+0000 b27422 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a71080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000062b:[<0x1000000055b/picture-27.jpg v1031>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000062b:[<0x1000000055b/picture-27.jpg v1031>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.912 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.636+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000606 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg auth v1017 ap=1 s=38592 n(v0 rc2023-10-16T14:35:33.538623+0000 b38592 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a72680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000606:[<0x1000000055b/picture-52.jpg v1017>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000606:[<0x1000000055b/picture-52.jpg v1017>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.912 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.636+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000058b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml auth v908 ap=1 s=11319 n(v0 rc2023-10-16T14:35:33.131630+0000 b11319 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000058b:[<0x1000000055b/article-12.xml v908>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000058b:[<0x1000000055b/article-12.xml v908>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.913 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.638+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005b0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml auth v1003 ap=1 s=25902 n(v0 rc2023-10-16T14:35:33.246628+0000 b25902 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a72c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005b0:[<0x1000000055b/article-1.xml v1003>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005b0:[<0x1000000055b/article-1.xml v1003>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.913 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.638+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000621 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg auth v1027 ap=1 s=33363 n(v0 rc2023-10-16T14:35:33.629622+0000 b33363 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a73700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000621:[<0x1000000055b/picture-34.jpg v1027>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000621:[<0x1000000055b/picture-34.jpg v1027>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.913 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.638+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005ef [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg auth v1025 ap=1 s=28990 n(v0 rc2023-10-16T14:35:33.465624+0000 b28990 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a56580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005ef:[<0x1000000055b/picture-67.jpg v1025>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005ef:[<0x1000000055b/picture-67.jpg v1025>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.913 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.638+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000604 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg auth v1029 ap=1 s=39278 n(v0 rc2023-10-16T14:35:33.533623+0000 b39278 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a56000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000604:[<0x1000000055b/picture-53.jpg v1029>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000604:[<0x1000000055b/picture-53.jpg v1029>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.913 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.638+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005c8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg auth v1021 ap=1 s=25268 n(v0 rc2023-10-16T14:35:33.332627+0000 b25268 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a56b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005c8:[<0x1000000055b/picture-90.jpg v1021>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005c8:[<0x1000000055b/picture-90.jpg v1021>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.914 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.639+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000063c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg auth v1005 ap=1 s=55213 n(v0 rc2023-10-16T14:35:33.738620+0000 b55213 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a57b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000063c:[<0x1000000055b/picture-16.jpg v1005>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000063c:[<0x1000000055b/picture-16.jpg v1005>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.914 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.639+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg auth v1007 ap=1 s=58094 n(v0 rc2023-10-16T14:35:33.431625+0000 b58094 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a58680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e5:[<0x1000000055b/picture-74.jpg v1007>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e5:[<0x1000000055b/picture-74.jpg v1007>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.914 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.641+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000005e2 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg auth v1009 ap=1 s=16767 n(v0 rc2023-10-16T14:35:33.421625+0000 b16767 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a13080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000005e2:[<0x1000000055b/picture-76.jpg v1009>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000005e2:[<0x1000000055b/picture-76.jpg v1009>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.914 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.641+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000624 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg auth v1019 ap=1 s=44838 n(v0 rc2023-10-16T14:35:33.648621+0000 b44838 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a13600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000624:[<0x1000000055b/picture-32.jpg v1019>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000624:[<0x1000000055b/picture-32.jpg v1019>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.914 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.642+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000057a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml auth v904 ap=1 s=28786 n(v0 rc2023-10-16T14:35:33.076631+0000 b28786 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a12b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000057a:[<0x1000000055b/article-17.xml v904>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000057a:[<0x1000000055b/article-17.xml v904>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.915 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.642+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000060c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg auth v1023 ap=1 s=38237 n(v0 rc2023-10-16T14:35:33.558623+0000 b38237 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a14100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000060c:[<0x1000000055b/picture-48.jpg v1023>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000060c:[<0x1000000055b/picture-48.jpg v1023>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:44.915 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:39:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:39:44.643+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000566 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml auth v906 ap=1 s=20103 n(v0 rc2023-10-16T14:35:33.021632+0000 b20103 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54a12580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000566:[<0x1000000055b/article-23.xml v906>,<0x10000000223/blog-6 v173298>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000566:[<0x1000000055b/article-23.xml v906>,<0x10000000223/blog-6 v190173>,<0x200000001fe/blogtest_in v2272>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v92>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:39:45.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:44 smithi186 ceph-mon[134892]: scrub queued for path: / 2023-10-16T14:39:45.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:44 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:45.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:44 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:39:45.360 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:44 smithi178 ceph-mon[98495]: scrub queued for path: / 2023-10-16T14:39:45.360 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:44 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:39:45.361 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:44 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:39:46.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000642 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000638 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000577 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000637 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000654 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000061f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005c0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000658 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005df (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000640 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000064e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.181 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000064a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000648 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000659 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.182 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000650 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000563 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000062a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005c2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000061c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.183 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000622 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000630 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005fe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.184 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000619 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000060a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000064c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.185 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005cc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000062d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.186 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000588 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000628 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000603 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.187 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.188 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000652 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000609 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.189 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000607 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.190 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000060f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000656 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000601 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000646 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.191 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000618 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.192 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005d4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000631 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.193 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000610 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000616 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.194 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005fa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000584 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.195 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005db (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005c6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.196 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000633 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000058f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000063e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000600 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.197 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000060d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000644 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000612 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.198 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005fb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000055e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000634 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.199 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000061e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000613 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000573 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.200 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000636 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000061b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000062b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.201 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000606 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000058b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000621 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000604 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.202 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005c8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000063c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.203 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000005e2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.204 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000057a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000060c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000566 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.205 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:45 smithi187 ceph-mon[135297]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-10-16T14:39:46.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000642 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000638 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000577 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000637 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000654 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.309 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000061f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.309 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005c0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.309 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.309 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000658 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005df (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000640 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000064e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000064a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000648 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000659 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000650 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000563 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000062a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005c2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000061c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000622 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000630 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005fe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000619 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000060a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000064c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005cc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000062d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000588 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000628 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000603 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000652 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000609 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000607 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000060f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000656 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000601 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000646 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000618 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005d4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000631 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000610 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000616 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005fa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000584 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005db (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005c6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000633 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000058f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000063e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000600 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000060d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000644 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000612 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005fb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000055e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000634 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000061e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000613 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000573 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.331 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.331 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000636 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.331 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000061b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.331 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000062b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000606 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000058b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000621 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000604 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005c8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.333 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000063c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000005e2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.334 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000057a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000060c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000566 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.335 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:45 smithi178 ceph-mon[98495]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-10-16T14:39:46.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000642 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000638 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000577 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000637 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000654 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000061f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005c0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000658 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005df (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000640 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000570 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000064e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000064a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000648 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000659 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000650 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000563 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000062a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005c2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000061c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005a9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000622 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000630 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000059b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005fe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000619 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000625 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000060a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000064c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000598 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005cc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000062d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000056d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000588 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000059e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ac (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000628 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000603 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005b7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000652 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000609 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000607 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000060f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000656 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000601 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000646 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005a3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005eb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000618 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005d4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000631 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000610 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000616 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005fa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000584 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005db (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005c6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000633 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000058f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005b4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000063e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000600 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000060d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000057d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000592 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000644 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000612 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005a7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005fb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000055e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000634 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000061e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000613 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000573 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000636 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000061b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000062b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000606 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000058b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000621 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005ef (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000604 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005c8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000063c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000005e2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000624 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000057a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000060c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000566 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:39:46.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:45 smithi186 ceph-mon[134892]: pgmap v520: 129 pgs: 129 active+clean; 1.4 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 254 op/s 2023-10-16T14:39:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:47 smithi187 ceph-mon[135297]: pgmap v521: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2023-10-16T14:39:48.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:47 smithi178 ceph-mon[98495]: pgmap v521: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2023-10-16T14:39:48.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:47 smithi186 ceph-mon[134892]: pgmap v521: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2023-10-16T14:39:49.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:39:48.929+0000 7f6dfdbef700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-16T14:39:49.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:48 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:49.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:48 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:49.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:48 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:50.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:49 smithi178 ceph-mon[98495]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2023-10-16T14:39:50.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:49 smithi178 ceph-mon[98495]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-16T14:39:50.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:49 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:39:50.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:49 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:49 smithi186 ceph-mon[134892]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2023-10-16T14:39:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:49 smithi186 ceph-mon[134892]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-16T14:39:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:49 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:39:50.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:49 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:49 smithi187 ceph-mon[135297]: pgmap v522: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 292 op/s 2023-10-16T14:39:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:49 smithi187 ceph-mon[135297]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-10-16T14:39:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:49 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] up:active 2023-10-16T14:39:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:49 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:51.080 INFO:tasks.workunit.client.0.smithi178.stdout: 358 553974 782 387610 682 272523 1423 2023-10-16T14:39:51.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.132+0000 7f1b97fff700 1 -- 172.21.15.178:0/2498927512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 msgr2=0x7f1b90108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:51.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.132+0000 7f1b97fff700 1 --2- 172.21.15.178:0/2498927512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b90108b60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1b8401a9d0 tx=0x7f1b8401ace0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.137+0000 7f1b97fff700 1 -- 172.21.15.178:0/2498927512 shutdown_connections 2023-10-16T14:39:51.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.137+0000 7f1b97fff700 1 --2- 172.21.15.178:0/2498927512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b9010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.137+0000 7f1b97fff700 1 --2- 172.21.15.178:0/2498927512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 0x7f1b90107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.137+0000 7f1b97fff700 1 --2- 172.21.15.178:0/2498927512 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b90108b60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.137+0000 7f1b97fff700 1 -- 172.21.15.178:0/2498927512 >> 172.21.15.178:0/2498927512 conn(0x7f1b900764b0 msgr2=0x7f1b900788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:51.180 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.176+0000 7f1b97fff700 1 -- 172.21.15.178:0/2498927512 shutdown_connections 2023-10-16T14:39:51.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.192+0000 7f1b97fff700 1 -- 172.21.15.178:0/2498927512 wait complete. 2023-10-16T14:39:51.200 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.196+0000 7f1b97fff700 1 Processor -- start 2023-10-16T14:39:51.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.208+0000 7f1b97fff700 1 -- start start 2023-10-16T14:39:51.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.220+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 0x7f1b9007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.225+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b9007dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.225+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b90080170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.290 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.225+0000 7f1b97fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90080910 con 0x7f1b901090a0 2023-10-16T14:39:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.225+0000 7f1b97fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90080a80 con 0x7f1b90108700 2023-10-16T14:39:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.225+0000 7f1b97fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90080c40 con 0x7f1b90107510 2023-10-16T14:39:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.228+0000 7f1b9659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b90080170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.229+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b9007dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.291 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.229+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b9007dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:38518/0 (socket says 172.21.15.178:38518) 2023-10-16T14:39:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.229+0000 7f1b9559a700 1 -- 172.21.15.178:0/3723377971 learned_addr learned my addr 172.21.15.178:0/3723377971 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.232+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 0x7f1b9007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.234+0000 7f1b9659c700 1 -- 172.21.15.178:0/3723377971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 msgr2=0x7f1b9007b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.234+0000 7f1b9659c700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 0x7f1b9007b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.237+0000 7f1b9659c700 1 -- 172.21.15.178:0/3723377971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 msgr2=0x7f1b9007dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.237+0000 7f1b9659c700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b9007dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.237+0000 7f1b9659c700 1 -- 172.21.15.178:0/3723377971 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8401a600 con 0x7f1b901090a0 2023-10-16T14:39:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.241+0000 7f1b9659c700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b90080170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b780 tx=0x7f1b8800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.245+0000 7f1b8effd700 1 -- 172.21.15.178:0/3723377971 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88010660 con 0x7f1b901090a0 2023-10-16T14:39:51.293 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.245+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b90080ec0 con 0x7f1b901090a0 2023-10-16T14:39:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.246+0000 7f1b8effd700 1 -- 172.21.15.178:0/3723377971 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b88010c80 con 0x7f1b901090a0 2023-10-16T14:39:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.250+0000 7f1b8effd700 1 -- 172.21.15.178:0/3723377971 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88010660 con 0x7f1b901090a0 2023-10-16T14:39:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.252+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b90081450 con 0x7f1b901090a0 2023-10-16T14:39:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.258+0000 7f1bab902b80 1 Processor -- start 2023-10-16T14:39:51.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.263+0000 7f1b8effd700 1 -- 172.21.15.178:0/3723377971 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1b8800f3c0 con 0x7f1b901090a0 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.265+0000 7f1bab902b80 1 -- start start 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.266+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 0x5647c94ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.266+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 0x5647c94ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 0x5647c94ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:38528/0 (socket says 172.21.15.178:38528) 2023-10-16T14:39:51.295 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52578/0 (socket says 172.21.15.178:52578) 2023-10-16T14:39:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1b9559a700 1 -- 172.21.15.178:0/4001868740 learned_addr learned my addr 172.21.15.178:0/4001868740 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ae630 0x5647c94aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c93ffc70 con 0x5647c94ad790 2023-10-16T14:39:51.296 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c93e4c60 con 0x5647c94ace30 2023-10-16T14:39:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.267+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c93e2580 con 0x5647c94ae630 2023-10-16T14:39:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.268+0000 7f1b95d9b700 1 -- 172.21.15.178:0/4001868740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ae630 msgr2=0x5647c94aeab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.268+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ae630 0x5647c94aeab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.268+0000 7f1b95d9b700 1 -- 172.21.15.178:0/4001868740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 msgr2=0x5647c94ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:51.297 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.268+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 0x5647c94ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.268+0000 7f1b95d9b700 1 -- 172.21.15.178:0/4001868740 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c939b3f0 con 0x5647c94ad790 2023-10-16T14:39:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.269+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009c00 tx=0x7f1b80009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae820368f956778b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.270+0000 7f1b8effd700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 0x7f1b7c0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.271+0000 7f1b9559a700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 0x7f1b7c0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.298 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.272+0000 7f1b9559a700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 0x7f1b7c0a3960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b8401a360 tx=0x7f1b84005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:51.299 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.281+0000 7f1b8effd700 1 -- 172.21.15.178:0/3723377971 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1b880cc5d0 con 0x7f1b901090a0 2023-10-16T14:39:51.321 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.308+0000 7f1b7bfff700 1 -- 172.21.15.178:0/4001868740 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80014600 con 0x5647c94ad790 2023-10-16T14:39:51.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.308+0000 7f1b7bfff700 1 -- 172.21.15.178:0/4001868740 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b80007e10 con 0x5647c94ad790 2023-10-16T14:39:51.322 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.308+0000 7f1b7bfff700 1 -- 172.21.15.178:0/4001868740 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800139d0 con 0x5647c94ad790 2023-10-16T14:39:51.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.345+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 msgr2=0x5647c94ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:51.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.345+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009c00 tx=0x7f1b80009f10 comp rx=0 tx=0).stop 2023-10-16T14:39:51.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.366+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 shutdown_connections 2023-10-16T14:39:51.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.366+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ae630 0x5647c94aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.366+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ace30 0x5647c94ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.366+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4001868740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c94ae050 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.366+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 >> 172.21.15.178:0/4001868740 conn(0x5647c93b8dd0 msgr2=0x5647c9485f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:51.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.367+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 shutdown_connections 2023-10-16T14:39:51.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.367+0000 7f1bab902b80 1 -- 172.21.15.178:0/4001868740 wait complete. 2023-10-16T14:39:51.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.368+0000 7f1bab902b80 1 -- start start 2023-10-16T14:39:51.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.368+0000 7f1bab902b80 1 Processor -- start 2023-10-16T14:39:51.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.370+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.371+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.372+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c94911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.372+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.425+0000 7f1bab902b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c93dde70 con 0x5647c94ad790 2023-10-16T14:39:51.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.425+0000 7f1bab902b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c937a710 con 0x5647c94ae630 2023-10-16T14:39:51.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.425+0000 7f1bab902b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c937a3b0 con 0x5647c94ace30 2023-10-16T14:39:51.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.425+0000 7f1bab902b80 10 client.0 _ll_register_callbacks cb 0x5647c91d7800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:51.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.378+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.426+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54460/0 (socket says 172.21.15.178:54460) 2023-10-16T14:39:51.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.426+0000 7f1b95d9b700 1 -- 172.21.15.178:0/1390517929 learned_addr learned my addr 172.21.15.178:0/1390517929 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:51.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.435+0000 7f1b95d9b700 1 -- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 msgr2=0x5647c94911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:51.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.380+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52592/0 (socket says 172.21.15.178:52592) 2023-10-16T14:39:51.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.405+0000 7f1b9659c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c94911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.451+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c94911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.451+0000 7f1b95d9b700 1 -- 172.21.15.178:0/1390517929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 msgr2=0x5647c948a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:51.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.451+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.451+0000 7f1b95d9b700 1 -- 172.21.15.178:0/1390517929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b80009860 con 0x5647c94ace30 2023-10-16T14:39:51.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.452+0000 7f1b9559a700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:51.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.453+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b80013d70 tx=0x7f1b80013e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33414844b8daead2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:51.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.453+0000 7f1b9659c700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c94911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:39:51.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.465+0000 7f1b797fa700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:39:51.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.467+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80005030 con 0x5647c94ace30 2023-10-16T14:39:51.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.467+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1b80005190 con 0x5647c94ace30 2023-10-16T14:39:51.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.473+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647c93e2ef0 con 0x5647c94ace30 2023-10-16T14:39:51.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.477+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b80005350 con 0x5647c94ace30 2023-10-16T14:39:51.536 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.534+0000 7f1bab902b80 10 client.26218 fetch_fsmap learned FSMap version 35 2023-10-16T14:39:51.537 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.534+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5647c9208de0 con 0x5647c94ace30 2023-10-16T14:39:51.539 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.535+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647c93e2ef0 con 0x5647c94ace30 2023-10-16T14:39:51.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.535+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1b8001e910 con 0x5647c94ace30 2023-10-16T14:39:51.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 10 client.26218 fetch_fsmap finished waiting for FSMap version 35 2023-10-16T14:39:51.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 10 client.26218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:51.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 20 client.26218 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:51.540 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c9495980 0x5647c9497da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:51.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 4 client.26218 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:39:51.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.536+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647c945ac10 con 0x5647c9495980 2023-10-16T14:39:51.541 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.537+0000 7f1b9559a700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c9495980 0x5647c9497da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:51.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.540+0000 7f1b9559a700 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c9495980 0x5647c9497da0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230e878e9191c1f3 server_cookie=a91e76a0f9856834 in_seq=0 out_seq=0 2023-10-16T14:39:51.543 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.540+0000 7f1b797fa700 10 client.26218 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:51.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.546+0000 7f1b797fa700 1 -- 172.21.15.178:0/1390517929 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647c945ac10 con 0x5647c9495980 2023-10-16T14:39:51.549 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.546+0000 7f1b797fa700 10 client.26218 handle_command_reply: tid=0 2023-10-16T14:39:51.550 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.546+0000 7f1bab902b80 1 client.26218 shutdown 2023-10-16T14:39:51.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.558+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 msgr2=0x5647c9489c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:51.564 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.558+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b80013d70 tx=0x7f1b80013e50 comp rx=0 tx=0).stop 2023-10-16T14:39:51.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 shutdown_connections 2023-10-16T14:39:51.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c9495980 0x5647c9497da0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c94911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ace30 0x5647c9489c40 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b80013d70 tx=0x7f1b80013e50 comp rx=0 tx=0).stop 2023-10-16T14:39:51.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 --2- 172.21.15.178:0/1390517929 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:51.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.559+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 >> 172.21.15.178:0/1390517929 conn(0x5647c93b8dd0 msgr2=0x5647c9485b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:51.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.560+0000 7f1b797fa700 0 client.26218 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:51.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.575+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 shutdown_connections 2023-10-16T14:39:51.578 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.575+0000 7f1bab902b80 1 -- 172.21.15.178:0/1390517929 wait complete. 2023-10-16T14:39:51.606 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:51.600+0000 7f1bab902b80 20 client.26218 trim_cache size 0 max 16384 2023-10-16T14:39:51.677 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:51 smithi187 ceph-mon[135297]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-16T14:39:51.816 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:51 smithi178 ceph-mon[98495]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-16T14:39:51.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:51 smithi186 ceph-mon[134892]: pgmap v523: 129 pgs: 129 active+clean; 1.4 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.9 MiB/s wr, 479 op/s 2023-10-16T14:39:52.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.119+0000 7f1bab902b80 1 Processor -- start 2023-10-16T14:39:52.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.119+0000 7f1bab902b80 1 -- start start 2023-10-16T14:39:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.120+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 0x5647c9497990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.120+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 0x5647c9497990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1b95d9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 0x5647c9497990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:54470/0 (socket says 172.21.15.178:54470) 2023-10-16T14:39:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52594/0 (socket says 172.21.15.178:52594) 2023-10-16T14:39:52.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1b95d9b700 1 -- 172.21.15.178:0/2633638309 learned_addr learned my addr 172.21.15.178:0/2633638309 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:52.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.121+0000 7f1bab902b80 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 0x5647c94887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.125+0000 7f1b9659c700 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 0x5647c94887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c925e920 con 0x5647c94ae630 2023-10-16T14:39:52.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c9484ee0 con 0x5647c9497ed0 2023-10-16T14:39:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c94858c0 con 0x5647c94ad790 2023-10-16T14:39:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1b9659c700 1 -- 172.21.15.178:0/2633638309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 msgr2=0x5647c9497990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1b9659c700 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 0x5647c9497990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1b9659c700 1 -- 172.21.15.178:0/2633638309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 msgr2=0x5647c9491180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:39:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.130+0000 7f1b9659c700 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.131+0000 7f1b9659c700 1 -- 172.21.15.178:0/2633638309 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647c91f1c60 con 0x5647c9497ed0 2023-10-16T14:39:52.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.131+0000 7f1b9559a700 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:39:52.192 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.138+0000 7f1b9659c700 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 0x5647c94887b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b750 tx=0x7f1b88095990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a18c6a938d4931b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:52.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.180+0000 7f1b8cff9700 1 -- 172.21.15.178:0/2633638309 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88095040 con 0x5647c9497ed0 2023-10-16T14:39:52.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.180+0000 7f1b8cff9700 1 -- 172.21.15.178:0/2633638309 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b88099520 con 0x5647c9497ed0 2023-10-16T14:39:52.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.180+0000 7f1b8cff9700 1 -- 172.21.15.178:0/2633638309 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880999e0 con 0x5647c9497ed0 2023-10-16T14:39:52.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.216+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 msgr2=0x5647c94887b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:52.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.216+0000 7f1bab902b80 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 0x5647c94887b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b750 tx=0x7f1b88095990 comp rx=0 tx=0).stop 2023-10-16T14:39:52.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.287+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 shutdown_connections 2023-10-16T14:39:52.315 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.287+0000 7f1bab902b80 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c9497ed0 0x5647c94887b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.287+0000 7f1bab902b80 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c94ad790 0x5647c9497990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.287+0000 7f1bab902b80 1 --2- 172.21.15.178:0/2633638309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ae630 0x5647c9491180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.316 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.287+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 >> 172.21.15.178:0/2633638309 conn(0x5647c93b8dd0 msgr2=0x5647c94847a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:52.329 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.327+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 shutdown_connections 2023-10-16T14:39:52.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.329+0000 7f1bab902b80 1 -- 172.21.15.178:0/2633638309 wait complete. 2023-10-16T14:39:52.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.329+0000 7f1bab902b80 1 -- start start 2023-10-16T14:39:52.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.329+0000 7f1bab902b80 1 Processor -- start 2023-10-16T14:39:52.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c9497ed0 0x5647c948c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c948fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c94902e0 con 0x5647c94ad790 2023-10-16T14:39:52.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c9490450 con 0x5647c94ae630 2023-10-16T14:39:52.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647c9490610 con 0x5647c9497ed0 2023-10-16T14:39:52.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.364+0000 7f1bab902b80 10 client.0 _ll_register_callbacks cb 0x7f1b80009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:39:52.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.370+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:52.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.371+0000 7f1b9559a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52596/0 (socket says 172.21.15.178:52596) 2023-10-16T14:39:52.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.371+0000 7f1b9559a700 1 -- 172.21.15.178:0/4283598927 learned_addr learned my addr 172.21.15.178:0/4283598927 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:39:52.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.378+0000 7f1b9559a700 1 -- 172.21.15.178:0/4283598927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c9497ed0 msgr2=0x5647c948c840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:52.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.378+0000 7f1b9559a700 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c9497ed0 0x5647c948c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.378+0000 7f1b9559a700 1 -- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 msgr2=0x5647c948fda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:39:52.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.378+0000 7f1b9559a700 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c948fda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.378+0000 7f1b9559a700 1 -- 172.21.15.178:0/4283598927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b90081450 con 0x5647c94ad790 2023-10-16T14:39:52.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.381+0000 7f1b9559a700 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5647c94986b0 tx=0x7f1b8401c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d4295a287eb6d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:39:52.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.399+0000 7f1b797fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:39:52.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.399+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84088ea0 con 0x5647c94ad790 2023-10-16T14:39:52.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.399+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1b84005680 con 0x5647c94ad790 2023-10-16T14:39:52.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.400+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b80007e10 con 0x5647c94ad790 2023-10-16T14:39:52.441 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.422+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b8401cb70 con 0x5647c94ad790 2023-10-16T14:39:52.442 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.439+0000 7f1bab902b80 10 client.16182 fetch_fsmap learned FSMap version 35 2023-10-16T14:39:52.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.439+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5647c922f250 con 0x5647c94ad790 2023-10-16T14:39:52.461 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.458+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b80007e10 con 0x5647c94ad790 2023-10-16T14:39:52.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.458+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 35) v1 ==== 7910+0+0 (secure 0 0 0) 0x7f1b84084420 con 0x5647c94ad790 2023-10-16T14:39:52.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 10 client.16182 fetch_fsmap finished waiting for FSMap version 35 2023-10-16T14:39:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:39:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 20 client.16182 populate_metadata read hostname 'smithi178' 2023-10-16T14:39:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c94f6090 0x5647c94f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:39:52.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 4 client.16182 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:39:52.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.484+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5647c93d6e50 con 0x5647c94f6090 2023-10-16T14:39:52.501 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.497+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c94f6090 0x5647c94f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:39:52.556 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.538+0000 7f1b797fa700 10 client.16182 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.540+0000 7f1b95d9b700 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c94f6090 0x5647c94f84b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b9becd80550842 server_cookie=c245f4c0f47a98e8 in_seq=0 out_seq=0 2023-10-16T14:39:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.543+0000 7f1b797fa700 1 -- 172.21.15.178:0/4283598927 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5647c93d6e50 con 0x5647c94f6090 2023-10-16T14:39:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.543+0000 7f1b797fa700 10 client.16182 handle_command_reply: tid=0 2023-10-16T14:39:52.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.543+0000 7f1bab902b80 1 client.16182 shutdown 2023-10-16T14:39:52.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.554+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 msgr2=0x5647c948cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:52.558 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.554+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x5647c94986b0 tx=0x7f1b8401c550 comp rx=0 tx=0).stop 2023-10-16T14:39:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 shutdown_connections 2023-10-16T14:39:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647c94f6090 0x5647c94f84b0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.565 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647c94ae630 0x5647c948fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647c9497ed0 0x5647c948c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 --2- 172.21.15.178:0/4283598927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647c94ad790 0x5647c948cd80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 >> 172.21.15.178:0/4283598927 conn(0x5647c93b8dd0 msgr2=0x5647c94b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.558+0000 7f1b797fa700 0 client.16182 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:39:52.566 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.560+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 shutdown_connections 2023-10-16T14:39:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.560+0000 7f1bab902b80 1 -- 172.21.15.178:0/4283598927 wait complete. 2023-10-16T14:39:52.567 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.562+0000 7f1bab902b80 20 client.16182 trim_cache size 0 max 16384 2023-10-16T14:39:52.569 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:39:52.569 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "scrub active (7625 inodes in the stack)", 2023-10-16T14:39:52.569 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:39:52.569 INFO:teuthology.orchestra.run.smithi178.stdout: "fca1b280-3016-4a9c-8fd4-d0bd2508c9a7": { 2023-10-16T14:39:52.570 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:39:52.570 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "fca1b280-3016-4a9c-8fd4-d0bd2508c9a7", 2023-10-16T14:39:52.570 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:39:52.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:39:52.570 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:39:52.571 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:39:52.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.708+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 msgr2=0x7f1b7c0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:52.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.708+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 0x7f1b7c0a3960 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1b8401a360 tx=0x7f1b84005fd0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.708+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 msgr2=0x7f1b90080170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:39:52.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.708+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b90080170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b780 tx=0x7f1b8800bb40 comp rx=0 tx=0).stop 2023-10-16T14:39:52.723 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 shutdown_connections 2023-10-16T14:39:52.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1b7c0a14a0 0x7f1b7c0a3960 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b901090a0 0x7f1b90080170 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b90107510 0x7f1b9007b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 --2- 172.21.15.178:0/3723377971 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1b90108700 0x7f1b9007dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:39:52.724 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.716+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 >> 172.21.15.178:0/3723377971 conn(0x7f1b900764b0 msgr2=0x7f1b9006fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:39:52.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.777+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 shutdown_connections 2023-10-16T14:39:52.807 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:39:52.780+0000 7f1b97fff700 1 -- 172.21.15.178:0/3723377971 wait complete. 2023-10-16T14:39:53.462 INFO:tasks.cephfs.filesystem:scrub status for tag:fca1b280-3016-4a9c-8fd4-d0bd2508c9a7 - {'path': '/', 'tag': 'fca1b280-3016-4a9c-8fd4-d0bd2508c9a7', 'options': 'recursive,force'} 2023-10-16T14:39:53.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:53 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:53.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:53 smithi186 ceph-mon[134892]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 513 op/s 2023-10-16T14:39:53.829 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:53 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:53.830 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:53 smithi178 ceph-mon[98495]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 513 op/s 2023-10-16T14:39:53.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:53 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:39:53.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:53 smithi187 ceph-mon[135297]: pgmap v524: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 513 op/s 2023-10-16T14:39:56.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:55 smithi187 ceph-mon[135297]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 591 op/s 2023-10-16T14:39:56.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:55 smithi178 ceph-mon[98495]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 591 op/s 2023-10-16T14:39:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:55 smithi186 ceph-mon[134892]: pgmap v525: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 591 op/s 2023-10-16T14:39:58.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:57 smithi187 ceph-mon[135297]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-16T14:39:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:39:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:39:58.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:57 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:39:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:57 smithi178 ceph-mon[98495]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-16T14:39:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:39:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:39:58.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:57 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:39:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:57 smithi186 ceph-mon[134892]: pgmap v526: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 711 op/s 2023-10-16T14:39:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:39:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:39:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:57 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:40:00.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:39:59 smithi187 ceph-mon[135297]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-16T14:40:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:39:59 smithi178 ceph-mon[98495]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-16T14:40:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:39:59.999+0000 7f6dfdbef700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-16T14:40:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:39:59.999+0000 7f6dfdbef700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-16T14:40:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:39:59.999+0000 7f6dfdbef700 -1 log_channel(cluster) log [ERR] : mds.i(mds.1): Metadata damage detected 2023-10-16T14:40:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:39:59 smithi186 ceph-mon[134892]: pgmap v527: 129 pgs: 129 active+clean; 1.5 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 682 op/s 2023-10-16T14:40:01.060 INFO:tasks.workunit.client.0.smithi178.stdout: 363 516547 136 363326 76 254142 753 2023-10-16T14:40:01.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:00 smithi187 ceph-mon[135297]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-16T14:40:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:00 smithi187 ceph-mon[135297]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-16T14:40:01.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:00 smithi187 ceph-mon[135297]: mds.i(mds.1): Metadata damage detected 2023-10-16T14:40:01.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-mon[98495]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-16T14:40:01.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-mon[98495]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-16T14:40:01.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:00 smithi178 ceph-mon[98495]: mds.i(mds.1): Metadata damage detected 2023-10-16T14:40:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:00 smithi186 ceph-mon[134892]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2023-10-16T14:40:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:00 smithi186 ceph-mon[134892]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2023-10-16T14:40:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:00 smithi186 ceph-mon[134892]: mds.i(mds.1): Metadata damage detected 2023-10-16T14:40:02.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:01 smithi187 ceph-mon[135297]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-16T14:40:02.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:01 smithi178 ceph-mon[98495]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-16T14:40:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:01 smithi186 ceph-mon[134892]: pgmap v528: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 654 op/s 2023-10-16T14:40:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:03 smithi187 ceph-mon[135297]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2023-10-16T14:40:04.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:03 smithi178 ceph-mon[98495]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2023-10-16T14:40:04.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:03 smithi186 ceph-mon[134892]: pgmap v529: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2023-10-16T14:40:04.912 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:04 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:05.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:04 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:05.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:04 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:05.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-mon[135297]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-16T14:40:05.926 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.664+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e15 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml auth v1037 ap=1 s=24368 n(v0 rc2023-10-16T14:35:40.485504+0000 b24368 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54210100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e15:[<0x10000000729/article-59.xml v1037>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e15:[<0x10000000729/article-59.xml v1037>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.926 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.665+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a0b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg auth v1019 ap=1 s=13289 n(v0 rc2023-10-16T14:35:37.111562+0000 b13289 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54210680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a0b:[<0x10000000729/picture-13.jpg v1019>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a0b:[<0x10000000729/picture-13.jpg v1019>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.927 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.668+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009cf [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg auth v1009 ap=1 s=35317 n(v0 rc2023-10-16T14:35:36.967564+0000 b35317 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54211180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009cf:[<0x10000000729/picture-18.jpg v1009>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x100000009cf:[<0x10000000729/picture-18.jpg v1009>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.927 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.670+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000754 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml auth v999 ap=1 s=24995 n(v0 rc2023-10-16T14:35:34.435608+0000 b24995 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54211700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000754:[<0x10000000729/article-74.xml v999>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000754:[<0x10000000729/article-74.xml v999>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.927 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.671+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000087a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml auth v981 ap=1 s=8665 n(v0 rc2023-10-16T14:35:35.201595+0000 b8665 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54200000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000087a:[<0x10000000729/article-18.xml v981>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000087a:[<0x10000000729/article-18.xml v981>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.927 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.671+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009c3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg auth v1013 ap=1 s=18360 n(v0 rc2023-10-16T14:35:36.791567+0000 b18360 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54200b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009c3:[<0x10000000729/picture-19.jpg v1013>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x100000009c3:[<0x10000000729/picture-19.jpg v1013>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.927 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.671+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ce [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml auth v991 ap=1 s=30328 n(v0 rc2023-10-16T14:35:34.750602+0000 b30328 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54201080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007ce:[<0x10000000729/article-51.xml v991>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007ce:[<0x10000000729/article-51.xml v991>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.928 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.671+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a60 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg auth v1017 ap=1 s=9254 n(v0 rc2023-10-16T14:35:37.358558+0000 b9254 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54202100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a60:[<0x10000000729/picture-4.jpg v1017>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a60:[<0x10000000729/picture-4.jpg v1017>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.928 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.672+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000084c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml auth v983 ap=1 s=28871 n(v0 rc2023-10-16T14:35:35.070597+0000 b28871 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54201b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000084c:[<0x10000000729/article-27.xml v983>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000084c:[<0x10000000729/article-27.xml v983>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.928 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.672+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a67 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg auth v1021 ap=1 s=11580 n(v0 rc2023-10-16T14:35:37.376557+0000 b11580 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54202c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a67:[<0x10000000729/picture-2.jpg v1021>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a67:[<0x10000000729/picture-2.jpg v1021>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.928 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.672+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e42 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml auth v1035 ap=1 s=17358 n(v0 rc2023-10-16T14:35:40.600502+0000 b17358 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54203700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e42:[<0x10000000729/article-20.xml v1035>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e42:[<0x10000000729/article-20.xml v1035>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.929 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.673+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a00 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg auth v1011 ap=1 s=11411 n(v0 rc2023-10-16T14:35:37.081562+0000 b11411 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a00:[<0x10000000729/picture-14.jpg v1011>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a00:[<0x10000000729/picture-14.jpg v1011>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.929 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.673+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000768 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml auth v997 ap=1 s=17345 n(v0 rc2023-10-16T14:35:34.478607+0000 b17345 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54203180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000768:[<0x10000000729/article-70.xml v997>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000768:[<0x10000000729/article-70.xml v997>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.929 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.674+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000796 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml auth v993 ap=1 s=18960 n(v0 rc2023-10-16T14:35:34.593605+0000 b18960 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c0580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000796:[<0x10000000729/article-61.xml v993>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000796:[<0x10000000729/article-61.xml v993>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.929 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.674+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml auth v979 ap=1 s=1672 n(v0 rc2023-10-16T14:35:34.841601+0000 b1672 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007f3:[<0x10000000729/article-45.xml v979>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007f3:[<0x10000000729/article-45.xml v979>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.929 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.675+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000807 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml auth v989 ap=1 s=26885 n(v0 rc2023-10-16T14:35:34.891600+0000 b26885 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c0b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000807:[<0x10000000729/article-41.xml v989>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000807:[<0x10000000729/article-41.xml v989>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.930 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.675+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000076c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml auth v995 ap=1 s=20706 n(v0 rc2023-10-16T14:35:34.491607+0000 b20706 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000076c:[<0x10000000729/article-69.xml v995>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000076c:[<0x10000000729/article-69.xml v995>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.930 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.675+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a47 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg auth v1033 ap=1 s=39783 n(v0 rc2023-10-16T14:35:37.268559+0000 b39783 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a47:[<0x10000000729/picture-7.jpg v1033>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a47:[<0x10000000729/picture-7.jpg v1033>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.930 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.677+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000839 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml auth v985 ap=1 s=16407 n(v0 rc2023-10-16T14:35:35.014598+0000 b16407 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000839:[<0x10000000729/article-31.xml v985>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000839:[<0x10000000729/article-31.xml v985>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.930 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.678+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a2f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg auth v1015 ap=1 s=43146 n(v0 rc2023-10-16T14:35:37.213560+0000 b43146 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a2f:[<0x10000000729/picture-9.jpg v1015>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a2f:[<0x10000000729/picture-9.jpg v1015>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.931 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.679+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000732 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml auth v996 ap=1 s=6137 n(v0 rc2023-10-16T14:35:34.379609+0000 b6137 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce540c3700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000732:[<0x10000000729/article-80.xml v996>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000732:[<0x10000000729/article-80.xml v996>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.931 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.680+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a6b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg auth v1000 ap=1 s=63652 n(v0 rc2023-10-16T14:35:37.388557+0000 b63652 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54309080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a6b:[<0x10000000729/picture-1.jpg v1000>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a6b:[<0x10000000729/picture-1.jpg v1000>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.931 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.680+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008a4 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml auth v980 ap=1 s=13134 n(v0 rc2023-10-16T14:35:35.332592+0000 b13134 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54309b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008a4:[<0x10000000729/article-9.xml v980>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008a4:[<0x10000000729/article-9.xml v980>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.934 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.681+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007f8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml auth v974 ap=1 s=24694 n(v0 rc2023-10-16T14:35:34.854601+0000 b24694 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5430a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007f8:[<0x10000000729/article-44.xml v974>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007f8:[<0x10000000729/article-44.xml v974>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.935 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.681+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a26 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg auth v1006 ap=1 s=34005 n(v0 rc2023-10-16T14:35:37.188561+0000 b34005 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54308b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a26:[<0x10000000729/picture-10.jpg v1006>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a26:[<0x10000000729/picture-10.jpg v1006>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.935 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.681+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg auth v1022 ap=1 s=9727 n(v0 rc2023-10-16T14:35:37.292559+0000 b9727 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5430a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a4f:[<0x10000000729/picture-6.jpg v1022>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a4f:[<0x10000000729/picture-6.jpg v1022>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.935 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.681+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000780 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml auth v994 ap=1 s=21121 n(v0 rc2023-10-16T14:35:34.539606+0000 b21121 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5430ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000780:[<0x10000000729/article-65.xml v994>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000780:[<0x10000000729/article-65.xml v994>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.935 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.681+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000085d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml auth v988 ap=1 s=26448 n(v0 rc2023-10-16T14:35:35.115596+0000 b26448 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000085d:[<0x10000000729/article-24.xml v988>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000085d:[<0x10000000729/article-24.xml v988>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.935 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.685+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000791 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml auth v992 ap=1 s=8215 n(v0 rc2023-10-16T14:35:34.576605+0000 b8215 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5430b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000791:[<0x10000000729/article-62.xml v992>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000791:[<0x10000000729/article-62.xml v992>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.936 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.685+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000970 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg auth v990 ap=1 s=64331 n(v0 rc2023-10-16T14:35:35.946582+0000 b64331 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000970:[<0x10000000729/picture-28.jpg v990>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000970:[<0x10000000729/picture-28.jpg v990>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.936 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.685+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000777 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml auth v984 ap=1 s=29902 n(v0 rc2023-10-16T14:35:34.515606+0000 b29902 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000777:[<0x10000000729/article-67.xml v984>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000777:[<0x10000000729/article-67.xml v984>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.936 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.685+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009da [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg auth v998 ap=1 s=18579 n(v0 rc2023-10-16T14:35:36.987564+0000 b18579 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009da:[<0x10000000729/picture-17.jpg v998>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x100000009da:[<0x10000000729/picture-17.jpg v998>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.936 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.685+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008c7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml auth v976 ap=1 s=14456 n(v0 rc2023-10-16T14:35:35.712586+0000 b14456 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008c7:[<0x10000000729/article-3.xml v976>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008c7:[<0x10000000729/article-3.xml v976>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.686+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000993 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg auth v986 ap=1 s=14104 n(v0 rc2023-10-16T14:35:36.062580+0000 b14104 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce5354f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000993:[<0x10000000729/picture-24.jpg v986>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000993:[<0x10000000729/picture-24.jpg v986>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.686+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008ae [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml auth v978 ap=1 s=24558 n(v0 rc2023-10-16T14:35:35.359592+0000 b24558 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54274580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008ae:[<0x10000000729/article-7.xml v978>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008ae:[<0x10000000729/article-7.xml v978>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.687+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007bb [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml auth v1000 ap=1 s=22182 n(v0 rc2023-10-16T14:35:34.694603+0000 b22182 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54275600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007bb:[<0x10000000729/article-55.xml v1000>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007bb:[<0x10000000729/article-55.xml v1000>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.687+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000098a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg auth v986 ap=1 s=60668 n(v0 rc2023-10-16T14:35:36.036580+0000 b60668 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54275b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000098a:[<0x10000000729/picture-25.jpg v986>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000098a:[<0x10000000729/picture-25.jpg v986>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.689+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008a8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml auth v980 ap=1 s=23556 n(v0 rc2023-10-16T14:35:35.344592+0000 b23556 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54276100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008a8:[<0x10000000729/article-8.xml v980>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008a8:[<0x10000000729/article-8.xml v980>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.937 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.689+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000772 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml auth v1004 ap=1 s=30199 n(v0 rc2023-10-16T14:35:34.502607+0000 b30199 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54277700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000772:[<0x10000000729/article-68.xml v1004>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000772:[<0x10000000729/article-68.xml v1004>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.938 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.690+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000955 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg auth v994 ap=1 s=26866 n(v0 rc2023-10-16T14:35:35.857583+0000 b26866 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54277180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000955:[<0x10000000729/picture-31.jpg v994>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000955:[<0x10000000729/picture-31.jpg v994>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.938 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.690+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000749 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml auth v1006 ap=1 s=20796 n(v0 rc2023-10-16T14:35:34.416608+0000 b20796 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000749:[<0x10000000729/article-76.xml v1006>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000749:[<0x10000000729/article-76.xml v1006>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.938 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.690+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a3b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg auth v1008 ap=1 s=26393 n(v0 rc2023-10-16T14:35:37.238560+0000 b26393 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a3b:[<0x10000000729/picture-8.jpg v1008>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a3b:[<0x10000000729/picture-8.jpg v1008>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.938 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.690+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e8d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg auth v1024 ap=1 s=47574 n(v0 rc2023-10-16T14:35:40.843498+0000 b47574 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e8d:[<0x10000000729/picture-88.jpg v1024>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e8d:[<0x10000000729/picture-88.jpg v1024>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.939 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.690+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000921 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml auth v984 ap=1 s=5167 n(v0 rc2023-10-16T14:35:35.719586+0000 b5167 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e54680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000921:[<0x10000000729/article-21.xml v984>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000921:[<0x10000000729/article-21.xml v984>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.939 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.691+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000982 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg auth v988 ap=1 s=61057 n(v0 rc2023-10-16T14:35:36.005581+0000 b61057 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000982:[<0x10000000729/picture-26.jpg v988>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000982:[<0x10000000729/picture-26.jpg v988>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.939 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.691+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000834 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml auth v996 ap=1 s=32208 n(v0 rc2023-10-16T14:35:35.002598+0000 b32208 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55e53b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000834:[<0x10000000729/article-32.xml v996>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000834:[<0x10000000729/article-32.xml v996>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.939 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.691+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000891 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml auth v982 ap=1 s=27372 n(v0 rc2023-10-16T14:35:35.272593+0000 b27372 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d98000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000891:[<0x10000000729/article-13.xml v982>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000891:[<0x10000000729/article-13.xml v982>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.939 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.693+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000967 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg auth v990 ap=1 s=31939 n(v0 rc2023-10-16T14:35:35.905583+0000 b31939 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d9b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000967:[<0x10000000729/picture-29.jpg v990>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000967:[<0x10000000729/picture-29.jpg v990>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.940 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.693+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000926 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml auth v976 ap=1 s=2863 n(v0 rc2023-10-16T14:35:35.737585+0000 b2863 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d9a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000926:[<0x10000000729/article-2.xml v976>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000926:[<0x10000000729/article-2.xml v976>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.940 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.696+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007fd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml auth v986 ap=1 s=10309 n(v0 rc2023-10-16T14:35:34.867600+0000 b10309 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d39b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007fd:[<0x10000000729/article-43.xml v986>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007fd:[<0x10000000729/article-43.xml v986>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.940 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.697+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000082f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml auth v984 ap=1 s=19860 n(v0 rc2023-10-16T14:35:34.990598+0000 b19860 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000082f:[<0x10000000729/article-33.xml v984>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000082f:[<0x10000000729/article-33.xml v984>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.940 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.697+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001324 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg auth v1112 ap=1 s=31495 n(v0 rc2023-10-16T14:35:45.878411+0000 b31495 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d3a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001324:[<0x10000000729/picture-81.jpg v1112>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2141>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000001324:[<0x10000000729/picture-81.jpg v1112>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.940 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.697+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a57 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg auth v1016 ap=1 s=10253 n(v0 rc2023-10-16T14:35:37.337558+0000 b10253 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d3ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a57:[<0x10000000729/picture-5.jpg v1016>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a57:[<0x10000000729/picture-5.jpg v1016>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.941 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.698+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007b0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml auth v992 ap=1 s=9018 n(v0 rc2023-10-16T14:35:34.665604+0000 b9018 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d06000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007b0:[<0x10000000729/article-57.xml v992>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007b0:[<0x10000000729/article-57.xml v992>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.941 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.701+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000737 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml auth v978 ap=1 s=2721 n(v0 rc2023-10-16T14:35:34.389609+0000 b2721 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d07600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000737:[<0x10000000729/article-79.xml v978>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000737:[<0x10000000729/article-79.xml v978>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.941 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.702+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007aa [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml auth v980 ap=1 s=10203 n(v0 rc2023-10-16T14:35:34.649604+0000 b10203 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d08100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007aa:[<0x10000000729/article-58.xml v980>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007aa:[<0x10000000729/article-58.xml v980>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.941 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.702+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ca [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml auth v988 ap=1 s=31160 n(v0 rc2023-10-16T14:35:34.739603+0000 b31160 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d0e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007ca:[<0x10000000729/article-52.xml v988>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007ca:[<0x10000000729/article-52.xml v988>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.941 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.702+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000e3c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg auth v1036 ap=1 s=56797 n(v0 rc2023-10-16T14:35:40.579502+0000 b56797 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d0eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000e3c:[<0x10000000729/picture-51.jpg v1036>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000e3c:[<0x10000000729/picture-51.jpg v1036>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.942 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.703+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a65 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg auth v998 ap=1 s=16544 n(v0 rc2023-10-16T14:35:37.368557+0000 b16544 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d0fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a65:[<0x10000000729/picture-3.jpg v998>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a65:[<0x10000000729/picture-3.jpg v998>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.942 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.703+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000848 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml auth v982 ap=1 s=4584 n(v0 rc2023-10-16T14:35:35.058597+0000 b4584 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d09700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000848:[<0x10000000729/article-28.xml v982>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000848:[<0x10000000729/article-28.xml v982>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.942 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.704+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a20 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg auth v996 ap=1 s=37201 n(v0 rc2023-10-16T14:35:37.163561+0000 b37201 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d10c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a20:[<0x10000000729/picture-11.jpg v996>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a20:[<0x10000000729/picture-11.jpg v996>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.942 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.704+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000082a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml auth v973 ap=1 s=4990 n(v0 rc2023-10-16T14:35:34.977599+0000 b4990 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d11700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000082a:[<0x10000000729/article-34.xml v973>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000082a:[<0x10000000729/article-34.xml v973>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.943 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.704+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008b8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml auth v969 ap=1 s=13391 n(v0 rc2023-10-16T14:35:35.388591+0000 b13391 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55d11180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008b8:[<0x10000000729/article-5.xml v969>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008b8:[<0x10000000729/article-5.xml v969>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.943 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.705+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000072c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml auth v977 ap=1 s=23603 n(v0 rc2023-10-16T14:35:34.371609+0000 b23603 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55cbb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000072c:[<0x10000000729/article-81.xml v977>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000072c:[<0x10000000729/article-81.xml v977>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.943 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.705+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000079c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml auth v965 ap=1 s=1852 n(v0 rc2023-10-16T14:35:34.609605+0000 b1852 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55cbc100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000079c:[<0x10000000729/article-60.xml v965>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000079c:[<0x10000000729/article-60.xml v965>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.943 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.705+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000843 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml auth v971 ap=1 s=12814 n(v0 rc2023-10-16T14:35:35.042597+0000 b12814 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55cbbb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000843:[<0x10000000729/article-29.xml v971>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000843:[<0x10000000729/article-29.xml v971>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.943 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.705+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000132b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml auth v1099 ap=1 s=2601 n(v0 rc2023-10-16T14:35:45.905411+0000 b2601 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55cbb600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000132b:[<0x10000000729/article-78.xml v1099>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2141>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x1000000132b:[<0x10000000729/article-78.xml v1099>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.944 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.705+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000867 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml auth v967 ap=1 s=13687 n(v0 rc2023-10-16T14:35:35.142596+0000 b13687 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55cbcc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000867:[<0x10000000729/article-22.xml v967>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000867:[<0x10000000729/article-22.xml v967>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.944 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.706+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000099c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg auth v990 ap=1 s=28136 n(v0 rc2023-10-16T14:35:36.092579+0000 b28136 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c70b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000099c:[<0x10000000729/picture-23.jpg v990>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000099c:[<0x10000000729/picture-23.jpg v990>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.944 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.706+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007dd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml auth v975 ap=1 s=21176 n(v0 rc2023-10-16T14:35:34.792602+0000 b21176 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c70000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007dd:[<0x10000000729/article-49.xml v975>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007dd:[<0x10000000729/article-49.xml v975>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.944 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.707+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000088e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml auth v984 ap=1 s=24929 n(v0 rc2023-10-16T14:35:35.264594+0000 b24929 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c71080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000088e:[<0x10000000729/article-14.xml v984>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000088e:[<0x10000000729/article-14.xml v984>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.944 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.708+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007ee [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml auth v978 ap=1 s=16683 n(v0 rc2023-10-16T14:35:34.829601+0000 b16683 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007ee:[<0x10000000729/article-46.xml v978>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007ee:[<0x10000000729/article-46.xml v978>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.945 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.708+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml auth v998 ap=1 s=24724 n(v0 rc2023-10-16T14:35:34.817601+0000 b24724 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c72680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007e9:[<0x10000000729/article-47.xml v998>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007e9:[<0x10000000729/article-47.xml v998>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.945 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.708+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000089a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml auth v982 ap=1 s=17974 n(v0 rc2023-10-16T14:35:35.301593+0000 b17974 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c73700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000089a:[<0x10000000729/article-11.xml v982>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000089a:[<0x10000000729/article-11.xml v982>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.945 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.708+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000763 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml auth v1004 ap=1 s=20810 n(v0 rc2023-10-16T14:35:34.469607+0000 b20810 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c72c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000763:[<0x10000000729/article-71.xml v1004>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000763:[<0x10000000729/article-71.xml v1004>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.945 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.709+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009a5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg auth v988 ap=1 s=28940 n(v0 rc2023-10-16T14:35:36.123579+0000 b28940 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c73180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009a5:[<0x10000000729/picture-22.jpg v988>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000009a5:[<0x10000000729/picture-22.jpg v988>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.945 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.709+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000884 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml auth v986 ap=1 s=8720 n(v0 rc2023-10-16T14:35:35.232594+0000 b8720 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c26b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000884:[<0x10000000729/article-16.xml v986>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000884:[<0x10000000729/article-16.xml v986>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.946 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.710+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008bf [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg auth v1006 ap=1 s=11372 n(v0 rc2023-10-16T14:35:35.425591+0000 b11372 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c26580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008bf:[<0x10000000729/picture-97.jpg v1006>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008bf:[<0x10000000729/picture-97.jpg v1006>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.946 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.713+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007c1 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml auth v1002 ap=1 s=29916 n(v0 rc2023-10-16T14:35:34.713603+0000 b29916 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c27600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007c1:[<0x10000000729/article-54.xml v1002>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007c1:[<0x10000000729/article-54.xml v1002>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.946 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.714+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000851 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml auth v994 ap=1 s=11826 n(v0 rc2023-10-16T14:35:35.084597+0000 b11826 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000851:[<0x10000000729/article-26.xml v994>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000851:[<0x10000000729/article-26.xml v994>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.946 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.714+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009f1 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg auth v1012 ap=1 s=63560 n(v0 rc2023-10-16T14:35:37.042563+0000 b63560 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c28100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009f1:[<0x10000000729/picture-15.jpg v1012>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x100000009f1:[<0x10000000729/picture-15.jpg v1012>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.946 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.714+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000f21 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml auth v1026 ap=1 s=5650 n(v0 rc2023-10-16T14:35:41.349489+0000 b5650 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c28680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000f21:[<0x10000000729/article-10.xml v1026>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2123>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000f21:[<0x10000000729/article-10.xml v1026>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.947 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.716+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a15 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg auth v1016 ap=1 s=40793 n(v0 rc2023-10-16T14:35:37.145561+0000 b40793 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c28c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000a15:[<0x10000000729/picture-12.jpg v1016>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000a15:[<0x10000000729/picture-12.jpg v1016>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.947 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.716+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000007e4 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml auth v1000 ap=1 s=26793 n(v0 rc2023-10-16T14:35:34.804602+0000 b26793 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c29700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000007e4:[<0x10000000729/article-48.xml v1000>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000007e4:[<0x10000000729/article-48.xml v1000>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.947 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.717+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000008bd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml auth v980 ap=1 s=4586 n(v0 rc2023-10-16T14:35:35.418591+0000 b4586 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c2e580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000008bd:[<0x10000000729/article-4.xml v980>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000008bd:[<0x10000000729/article-4.xml v980>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.947 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.717+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000816 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml auth v996 ap=1 s=3577 n(v0 rc2023-10-16T14:35:34.929599+0000 b3577 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c2f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000816:[<0x10000000729/article-38.xml v996>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000816:[<0x10000000729/article-38.xml v996>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.947 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.718+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000934 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml auth v992 ap=1 s=10910 n(v0 rc2023-10-16T14:35:35.769585+0000 b10910 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c2eb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000934:[<0x10000000729/article-25.xml v992>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000934:[<0x10000000729/article-25.xml v992>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.948 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.718+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000075e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml auth v990 ap=1 s=8968 n(v0 rc2023-10-16T14:35:34.457607+0000 b8968 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c2fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000075e:[<0x10000000729/article-72.xml v990>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000075e:[<0x10000000729/article-72.xml v990>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.948 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.719+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009af [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg auth v980 ap=1 s=59210 n(v0 rc2023-10-16T14:35:36.148578+0000 b59210 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c30100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009af:[<0x10000000729/picture-21.jpg v980>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000009af:[<0x10000000729/picture-21.jpg v980>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.948 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.719+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000825 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml auth v984 ap=1 s=2466 n(v0 rc2023-10-16T14:35:34.965599+0000 b2466 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55c30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000825:[<0x10000000729/article-35.xml v984>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000825:[<0x10000000729/article-35.xml v984>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.948 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.722+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000080c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml auth v972 ap=1 s=26008 n(v0 rc2023-10-16T14:35:34.903600+0000 b26008 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bf6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000080c:[<0x10000000729/article-40.xml v972>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000080c:[<0x10000000729/article-40.xml v972>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.948 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.722+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000095e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg auth v982 ap=1 s=51928 n(v0 rc2023-10-16T14:35:35.881583+0000 b51928 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bf6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000095e:[<0x10000000729/picture-30.jpg v982>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000095e:[<0x10000000729/picture-30.jpg v982>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.949 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.723+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000092a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg auth v986 ap=1 s=47388 n(v0 rc2023-10-16T14:35:35.749585+0000 b47388 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bf8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000092a:[<0x10000000729/picture-36.jpg v986>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000092a:[<0x10000000729/picture-36.jpg v986>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.949 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.723+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009b9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg auth v978 ap=1 s=29694 n(v0 rc2023-10-16T14:35:36.188578+0000 b29694 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bf8c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009b9:[<0x10000000729/picture-20.jpg v978>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x100000009b9:[<0x10000000729/picture-20.jpg v978>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.949 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.723+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000786 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml auth v979 ap=1 s=26850 n(v0 rc2023-10-16T14:35:34.550606+0000 b26850 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb2b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000786:[<0x10000000729/article-64.xml v979>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000786:[<0x10000000729/article-64.xml v979>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.949 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.724+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000802 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml auth v988 ap=1 s=10180 n(v0 rc2023-10-16T14:35:34.879600+0000 b10180 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb2580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000802:[<0x10000000729/article-42.xml v988>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000802:[<0x10000000729/article-42.xml v988>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.950 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.724+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000074d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml auth v1007 ap=1 s=25796 n(v0 rc2023-10-16T14:35:34.426608+0000 b25796 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000074d:[<0x10000000729/article-75.xml v1007>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000074d:[<0x10000000729/article-75.xml v1007>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.950 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.724+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000979 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg auth v993 ap=1 s=30918 n(v0 rc2023-10-16T14:35:35.979581+0000 b30918 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb5180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000979:[<0x10000000729/picture-27.jpg v993>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000979:[<0x10000000729/picture-27.jpg v993>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.950 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.725+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000757 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml auth v1005 ap=1 s=24682 n(v0 rc2023-10-16T14:35:34.447608+0000 b24682 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb5700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000757:[<0x10000000729/article-73.xml v1005>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000757:[<0x10000000729/article-73.xml v1005>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.950 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.725+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000811 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml auth v981 ap=1 s=19599 n(v0 rc2023-10-16T14:35:34.916600+0000 b19599 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b84000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000811:[<0x10000000729/article-39.xml v981>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000811:[<0x10000000729/article-39.xml v981>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.950 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.725+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000081b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml auth v1001 ap=1 s=17800 n(v0 rc2023-10-16T14:35:34.941599+0000 b17800 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55bb4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000081b:[<0x10000000729/article-37.xml v1001>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000081b:[<0x10000000729/article-37.xml v1001>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.951 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.725+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000896 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml auth v985 ap=1 s=22189 n(v0 rc2023-10-16T14:35:35.286593+0000 b22189 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b84b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000896:[<0x10000000729/article-12.xml v985>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000896:[<0x10000000729/article-12.xml v985>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.951 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.725+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000940 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg auth v997 ap=1 s=50885 n(v0 rc2023-10-16T14:35:35.797584+0000 b50885 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b85b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000940:[<0x10000000729/picture-34.jpg v997>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000940:[<0x10000000729/picture-34.jpg v997>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.951 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.726+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000820 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml auth v999 ap=1 s=23103 n(v0 rc2023-10-16T14:35:34.954599+0000 b23103 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b84580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000820:[<0x10000000729/article-36.xml v999>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000820:[<0x10000000729/article-36.xml v999>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.951 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.726+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000092f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml auth v983 ap=1 s=11022 n(v0 rc2023-10-16T14:35:35.762585+0000 b11022 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b85080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000092f:[<0x10000000729/article-1.xml v983>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000092f:[<0x10000000729/article-1.xml v983>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.951 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.726+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000077c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml auth v1003 ap=1 s=25915 n(v0 rc2023-10-16T14:35:34.526606+0000 b25915 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b86c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000077c:[<0x10000000729/article-66.xml v1003>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000077c:[<0x10000000729/article-66.xml v1003>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.952 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.726+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000742 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml auth v977 ap=1 s=25110 n(v0 rc2023-10-16T14:35:34.408608+0000 b25110 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000742:[<0x10000000729/article-77.xml v977>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000742:[<0x10000000729/article-77.xml v977>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.952 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.726+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009e5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg auth v1015 ap=1 s=48573 n(v0 rc2023-10-16T14:35:37.015564+0000 b48573 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b87180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x100000009e5:[<0x10000000729/picture-16.jpg v1015>,<0x10000000223/blog-10 v176302>,<0x200000001fe/blogtest_in v2106>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x100000009e5:[<0x10000000729/picture-16.jpg v1015>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.952 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.727+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000fc9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml auth v1037 ap=1 s=4613 n(v0 rc2023-10-16T14:35:43.081459+0000 b4613 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b1d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000fc9:[<0x10000000729/article-98.xml v1037>,<0x10000000223/blog-10 v177949>,<0x200000001fe/blogtest_in v2141>,<0x200000001f7/src v1715>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v86>,<0x10000000000/_nogroup v71>,<0x1/volumes v79>]//[3]","memoryvalue":"(4)0x10000000fc9:[<0x10000000729/article-98.xml v1037>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.952 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.727+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000087e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml auth v989 ap=1 s=20011 n(v0 rc2023-10-16T14:35:35.217594+0000 b20011 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b1e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000087e:[<0x10000000729/article-17.xml v989>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000087e:[<0x10000000729/article-17.xml v989>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.952 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.727+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000094c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg auth v995 ap=1 s=29969 n(v0 rc2023-10-16T14:35:35.832584+0000 b29969 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b1e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000094c:[<0x10000000729/picture-32.jpg v995>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000094c:[<0x10000000729/picture-32.jpg v995>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.953 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.728+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000078a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml auth v987 ap=1 s=6127 n(v0 rc2023-10-16T14:35:34.563606+0000 b6127 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b1ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000078a:[<0x10000000729/article-63.xml v987>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x1000000078a:[<0x10000000729/article-63.xml v987>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:05.953 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:05.728+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000862 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml auth v991 ap=1 s=23698 n(v0 rc2023-10-16T14:35:35.129596+0000 b23698 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce55b1db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000862:[<0x10000000729/article-23.xml v991>,<0x10000000223/blog-10 v173430>,<0x200000001fe/blogtest_in v2104>,<0x200000001f7/src v1713>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v84>,<0x10000000000/_nogroup v68>,<0x1/volumes v77>]//[3]","memoryvalue":"(4)0x10000000862:[<0x10000000729/article-23.xml v991>,<0x10000000223/blog-10 v195659>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:05 smithi178 ceph-mon[98495]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-16T14:40:06.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:05 smithi186 ceph-mon[134892]: pgmap v530: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 473 op/s 2023-10-16T14:40:06.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.368+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d01 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml auth v987 ap=1 s=31956 n(v0 rc2023-10-16T14:35:56.767224+0000 b31956 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce5320d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d01:[<0x10000001cf4/article-59.xml v987>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d01:[<0x10000001cf4/article-59.xml v987>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.370+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e86 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg auth v965 ap=1 s=34962 n(v0 rc2023-10-16T14:35:58.145201+0000 b34962 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53216000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e86:[<0x10000001cf4/picture-13.jpg v965>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e86:[<0x10000001cf4/picture-13.jpg v965>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.677 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.372+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e77 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg auth v979 ap=1 s=56188 n(v0 rc2023-10-16T14:35:58.106201+0000 b56188 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53216580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e77:[<0x10000001cf4/picture-18.jpg v979>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e77:[<0x10000001cf4/picture-18.jpg v979>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.678 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.373+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d65 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml auth v993 ap=1 s=24149 n(v0 rc2023-10-16T14:35:57.002220+0000 b24149 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53216b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d65:[<0x10000001cf4/article-18.xml v993>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d65:[<0x10000001cf4/article-18.xml v993>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.678 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.376+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e74 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg auth v995 ap=1 s=48597 n(v0 rc2023-10-16T14:35:58.099202+0000 b48597 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53217600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e74:[<0x10000001cf4/picture-19.jpg v995>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e74:[<0x10000001cf4/picture-19.jpg v995>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.678 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.380+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001db7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg auth v999 ap=1 s=33779 n(v0 rc2023-10-16T14:35:57.334215+0000 b33779 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53218680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001db7:[<0x10000001cf4/picture-82.jpg v999>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001db7:[<0x10000001cf4/picture-82.jpg v999>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.678 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.381+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d46 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml auth v989 ap=1 s=14608 n(v0 rc2023-10-16T14:35:56.932222+0000 b14608 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53218c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d46:[<0x10000001cf4/article-27.xml v989>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d46:[<0x10000001cf4/article-27.xml v989>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.679 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.381+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ea0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg auth v997 ap=1 s=9310 n(v0 rc2023-10-16T14:35:58.204200+0000 b9310 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53219180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ea0:[<0x10000001cf4/picture-4.jpg v997>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ea0:[<0x10000001cf4/picture-4.jpg v997>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.679 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.384+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e44 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg auth v977 ap=1 s=23344 n(v0 rc2023-10-16T14:35:57.996203+0000 b23344 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53219700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e44:[<0x10000001cf4/picture-35.jpg v977>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e44:[<0x10000001cf4/picture-35.jpg v977>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.679 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.386+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ea6 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg auth v1003 ap=1 s=12237 n(v0 rc2023-10-16T14:35:58.215200+0000 b12237 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ea6:[<0x10000001cf4/picture-2.jpg v1003>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ea6:[<0x10000001cf4/picture-2.jpg v1003>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.679 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.387+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dc3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg auth v1001 ap=1 s=38484 n(v0 rc2023-10-16T14:35:57.411213+0000 b38484 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dc3:[<0x10000001cf4/picture-78.jpg v1001>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dc3:[<0x10000001cf4/picture-78.jpg v1001>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.680 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.388+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d5d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml auth v991 ap=1 s=26354 n(v0 rc2023-10-16T14:35:56.987221+0000 b26354 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d5d:[<0x10000001cf4/article-20.xml v991>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d5d:[<0x10000001cf4/article-20.xml v991>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.680 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.391+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cfd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml auth v985 ap=1 s=5259 n(v0 rc2023-10-16T14:35:56.756225+0000 b5259 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cfd:[<0x10000001cf4/article-61.xml v985>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cfd:[<0x10000001cf4/article-61.xml v985>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.680 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.392+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d25 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml auth v967 ap=1 s=30498 n(v0 rc2023-10-16T14:35:56.856223+0000 b30498 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d25:[<0x10000001cf4/article-41.xml v967>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d25:[<0x10000001cf4/article-41.xml v967>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.680 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.393+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d1d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml auth v975 ap=1 s=24295 n(v0 rc2023-10-16T14:35:56.839223+0000 b24295 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d1d:[<0x10000001cf4/article-45.xml v975>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d1d:[<0x10000001cf4/article-45.xml v975>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.681 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.394+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg auth v969 ap=1 s=39698 n(v0 rc2023-10-16T14:35:57.796207+0000 b39698 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001df9:[<0x10000001cf4/picture-61.jpg v969>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001df9:[<0x10000001cf4/picture-61.jpg v969>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.681 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.396+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e97 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg auth v973 ap=1 s=11561 n(v0 rc2023-10-16T14:35:58.186200+0000 b11561 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e97:[<0x10000001cf4/picture-7.jpg v973>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e97:[<0x10000001cf4/picture-7.jpg v973>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.681 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.397+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d3a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml auth v981 ap=1 s=9353 n(v0 rc2023-10-16T14:35:56.906222+0000 b9353 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d3a:[<0x10000001cf4/article-31.xml v981>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d3a:[<0x10000001cf4/article-31.xml v981>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.681 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.398+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e92 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg auth v971 ap=1 s=20896 n(v0 rc2023-10-16T14:35:58.174200+0000 b20896 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f0f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e92:[<0x10000001cf4/picture-9.jpg v971>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e92:[<0x10000001cf4/picture-9.jpg v971>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.681 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.400+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg auth v986 ap=1 s=56938 n(v0 rc2023-10-16T14:35:57.759207+0000 b56938 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53988b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001df3:[<0x10000001cf4/picture-64.jpg v986>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001df3:[<0x10000001cf4/picture-64.jpg v986>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.682 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.401+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e90 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg auth v960 ap=1 s=18185 n(v0 rc2023-10-16T14:35:58.167200+0000 b18185 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53989600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e90:[<0x10000001cf4/picture-10.jpg v960>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e90:[<0x10000001cf4/picture-10.jpg v960>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.682 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.402+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ea9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg auth v968 ap=1 s=53520 n(v0 rc2023-10-16T14:35:58.221199+0000 b53520 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53989b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ea9:[<0x10000001cf4/picture-1.jpg v968>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ea9:[<0x10000001cf4/picture-1.jpg v968>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.682 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.405+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d1f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml auth v962 ap=1 s=21464 n(v0 rc2023-10-16T14:35:56.843223+0000 b21464 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce5398b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d1f:[<0x10000001cf4/article-44.xml v962>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d1f:[<0x10000001cf4/article-44.xml v962>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.682 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.406+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d80 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml auth v992 ap=1 s=16834 n(v0 rc2023-10-16T14:35:57.070219+0000 b16834 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce5398a680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d80:[<0x10000001cf4/article-9.xml v992>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d80:[<0x10000001cf4/article-9.xml v992>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.682 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.407+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001db3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg auth v996 ap=1 s=39700 n(v0 rc2023-10-16T14:35:57.315215+0000 b39700 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce5398ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001db3:[<0x10000001cf4/picture-83.jpg v996>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001db3:[<0x10000001cf4/picture-83.jpg v996>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.683 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.409+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e9a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg auth v1002 ap=1 s=27990 n(v0 rc2023-10-16T14:35:58.191200+0000 b27990 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce5398b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e9a:[<0x10000001cf4/picture-6.jpg v1002>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e9a:[<0x10000001cf4/picture-6.jpg v1002>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.683 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.410+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml auth v990 ap=1 s=25896 n(v0 rc2023-10-16T14:35:56.735225+0000 b25896 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53992000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cf5:[<0x10000001cf4/article-65.xml v990>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cf5:[<0x10000001cf4/article-65.xml v990>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.683 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.411+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cfb [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml auth v970 ap=1 s=5225 n(v0 rc2023-10-16T14:35:56.751225+0000 b5225 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53992580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cfb:[<0x10000001cf4/article-62.xml v970>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cfb:[<0x10000001cf4/article-62.xml v970>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.683 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.411+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d4f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml auth v974 ap=1 s=5444 n(v0 rc2023-10-16T14:35:56.962221+0000 b5444 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53992b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d4f:[<0x10000001cf4/article-24.xml v974>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d4f:[<0x10000001cf4/article-24.xml v974>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.683 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.416+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e59 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg auth v988 ap=1 s=15783 n(v0 rc2023-10-16T14:35:58.028203+0000 b15783 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53993b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e59:[<0x10000001cf4/picture-28.jpg v988>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e59:[<0x10000001cf4/picture-28.jpg v988>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.684 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.416+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e3e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg auth v1000 ap=1 s=57436 n(v0 rc2023-10-16T14:35:57.981204+0000 b57436 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53994680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e3e:[<0x10000001cf4/picture-37.jpg v1000>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e3e:[<0x10000001cf4/picture-37.jpg v1000>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.684 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.416+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e7a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg auth v978 ap=1 s=32734 n(v0 rc2023-10-16T14:35:58.114201+0000 b32734 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53994c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e7a:[<0x10000001cf4/picture-17.jpg v978>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e7a:[<0x10000001cf4/picture-17.jpg v978>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.684 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.417+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001da1 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg auth v994 ap=1 s=19568 n(v0 rc2023-10-16T14:35:57.247216+0000 b19568 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53995180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001da1:[<0x10000001cf4/picture-85.jpg v994>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001da1:[<0x10000001cf4/picture-85.jpg v994>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.684 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.417+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e65 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg auth v966 ap=1 s=10599 n(v0 rc2023-10-16T14:35:58.060202+0000 b10599 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f6000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e65:[<0x10000001cf4/picture-24.jpg v966>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e65:[<0x10000001cf4/picture-24.jpg v966>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.685 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.419+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e4a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg auth v972 ap=1 s=40366 n(v0 rc2023-10-16T14:35:58.007203+0000 b40366 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f6580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e4a:[<0x10000001cf4/picture-33.jpg v972>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e4a:[<0x10000001cf4/picture-33.jpg v972>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.685 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.421+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d86 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml auth v980 ap=1 s=22894 n(v0 rc2023-10-16T14:35:57.088219+0000 b22894 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f7080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d86:[<0x10000001cf4/article-7.xml v980>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d86:[<0x10000001cf4/article-7.xml v980>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.685 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.422+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e03 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg auth v984 ap=1 s=42738 n(v0 rc2023-10-16T14:35:57.855206+0000 b42738 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e03:[<0x10000001cf4/picture-57.jpg v984>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e03:[<0x10000001cf4/picture-57.jpg v984>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.685 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.423+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e2f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg auth v998 ap=1 s=12616 n(v0 rc2023-10-16T14:35:57.955204+0000 b12616 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e2f:[<0x10000001cf4/picture-42.jpg v998>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e2f:[<0x10000001cf4/picture-42.jpg v998>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.685 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.423+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e38 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg auth v964 ap=1 s=47779 n(v0 rc2023-10-16T14:35:57.969204+0000 b47779 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e38:[<0x10000001cf4/picture-39.jpg v964>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e38:[<0x10000001cf4/picture-39.jpg v964>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.686 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.423+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001de5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg auth v976 ap=1 s=40961 n(v0 rc2023-10-16T14:35:57.662209+0000 b40961 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f7b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001de5:[<0x10000001cf4/picture-68.jpg v976>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001de5:[<0x10000001cf4/picture-68.jpg v976>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.686 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.427+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e62 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg auth v988 ap=1 s=12866 n(v0 rc2023-10-16T14:35:58.051202+0000 b12866 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce543f9700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e62:[<0x10000001cf4/picture-25.jpg v988>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e62:[<0x10000001cf4/picture-25.jpg v988>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.686 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.427+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d83 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml auth v958 ap=1 s=22698 n(v0 rc2023-10-16T14:35:57.079219+0000 b22698 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d2000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d83:[<0x10000001cf4/article-8.xml v958>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d83:[<0x10000001cf4/article-8.xml v958>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.686 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.430+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e50 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg auth v980 ap=1 s=13963 n(v0 rc2023-10-16T14:35:58.015203+0000 b13963 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d3080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e50:[<0x10000001cf4/picture-31.jpg v980>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e50:[<0x10000001cf4/picture-31.jpg v980>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.686 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.431+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e95 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg auth v976 ap=1 s=48322 n(v0 rc2023-10-16T14:35:58.181200+0000 b48322 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e95:[<0x10000001cf4/picture-8.jpg v976>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e95:[<0x10000001cf4/picture-8.jpg v976>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.687 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.432+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e5f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg auth v966 ap=1 s=36414 n(v0 rc2023-10-16T14:35:58.043202+0000 b36414 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e5f:[<0x10000001cf4/picture-26.jpg v966>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e5f:[<0x10000001cf4/picture-26.jpg v966>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.687 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.433+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e19 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg auth v974 ap=1 s=61903 n(v0 rc2023-10-16T14:35:57.921205+0000 b61903 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d3600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e19:[<0x10000001cf4/picture-49.jpg v974>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e19:[<0x10000001cf4/picture-49.jpg v974>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.687 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.436+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d37 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml auth v970 ap=1 s=24479 n(v0 rc2023-10-16T14:35:56.900222+0000 b24479 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d4680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d37:[<0x10000001cf4/article-32.xml v970>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d37:[<0x10000001cf4/article-32.xml v970>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.687 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.438+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d58 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml auth v962 ap=1 s=2782 n(v0 rc2023-10-16T14:35:56.977221+0000 b2782 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce4f5d4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d58:[<0x10000001cf4/article-21.xml v962>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d58:[<0x10000001cf4/article-21.xml v962>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.687 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.440+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d74 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml auth v990 ap=1 s=28253 n(v0 rc2023-10-16T14:35:57.042220+0000 b28253 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a70000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d74:[<0x10000001cf4/article-13.xml v990>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d74:[<0x10000001cf4/article-13.xml v990>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.688 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.444+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d89 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml auth v982 ap=1 s=20972 n(v0 rc2023-10-16T14:35:57.094219+0000 b20972 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a71080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d89:[<0x10000001cf4/article-6.xml v982>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d89:[<0x10000001cf4/article-6.xml v982>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.688 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.445+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d95 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml auth v984 ap=1 s=23322 n(v0 rc2023-10-16T14:35:57.133218+0000 b23322 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a71600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d95:[<0x10000001cf4/article-2.xml v984>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d95:[<0x10000001cf4/article-2.xml v984>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.688 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.446+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e0a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg auth v972 ap=1 s=13639 n(v0 rc2023-10-16T14:35:57.880205+0000 b13639 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a72100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e0a:[<0x10000001cf4/picture-54.jpg v972>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e0a:[<0x10000001cf4/picture-54.jpg v972>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.688 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.446+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e56 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg auth v964 ap=1 s=51307 n(v0 rc2023-10-16T14:35:58.023203+0000 b51307 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a72680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e56:[<0x10000001cf4/picture-29.jpg v964>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e56:[<0x10000001cf4/picture-29.jpg v964>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.688 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.446+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d07 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml auth v968 ap=1 s=14802 n(v0 rc2023-10-16T14:35:56.783224+0000 b14802 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a73180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d07:[<0x10000001cf4/article-56.xml v968>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d07:[<0x10000001cf4/article-56.xml v968>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.689 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.448+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d9b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg auth v974 ap=1 s=48453 n(v0 rc2023-10-16T14:35:57.144218+0000 b48453 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a76580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d9b:[<0x10000001cf4/picture-86.jpg v974>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d9b:[<0x10000001cf4/picture-86.jpg v974>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.689 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.454+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dba [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg auth v994 ap=1 s=53351 n(v0 rc2023-10-16T14:35:57.348214+0000 b53351 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a77b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dba:[<0x10000001cf4/picture-81.jpg v994>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dba:[<0x10000001cf4/picture-81.jpg v994>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.692 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.456+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e16 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg auth v984 ap=1 s=8714 n(v0 rc2023-10-16T14:35:57.916205+0000 b8714 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a78c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e16:[<0x10000001cf4/picture-50.jpg v984>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e16:[<0x10000001cf4/picture-50.jpg v984>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.692 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.459+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d05 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml auth v964 ap=1 s=1327 n(v0 rc2023-10-16T14:35:56.777224+0000 b1327 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a79700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d05:[<0x10000001cf4/article-57.xml v964>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d05:[<0x10000001cf4/article-57.xml v964>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.692 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.461+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d03 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml auth v962 ap=1 s=12116 n(v0 rc2023-10-16T14:35:56.772224+0000 b12116 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53857600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d03:[<0x10000001cf4/article-58.xml v962>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d03:[<0x10000001cf4/article-58.xml v962>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.692 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.464+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d21 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml auth v992 ap=1 s=1261 n(v0 rc2023-10-16T14:35:56.848223+0000 b1261 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a77080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d21:[<0x10000001cf4/article-43.xml v992>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d21:[<0x10000001cf4/article-43.xml v992>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.693 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.466+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d0d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml auth v978 ap=1 s=3056 n(v0 rc2023-10-16T14:35:56.798224+0000 b3056 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53857b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d0d:[<0x10000001cf4/article-53.xml v978>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d0d:[<0x10000001cf4/article-53.xml v978>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.693 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.468+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001def [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg auth v968 ap=1 s=9248 n(v0 rc2023-10-16T14:35:57.741208+0000 b9248 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53856580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001def:[<0x10000001cf4/picture-65.jpg v968>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001def:[<0x10000001cf4/picture-65.jpg v968>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.693 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.469+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e9d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg auth v982 ap=1 s=51270 n(v0 rc2023-10-16T14:35:58.197200+0000 b51270 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a78100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e9d:[<0x10000001cf4/picture-5.jpg v982>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e9d:[<0x10000001cf4/picture-5.jpg v982>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.693 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.472+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e01 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg auth v980 ap=1 s=46946 n(v0 rc2023-10-16T14:35:57.847206+0000 b46946 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53a79180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e01:[<0x10000001cf4/picture-58.jpg v980>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e01:[<0x10000001cf4/picture-58.jpg v980>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.693 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.473+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d43 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml auth v966 ap=1 s=31787 n(v0 rc2023-10-16T14:35:56.925222+0000 b31787 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53858c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d43:[<0x10000001cf4/article-28.xml v966>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d43:[<0x10000001cf4/article-28.xml v966>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.694 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.473+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d0f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml auth v996 ap=1 s=19248 n(v0 rc2023-10-16T14:35:56.804224+0000 b19248 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53859700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d0f:[<0x10000001cf4/article-52.xml v996>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d0f:[<0x10000001cf4/article-52.xml v996>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.694 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.473+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dc6 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg auth v972 ap=1 s=64571 n(v0 rc2023-10-16T14:35:57.436213+0000 b64571 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f70000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dc6:[<0x10000001cf4/picture-77.jpg v972>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dc6:[<0x10000001cf4/picture-77.jpg v972>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.694 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.474+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001de2 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg auth v976 ap=1 s=45666 n(v0 rc2023-10-16T14:35:57.647209+0000 b45666 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f70580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001de2:[<0x10000001cf4/picture-69.jpg v976>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001de2:[<0x10000001cf4/picture-69.jpg v976>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.694 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.475+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e13 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg auth v970 ap=1 s=54603 n(v0 rc2023-10-16T14:35:57.908205+0000 b54603 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f70b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e13:[<0x10000001cf4/picture-51.jpg v970>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e13:[<0x10000001cf4/picture-51.jpg v970>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.695 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.475+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg auth v988 ap=1 s=30838 n(v0 rc2023-10-16T14:35:57.785207+0000 b30838 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f71600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001df7:[<0x10000001cf4/picture-62.jpg v988>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001df7:[<0x10000001cf4/picture-62.jpg v988>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1723>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.695 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.475+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e22 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg auth v990 ap=1 s=26936 n(v0 rc2023-10-16T14:35:57.936204+0000 b26936 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53f71b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e22:[<0x10000001cf4/picture-46.jpg v990>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e22:[<0x10000001cf4/picture-46.jpg v990>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.695 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.476+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ea3 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg auth v967 ap=1 s=18731 n(v0 rc2023-10-16T14:35:58.209200+0000 b18731 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f72680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ea3:[<0x10000001cf4/picture-3.jpg v967>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ea3:[<0x10000001cf4/picture-3.jpg v967>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.695 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.479+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e8c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg auth v953 ap=1 s=31431 n(v0 rc2023-10-16T14:35:58.160200+0000 b31431 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f73700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e8c:[<0x10000001cf4/picture-11.jpg v953>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e8c:[<0x10000001cf4/picture-11.jpg v953>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.695 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.479+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ddf [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg auth v965 ap=1 s=36880 n(v0 rc2023-10-16T14:35:57.621210+0000 b36880 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ddf:[<0x10000001cf4/picture-70.jpg v965>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ddf:[<0x10000001cf4/picture-70.jpg v965>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2292>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.696 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.481+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001ddc [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg auth v957 ap=1 s=31058 n(v0 rc2023-10-16T14:35:57.610210+0000 b31058 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001ddc:[<0x10000001cf4/picture-71.jpg v957>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001ddc:[<0x10000001cf4/picture-71.jpg v957>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.696 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.485+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e08 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg auth v977 ap=1 s=29743 n(v0 rc2023-10-16T14:35:57.870205+0000 b29743 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f73180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e08:[<0x10000001cf4/picture-55.jpg v977>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e08:[<0x10000001cf4/picture-55.jpg v977>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.696 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.488+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d8c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml auth v975 ap=1 s=23252 n(v0 rc2023-10-16T14:35:57.104219+0000 b23252 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d8c:[<0x10000001cf4/article-5.xml v975>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d8c:[<0x10000001cf4/article-5.xml v975>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.696 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.490+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d33 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml auth v973 ap=1 s=29259 n(v0 rc2023-10-16T14:35:56.891222+0000 b29259 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d33:[<0x10000001cf4/article-34.xml v973>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d33:[<0x10000001cf4/article-34.xml v973>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.696 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.492+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d40 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml auth v969 ap=1 s=1550 n(v0 rc2023-10-16T14:35:56.919222+0000 b1550 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d40:[<0x10000001cf4/article-29.xml v969>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d40:[<0x10000001cf4/article-29.xml v969>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.697 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.493+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cff [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml auth v955 ap=1 s=7920 n(v0 rc2023-10-16T14:35:56.762224+0000 b7920 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cff:[<0x10000001cf4/article-60.xml v955>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cff:[<0x10000001cf4/article-60.xml v955>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.697 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.494+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d55 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml auth v961 ap=1 s=1221 n(v0 rc2023-10-16T14:35:56.971221+0000 b1221 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d55:[<0x10000001cf4/article-22.xml v961>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d55:[<0x10000001cf4/article-22.xml v961>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.697 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.494+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e35 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg auth v979 ap=1 s=20662 n(v0 rc2023-10-16T14:35:57.964204+0000 b20662 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e35:[<0x10000001cf4/picture-40.jpg v979>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e35:[<0x10000001cf4/picture-40.jpg v979>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.697 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.496+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001daf [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg auth v959 ap=1 s=21458 n(v0 rc2023-10-16T14:35:57.284216+0000 b21458 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54114580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001daf:[<0x10000001cf4/picture-84.jpg v959>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001daf:[<0x10000001cf4/picture-84.jpg v959>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.697 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.496+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e68 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg auth v942 ap=1 s=34581 n(v0 rc2023-10-16T14:35:58.068202+0000 b34581 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54115600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e68:[<0x10000001cf4/picture-23.jpg v942>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e68:[<0x10000001cf4/picture-23.jpg v942>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.698 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.499+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d15 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml auth v963 ap=1 s=14385 n(v0 rc2023-10-16T14:35:56.819223+0000 b14385 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce54114b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d15:[<0x10000001cf4/article-49.xml v963>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d15:[<0x10000001cf4/article-49.xml v963>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.698 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.500+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001df5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg auth v971 ap=1 s=25793 n(v0 rc2023-10-16T14:35:57.771207+0000 b25793 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 authpin=1 scrubqueue=0 0x55ce53f7cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001df5:[<0x10000001cf4/picture-63.jpg v971>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001df5:[<0x10000001cf4/picture-63.jpg v971>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.698 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.501+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dc0 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg auth v944 ap=1 s=42060 n(v0 rc2023-10-16T14:35:57.383214+0000 b42060 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54115b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dc0:[<0x10000001cf4/picture-79.jpg v944>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dc0:[<0x10000001cf4/picture-79.jpg v944>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.698 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.501+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d71 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml auth v952 ap=1 s=5378 n(v0 rc2023-10-16T14:35:57.034220+0000 b5378 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54116100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d71:[<0x10000001cf4/article-14.xml v952>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d71:[<0x10000001cf4/article-14.xml v952>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.699 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.501+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e25 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg auth v940 ap=1 s=38530 n(v0 rc2023-10-16T14:35:57.941204+0000 b38530 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54116680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e25:[<0x10000001cf4/picture-45.jpg v940>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e25:[<0x10000001cf4/picture-45.jpg v940>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.699 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.504+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e32 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg auth v938 ap=1 s=10462 n(v0 rc2023-10-16T14:35:57.959204+0000 b10462 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54116c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e32:[<0x10000001cf4/picture-41.jpg v938>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e32:[<0x10000001cf4/picture-41.jpg v938>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.699 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.505+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dfb [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg auth v950 ap=1 s=8388 n(v0 rc2023-10-16T14:35:57.813206+0000 b8388 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54117180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dfb:[<0x10000001cf4/picture-60.jpg v950>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dfb:[<0x10000001cf4/picture-60.jpg v950>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.699 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.505+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d1b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml auth v932 ap=1 s=3913 n(v0 rc2023-10-16T14:35:56.834223+0000 b3913 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce54117700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d1b:[<0x10000001cf4/article-46.xml v932>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d1b:[<0x10000001cf4/article-46.xml v932>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.699 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.505+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dec [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg auth v974 ap=1 s=58981 n(v0 rc2023-10-16T14:35:57.724208+0000 b58981 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b26000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dec:[<0x10000001cf4/picture-66.jpg v974>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dec:[<0x10000001cf4/picture-66.jpg v974>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.700 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.505+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d19 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml auth v966 ap=1 s=6583 n(v0 rc2023-10-16T14:35:56.830223+0000 b6583 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b26b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d19:[<0x10000001cf4/article-47.xml v966>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d19:[<0x10000001cf4/article-47.xml v966>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.700 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.506+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e6b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg auth v964 ap=1 s=8443 n(v0 rc2023-10-16T14:35:58.076202+0000 b8443 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b27080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e6b:[<0x10000001cf4/picture-22.jpg v964>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e6b:[<0x10000001cf4/picture-22.jpg v964>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.700 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.506+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d7a [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml auth v954 ap=1 s=15224 n(v0 rc2023-10-16T14:35:57.056219+0000 b15224 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b27600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d7a:[<0x10000001cf4/article-11.xml v954>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d7a:[<0x10000001cf4/article-11.xml v954>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.700 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.506+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dbd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg auth v972 ap=1 s=24204 n(v0 rc2023-10-16T14:35:57.368214+0000 b24204 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b26580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dbd:[<0x10000001cf4/picture-80.jpg v972>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dbd:[<0x10000001cf4/picture-80.jpg v972>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.701 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.506+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d6b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml auth v930 ap=1 s=26125 n(v0 rc2023-10-16T14:35:57.016220+0000 b26125 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b27b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d6b:[<0x10000001cf4/article-16.xml v930>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d6b:[<0x10000001cf4/article-16.xml v930>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.701 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.506+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d49 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml auth v962 ap=1 s=7857 n(v0 rc2023-10-16T14:35:56.944221+0000 b7857 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b28680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d49:[<0x10000001cf4/article-26.xml v962>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d49:[<0x10000001cf4/article-26.xml v962>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.701 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.507+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e80 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg auth v968 ap=1 s=57439 n(v0 rc2023-10-16T14:35:58.131201+0000 b57439 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b28c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e80:[<0x10000001cf4/picture-15.jpg v968>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e80:[<0x10000001cf4/picture-15.jpg v968>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.701 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.507+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d0b [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml auth v936 ap=1 s=11452 n(v0 rc2023-10-16T14:35:56.793224+0000 b11452 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b28100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d0b:[<0x10000001cf4/article-54.xml v936>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d0b:[<0x10000001cf4/article-54.xml v936>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.701 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.507+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e89 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg auth v978 ap=1 s=14157 n(v0 rc2023-10-16T14:35:58.153201+0000 b14157 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e89:[<0x10000001cf4/picture-12.jpg v978>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e89:[<0x10000001cf4/picture-12.jpg v978>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.702 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.507+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d7d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml auth v956 ap=1 s=31564 n(v0 rc2023-10-16T14:35:57.062219+0000 b31564 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d7d:[<0x10000001cf4/article-10.xml v956>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d7d:[<0x10000001cf4/article-10.xml v956>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.702 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.508+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e06 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg auth v928 ap=1 s=32886 n(v0 rc2023-10-16T14:35:57.863206+0000 b32886 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b29180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e06:[<0x10000001cf4/picture-56.jpg v928>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e06:[<0x10000001cf4/picture-56.jpg v928>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.702 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.509+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e29 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg auth v934 ap=1 s=20966 n(v0 rc2023-10-16T14:35:57.945204+0000 b20966 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e29:[<0x10000001cf4/picture-44.jpg v934>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e29:[<0x10000001cf4/picture-44.jpg v934>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.702 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.510+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dfe [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg auth v958 ap=1 s=52105 n(v0 rc2023-10-16T14:35:57.830206+0000 b52105 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dfe:[<0x10000001cf4/picture-59.jpg v958>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dfe:[<0x10000001cf4/picture-59.jpg v958>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.702 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.511+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d4c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml auth v980 ap=1 s=3762 n(v0 rc2023-10-16T14:35:56.958221+0000 b3762 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d4c:[<0x10000001cf4/article-25.xml v980>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d4c:[<0x10000001cf4/article-25.xml v980>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.703 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.511+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dd5 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg auth v984 ap=1 s=62837 n(v0 rc2023-10-16T14:35:57.550211+0000 b62837 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd4580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dd5:[<0x10000001cf4/picture-73.jpg v984>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dd5:[<0x10000001cf4/picture-73.jpg v984>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.703 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.512+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e6e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg auth v968 ap=1 s=14215 n(v0 rc2023-10-16T14:35:58.084202+0000 b14215 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd4b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e6e:[<0x10000001cf4/picture-21.jpg v968>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e6e:[<0x10000001cf4/picture-21.jpg v968>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.703 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.512+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dcd [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg auth v972 ap=1 s=51177 n(v0 rc2023-10-16T14:35:57.514212+0000 b51177 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd5600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dcd:[<0x10000001cf4/picture-75.jpg v972>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dcd:[<0x10000001cf4/picture-75.jpg v972>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.703 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.513+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d27 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml auth v988 ap=1 s=17637 n(v0 rc2023-10-16T14:35:56.861223+0000 b17637 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd6100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d27:[<0x10000001cf4/article-40.xml v988>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d27:[<0x10000001cf4/article-40.xml v988>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.704 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.513+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e53 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg auth v978 ap=1 s=64995 n(v0 rc2023-10-16T14:35:58.019203+0000 b64995 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd6680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e53:[<0x10000001cf4/picture-30.jpg v978>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e53:[<0x10000001cf4/picture-30.jpg v978>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.704 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.513+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d6e [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml auth v974 ap=1 s=1549 n(v0 rc2023-10-16T14:35:57.025220+0000 b1549 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55dd7180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d6e:[<0x10000001cf4/article-15.xml v974>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d6e:[<0x10000001cf4/article-15.xml v974>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.704 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.516+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e41 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg auth v970 ap=1 s=33662 n(v0 rc2023-10-16T14:35:57.989203+0000 b33662 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d64b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e41:[<0x10000001cf4/picture-36.jpg v970>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e41:[<0x10000001cf4/picture-36.jpg v970>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.704 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.516+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e2c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg auth v986 ap=1 s=56124 n(v0 rc2023-10-16T14:35:57.950204+0000 b56124 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d65080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e2c:[<0x10000001cf4/picture-43.jpg v986>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e2c:[<0x10000001cf4/picture-43.jpg v986>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.705 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.516+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d62 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml auth v966 ap=1 s=9160 n(v0 rc2023-10-16T14:35:56.995220+0000 b9160 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d65600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d62:[<0x10000001cf4/article-19.xml v966>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d62:[<0x10000001cf4/article-19.xml v966>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.705 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.517+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e71 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg auth v982 ap=1 s=62826 n(v0 rc2023-10-16T14:35:58.091202+0000 b62826 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d65b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e71:[<0x10000001cf4/picture-20.jpg v982>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e71:[<0x10000001cf4/picture-20.jpg v982>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.705 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.519+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d23 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml auth v976 ap=1 s=30763 n(v0 rc2023-10-16T14:35:56.852223+0000 b30763 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d66680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d23:[<0x10000001cf4/article-42.xml v976>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d23:[<0x10000001cf4/article-42.xml v976>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.705 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.519+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e1f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg auth v976 ap=1 s=21408 n(v0 rc2023-10-16T14:35:57.931204+0000 b21408 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b29700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e1f:[<0x10000001cf4/picture-47.jpg v976>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e1f:[<0x10000001cf4/picture-47.jpg v976>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.705 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.520+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d17 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml auth v970 ap=1 s=5866 n(v0 rc2023-10-16T14:35:56.825223+0000 b5866 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce53b2b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d17:[<0x10000001cf4/article-48.xml v970>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d17:[<0x10000001cf4/article-48.xml v970>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.706 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.520+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf7 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml auth v993 ap=1 s=12050 n(v0 rc2023-10-16T14:35:56.740225+0000 b12050 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d67180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cf7:[<0x10000001cf4/article-64.xml v993>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cf7:[<0x10000001cf4/article-64.xml v993>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.706 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.520+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d2f [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml auth v999 ap=1 s=3784 n(v0 rc2023-10-16T14:35:56.882222+0000 b3784 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d2f:[<0x10000001cf4/article-36.xml v999>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d2f:[<0x10000001cf4/article-36.xml v999>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.706 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.521+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e5c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg auth v967 ap=1 s=50274 n(v0 rc2023-10-16T14:35:58.036203+0000 b50274 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e5c:[<0x10000001cf4/picture-27.jpg v967>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e5c:[<0x10000001cf4/picture-27.jpg v967>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.706 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.521+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d77 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml auth v991 ap=1 s=5918 n(v0 rc2023-10-16T14:35:57.050220+0000 b5918 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d77:[<0x10000001cf4/article-12.xml v991>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d77:[<0x10000001cf4/article-12.xml v991>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.706 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.521+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e10 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg auth v987 ap=1 s=24785 n(v0 rc2023-10-16T14:35:57.900205+0000 b24785 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e10:[<0x10000001cf4/picture-52.jpg v987>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e10:[<0x10000001cf4/picture-52.jpg v987>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.707 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.522+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d2d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml auth v977 ap=1 s=30721 n(v0 rc2023-10-16T14:35:56.877223+0000 b30721 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d2d:[<0x10000001cf4/article-37.xml v977>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d2d:[<0x10000001cf4/article-37.xml v977>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.707 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.522+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d29 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml auth v975 ap=1 s=31945 n(v0 rc2023-10-16T14:35:56.865223+0000 b31945 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d29:[<0x10000001cf4/article-39.xml v975>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d29:[<0x10000001cf4/article-39.xml v975>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.707 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.522+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d98 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml auth v981 ap=1 s=8085 n(v0 rc2023-10-16T14:35:57.139218+0000 b8085 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d98:[<0x10000001cf4/article-1.xml v981>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d98:[<0x10000001cf4/article-1.xml v981>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.707 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.524+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e47 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg auth v995 ap=1 s=29925 n(v0 rc2023-10-16T14:35:58.003203+0000 b29925 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55d6f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e47:[<0x10000001cf4/picture-34.jpg v995>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e47:[<0x10000001cf4/picture-34.jpg v995>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.707 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.527+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e0d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg auth v997 ap=1 s=62251 n(v0 rc2023-10-16T14:35:57.891205+0000 b62251 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a94000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e0d:[<0x10000001cf4/picture-53.jpg v997>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e0d:[<0x10000001cf4/picture-53.jpg v997>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.708 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.528+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001de8 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg auth v1001 ap=1 s=39721 n(v0 rc2023-10-16T14:35:57.692209+0000 b39721 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a94580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001de8:[<0x10000001cf4/picture-67.jpg v1001>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001de8:[<0x10000001cf4/picture-67.jpg v1001>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.708 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.531+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e7d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg auth v969 ap=1 s=52764 n(v0 rc2023-10-16T14:35:58.122201+0000 b52764 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a95600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e7d:[<0x10000001cf4/picture-16.jpg v969>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e7d:[<0x10000001cf4/picture-16.jpg v969>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.708 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.536+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d52 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml auth v983 ap=1 s=1935 n(v0 rc2023-10-16T14:35:56.967221+0000 b1935 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a97180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d52:[<0x10000001cf4/article-23.xml v983>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d52:[<0x10000001cf4/article-23.xml v983>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.708 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.536+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dd2 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg auth v979 ap=1 s=18855 n(v0 rc2023-10-16T14:35:57.527211+0000 b18855 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a96100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dd2:[<0x10000001cf4/picture-74.jpg v979>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dd2:[<0x10000001cf4/picture-74.jpg v979>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.709 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.537+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001dca [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg auth v985 ap=1 s=50737 n(v0 rc2023-10-16T14:35:57.479212+0000 b50737 1=1+0) (iversion lock) caps={15381=pAsLsXsFsc/-@1} | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce559ea000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001dca:[<0x10000001cf4/picture-76.jpg v985>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001dca:[<0x10000001cf4/picture-76.jpg v985>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.709 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.538+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e1c [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg auth v971 ap=1 s=63245 n(v0 rc2023-10-16T14:35:57.926204+0000 b63245 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce559eb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e1c:[<0x10000001cf4/picture-48.jpg v971>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e1c:[<0x10000001cf4/picture-48.jpg v971>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.709 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.540+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001d68 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml auth v989 ap=1 s=23721 n(v0 rc2023-10-16T14:35:57.008220+0000 b23721 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce55a97700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001d68:[<0x10000001cf4/article-17.xml v989>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001d68:[<0x10000001cf4/article-17.xml v989>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.709 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.541+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001e4d [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg auth v1003 ap=1 s=39398 n(v0 rc2023-10-16T14:35:58.011203+0000 b39398 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce559ea580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001e4d:[<0x10000001cf4/picture-32.jpg v1003>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001e4d:[<0x10000001cf4/picture-32.jpg v1003>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:06.710 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:40:06.542+0000 7fe550ed5700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000001cf9 [2,head] /volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml auth v973 ap=1 s=5964 n(v0 rc2023-10-16T14:35:56.745225+0000 b5964 1=1+0) (ifile excl) (iversion lock) cr={15381=0-4194304@1} caps={15381=pAsLsXsFsxcrwb/pFscr@1},l=15381 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x55ce559eab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000001cf9:[<0x10000001cf4/article-63.xml v973>,<0x10000000223/blog-38 v185241>,<0x200000001fe/blogtest_in v2207>,<0x200000001f7/src v1717>,<0x20000000001/blogbench-1.0 v1555>,<0x20000000000/tmp v79>,<0x100000001fb/client.0 v70>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v97>,<0x10000000001/sv_1 v88>,<0x10000000000/_nogroup v73>,<0x1/volumes v81>]//[3]","memoryvalue":"(4)0x10000001cf9:[<0x10000001cf4/article-63.xml v973>,<0x10000000223/blog-38 v197967>,<0x200000001fe/blogtest_in v2307>,<0x200000001f7/src v1731>,<0x20000000001/blogbench-1.0 v1598>,<0x20000000000/tmp v119>,<0x100000001fb/client.0 v113>,<0x100000001fa/0041f271-b242-4329-b233-848e4005d408 v151>,<0x10000000001/sv_1 v94>,<0x10000000000/_nogroup v22>,<0x1/volumes v77>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-16T14:40:07.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009cf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000754 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000087a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009c3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a60 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000084c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a67 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e42 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a00 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000768 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.307 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000796 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007f3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.309 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000807 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000076c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000839 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000732 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008a4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.311 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a26 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000780 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000085d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.312 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000791 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000970 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000777 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009da (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.313 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008c7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000993 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008ae (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007bb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000098a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.314 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008a8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000772 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000955 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.315 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000749 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a3b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e8d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000921 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000982 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.316 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000834 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000891 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000967 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000926 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.317 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000082f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001324 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a57 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000737 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.318 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007aa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000e3c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000848 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a20 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.319 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000082a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008b8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000072c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000079c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000843 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.320 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000132b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000867 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000099c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000088e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.321 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000089a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000763 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009a5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.322 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000884 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008bf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007c1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000851 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.323 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000f21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000a15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000008bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000816 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000934 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000075e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009af (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000825 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000095e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000092a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000786 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.326 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.327 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000074d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000979 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000757 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000081b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.328 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000896 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000940 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000820 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000092f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000077c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.329 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000742 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x100000009e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000fc9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000087e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000094c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.330 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x1000000078a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.332 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009cf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000754 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000087a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009c3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.333 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a60 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000084c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a67 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e42 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a00 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.334 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000768 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000796 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007f3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000807 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000076c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.335 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000839 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000732 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.336 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008a4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a26 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000780 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000085d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.337 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000791 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000970 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000777 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009da (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008c7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.338 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000993 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008ae (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007bb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000098a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008a8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.339 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000772 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000955 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000749 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a3b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e8d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.340 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000921 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000982 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000834 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000891 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000967 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.341 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000926 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000082f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001324 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a57 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.342 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000737 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007aa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000e3c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.343 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000848 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a20 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000082a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008b8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000072c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.344 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000079c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000843 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000132b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000867 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000099c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.345 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000088e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000089a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.346 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000763 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009a5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000884 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008bf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007c1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.347 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000851 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000f21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000a15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.348 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000008bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000816 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000934 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000075e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009af (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.349 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000825 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000095e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000092a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000786 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.350 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000074d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000979 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000757 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.351 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000081b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000896 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000940 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000820 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000092f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000077c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000742 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x100000009e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000fc9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000087e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000094c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.353 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x1000000078a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000000862 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.354 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001db7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d46 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ea0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.355 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e44 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ea6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dc3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d5d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cfd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.356 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d1d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001df9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e97 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d3a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e92 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.357 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001df3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e90 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ea9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.358 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001db3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e9a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cf5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.359 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e59 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e3e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001da1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e4a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.360 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e38 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001de5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.361 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.362 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d83 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.362 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e50 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.362 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.362 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e5f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.362 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d37 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.363 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e0a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e56 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d07 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d9b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dba (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.364 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e16 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d05 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001def (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.365 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e9d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d43 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d0f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dc6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.366 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001de2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e13 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001df7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e22 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ea3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ddf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001ddc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e08 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.368 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d33 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d40 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cff (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d55 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e35 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.369 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001daf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001df5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dc0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.370 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e32 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d1b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.371 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dbd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.372 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d49 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e06 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.373 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dfe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d4c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dd5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.374 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dcd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d27 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e53 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e41 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e2c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.375 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d23 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d17 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.376 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cf7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e5c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e10 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.377 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d2d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d98 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001de8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.378 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d52 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dd2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001dca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e1c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.379 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001d68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.380 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001e4d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.380 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:06 smithi186 ceph-mon[134892]: Scrub error on inode 0x10000001cf9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.381 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000000862 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.381 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.381 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.382 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.382 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.382 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.382 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001db7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.382 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d46 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ea0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e44 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ea6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dc3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d5d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.384 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cfd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.384 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.384 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d1d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.384 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001df9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.384 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e97 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.385 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d3a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.385 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e92 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.385 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001df3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.385 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e90 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.385 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ea9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001db3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e9a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cf5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.386 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e59 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e3e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.387 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001da1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.388 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.388 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e4a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.388 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.388 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.388 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.389 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e38 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.389 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001de5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.389 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.389 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d83 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.389 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e50 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.390 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.390 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e5f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.390 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.390 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d37 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.390 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e0a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.391 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e56 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d07 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d9b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dba (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e16 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d05 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.392 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001def (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e9d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.393 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.394 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d43 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.394 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d0f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.394 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dc6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.394 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001de2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.394 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e13 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001df7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e22 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ea3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ddf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.395 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001ddc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.396 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e08 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.396 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.396 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d33 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.396 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d40 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.396 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cff (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.397 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d55 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.397 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e35 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.397 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001daf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.397 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.397 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.398 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001df5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.398 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dc0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.398 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.398 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.398 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e32 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.399 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.399 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d1b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.399 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.399 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.399 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.400 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.400 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dbd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.400 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.400 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d49 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.400 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.401 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.401 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.401 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.401 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e06 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.401 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dfe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d4c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dd5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.402 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dcd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d27 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e53 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e41 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e2c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.403 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.404 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.404 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d23 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.404 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.404 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d17 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.404 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cf7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e5c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e10 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d2d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.405 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.406 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d98 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.406 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.406 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.406 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001de8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.406 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.407 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d52 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.407 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dd2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.407 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001dca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.407 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e1c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.407 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001d68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.408 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001e4d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.408 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:06 smithi178 ceph-mon[98495]: Scrub error on inode 0x10000001cf9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009cf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000754 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000087a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009c3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007ce (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a60 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000084c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a67 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e42 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a00 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000768 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000796 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007f3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000807 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000076c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000839 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000732 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008a4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007f8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.429 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a26 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000780 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000085d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000791 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.430 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000970 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000777 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.431 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009da (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.433 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008c7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.433 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000993 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.433 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008ae (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.433 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007bb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.433 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000098a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.434 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008a8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.434 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000772 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.434 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000955 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.434 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000749 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.434 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a3b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e8d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000921 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000982 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000834 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000891 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.435 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000967 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000926 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007fd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000082f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001324 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a57 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007b0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000737 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007aa (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007ca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000e3c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000848 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a20 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000082a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008b8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000072c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000079c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000843 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000132b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000867 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000099c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.439 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007dd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.440 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000088e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.440 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007ee (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.440 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007e9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.440 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000089a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.440 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000763 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009a5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000884 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008bf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007c1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.441 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000851 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009f1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000f21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000a15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000007e4 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.442 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000008bd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000816 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000934 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000075e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009af (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000825 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.443 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000080c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.444 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000095e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.444 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000092a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.444 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009b9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.444 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000786 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.444 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000802 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.445 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000074d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.445 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000979 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.445 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000757 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.445 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000811 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.445 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000081b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.446 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000896 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.446 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000940 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.446 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000820 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.446 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000092f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.446 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000077c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.447 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000742 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.447 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x100000009e5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.447 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000fc9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.447 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000087e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.447 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000094c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x1000000078a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000000862 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.448 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.449 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.449 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001db7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.449 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d46 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.449 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ea0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.449 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e44 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.450 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ea6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.450 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dc3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.450 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d5d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.450 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cfd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.450 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.451 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d1d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.451 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001df9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.451 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e97 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.451 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d3a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.451 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e92 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.452 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001df3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.452 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e90 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.452 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ea9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.452 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.452 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.453 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001db3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.453 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e9a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.453 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cf5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.453 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.453 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d4f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e59 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e3e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001da1 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e65 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.454 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e4a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d86 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e38 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.455 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001de5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d83 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e50 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e5f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.456 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d37 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d58 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d74 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.457 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d95 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.458 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e0a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.458 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e56 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.458 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d07 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.458 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d9b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.458 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dba (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e16 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d05 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d03 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d21 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.459 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.460 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001def (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.460 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e9d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.460 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e01 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.460 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d43 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.460 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d0f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.461 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dc6 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.461 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001de2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.461 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e13 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.461 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001df7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.461 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e22 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.462 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ea3 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.462 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.462 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ddf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.462 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001ddc (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.462 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e08 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d8c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d33 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d40 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cff (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d55 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.463 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e35 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001daf (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d15 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001df5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.464 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dc0 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.465 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.465 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e25 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.465 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e32 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.465 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dfb (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.465 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d1b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.466 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dec (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.466 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d19 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.466 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.466 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d7a (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.466 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dbd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d6b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d49 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e80 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d0b (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e89 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.467 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.468 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e06 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.468 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.468 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dfe (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.468 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d4c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.468 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dd5 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dcd (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d27 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e53 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d6e (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.469 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e41 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.470 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e2c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.470 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d62 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.470 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e71 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.470 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d23 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e1f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d17 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cf7 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d2f (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e5c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.471 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d77 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.472 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e10 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.472 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d2d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.472 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d29 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.472 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d98 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.472 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e47 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e0d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001de8 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e7d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d52 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dd2 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.473 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001dca (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.474 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e1c (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.474 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001d68 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.474 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001e4d (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg) see mds.i log and `damage ls` output for details 2023-10-16T14:40:07.474 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:06 smithi187 ceph-mon[135297]: Scrub error on inode 0x10000001cf9 (/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml) see mds.i log and `damage ls` output for details 2023-10-16T14:40:08.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:07 smithi186 ceph-mon[134892]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-16T14:40:08.338 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:07 smithi178 ceph-mon[98495]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-16T14:40:08.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:07 smithi187 ceph-mon[135297]: pgmap v531: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 440 op/s 2023-10-16T14:40:10.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:09 smithi186 ceph-mon[134892]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2023-10-16T14:40:10.383 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:09 smithi178 ceph-mon[98495]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2023-10-16T14:40:10.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:09 smithi187 ceph-mon[135297]: pgmap v532: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 332 op/s 2023-10-16T14:40:11.084 INFO:tasks.workunit.client.0.smithi178.stdout: 381 590607 1148 413596 1044 299116 2109 2023-10-16T14:40:11.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:11 smithi178 ceph-mon[98495]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-10-16T14:40:11.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:11 smithi186 ceph-mon[134892]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-10-16T14:40:11.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:11 smithi187 ceph-mon[135297]: pgmap v533: 129 pgs: 129 active+clean; 1.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 340 op/s 2023-10-16T14:40:12.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:12 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:12.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:12 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:12.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:12 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:13.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:13 smithi178 ceph-mon[98495]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 366 op/s 2023-10-16T14:40:13.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:13 smithi186 ceph-mon[134892]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 366 op/s 2023-10-16T14:40:13.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:13 smithi187 ceph-mon[135297]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 366 op/s 2023-10-16T14:40:16.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:15 smithi187 ceph-mon[135297]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-10-16T14:40:16.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:15 smithi178 ceph-mon[98495]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-10-16T14:40:16.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:15 smithi186 ceph-mon[134892]: pgmap v535: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 339 op/s 2023-10-16T14:40:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:18.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:17 smithi187 ceph-mon[135297]: pgmap v536: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 343 op/s 2023-10-16T14:40:18.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:18.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.330 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:17 smithi186 ceph-mon[134892]: pgmap v536: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 343 op/s 2023-10-16T14:40:18.374 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.375 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.375 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.375 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.375 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.375 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:18.376 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:18.377 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:17 smithi178 ceph-mon[98495]: pgmap v536: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 343 op/s 2023-10-16T14:40:20.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:19 smithi178 ceph-mon[98495]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2023-10-16T14:40:20.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:19 smithi186 ceph-mon[134892]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2023-10-16T14:40:20.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:19 smithi187 ceph-mon[135297]: pgmap v537: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2023-10-16T14:40:21.092 INFO:tasks.workunit.client.0.smithi178.stdout: 395 577701 881 406007 830 283450 1676 2023-10-16T14:40:21.825 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:21 smithi178 ceph-mon[98495]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 442 op/s 2023-10-16T14:40:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:21 smithi186 ceph-mon[134892]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 442 op/s 2023-10-16T14:40:21.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:21 smithi187 ceph-mon[135297]: pgmap v538: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.9 MiB/s wr, 442 op/s 2023-10-16T14:40:23.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:40:24.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:23 smithi187 ceph-mon[135297]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 451 op/s 2023-10-16T14:40:24.308 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:23 smithi178 ceph-mon[98495]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 451 op/s 2023-10-16T14:40:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:23 smithi186 ceph-mon[134892]: pgmap v539: 129 pgs: 129 active+clean; 1.6 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.6 MiB/s wr, 451 op/s 2023-10-16T14:40:25.696 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.689+0000 7f952a39d700 1 -- 172.21.15.178:0/3699232254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108ea0 msgr2=0x7f952410f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:25.697 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.689+0000 7f952a39d700 1 --2- 172.21.15.178:0/3699232254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108ea0 0x7f952410f750 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f951c00b410 tx=0x7f951c00b720 comp rx=0 tx=0).stop 2023-10-16T14:40:25.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.804+0000 7f952a39d700 1 -- 172.21.15.178:0/3699232254 shutdown_connections 2023-10-16T14:40:25.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.804+0000 7f952a39d700 1 --2- 172.21.15.178:0/3699232254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108ea0 0x7f952410f750 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:25.866 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.804+0000 7f952a39d700 1 --2- 172.21.15.178:0/3699232254 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:25.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.804+0000 7f952a39d700 1 --2- 172.21.15.178:0/3699232254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108500 0x7f9524108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:25.867 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.804+0000 7f952a39d700 1 -- 172.21.15.178:0/3699232254 >> 172.21.15.178:0/3699232254 conn(0x7f95240762e0 msgr2=0x7f9524078740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:25.952 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:25.946+0000 7f952a39d700 1 -- 172.21.15.178:0/3699232254 shutdown_connections 2023-10-16T14:40:26.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.015+0000 7f952a39d700 1 -- 172.21.15.178:0/3699232254 wait complete. 2023-10-16T14:40:26.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.017+0000 7f952a39d700 1 Processor -- start 2023-10-16T14:40:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 -- start start 2023-10-16T14:40:26.077 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108500 0x7f9524070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 0x7f9524071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.078 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9524072370 con 0x7f9524108500 2023-10-16T14:40:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240724e0 con 0x7f9524107310 2023-10-16T14:40:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.068+0000 7f952a39d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95240726a0 con 0x7f9524108ea0 2023-10-16T14:40:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.069+0000 7f952939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.079 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.069+0000 7f952939b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524070820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:54442/0 (socket says 172.21.15.178:54442) 2023-10-16T14:40:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.069+0000 7f952939b700 1 -- 172.21.15.178:0/3440002032 learned_addr learned my addr 172.21.15.178:0/3440002032 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:26.080 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.070+0000 7f9529b9c700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 0x7f9524071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.073+0000 7f9529b9c700 1 -- 172.21.15.178:0/3440002032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 msgr2=0x7f9524070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.073+0000 7f9529b9c700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.081 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.073+0000 7f9529b9c700 1 -- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108500 msgr2=0x7f9524070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.073+0000 7f9529b9c700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108500 0x7f9524070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.082 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.073+0000 7f9529b9c700 1 -- 172.21.15.178:0/3440002032 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009d00 con 0x7f9524108ea0 2023-10-16T14:40:26.083 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.074+0000 7f9529b9c700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 0x7f9524071da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f951c00d010 tx=0x7f951c003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:26.088 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.085+0000 7f951a7fc700 1 -- 172.21.15.178:0/3440002032 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00e050 con 0x7f9524108ea0 2023-10-16T14:40:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.085+0000 7f951a7fc700 1 -- 172.21.15.178:0/3440002032 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951c005780 con 0x7f9524108ea0 2023-10-16T14:40:26.089 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.085+0000 7f951a7fc700 1 -- 172.21.15.178:0/3440002032 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c01bc10 con 0x7f9524108ea0 2023-10-16T14:40:26.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.102+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95240728c0 con 0x7f9524108ea0 2023-10-16T14:40:26.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.102+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9524072d50 con 0x7f9524108ea0 2023-10-16T14:40:26.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:25 smithi187 ceph-mon[135297]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 522 op/s 2023-10-16T14:40:26.285 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.222+0000 7f951a7fc700 1 -- 172.21.15.178:0/3440002032 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f951c01bd70 con 0x7f9524108ea0 2023-10-16T14:40:26.294 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.289+0000 7f951a7fc700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 0x7f95100a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.310 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.292+0000 7f951a7fc700 1 -- 172.21.15.178:0/3440002032 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f951c0ce2c0 con 0x7f9524108ea0 2023-10-16T14:40:26.311 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.297+0000 7f952939b700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 0x7f95100a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.324 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:25 smithi178 ceph-mon[98495]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 522 op/s 2023-10-16T14:40:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:25 smithi186 ceph-mon[134892]: pgmap v540: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.4 MiB/s wr, 522 op/s 2023-10-16T14:40:26.332 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.328+0000 7f952939b700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 0x7f95100a3c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f952000bcd0 tx=0x7f952000b2c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:26.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.405+0000 7f953ef31b80 1 Processor -- start 2023-10-16T14:40:26.507 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 -- start start 2023-10-16T14:40:26.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739881fb0 0x5637398823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563739883800 0x563739883c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.508 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637397f6c70 con 0x563739881fb0 2023-10-16T14:40:26.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637397dbc60 con 0x563739882910 2023-10-16T14:40:26.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.486+0000 7f953ef31b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637397d9580 con 0x563739883800 2023-10-16T14:40:26.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.493+0000 7f9528b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.509 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.493+0000 7f9528b9a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:54446/0 (socket says 172.21.15.178:54446) 2023-10-16T14:40:26.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.493+0000 7f9528b9a700 1 -- 172.21.15.178:0/1645358533 learned_addr learned my addr 172.21.15.178:0/1645358533 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:26.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.503+0000 7f9528b9a700 1 -- 172.21.15.178:0/1645358533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563739883800 msgr2=0x563739883c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:26.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.503+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563739883800 0x563739883c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.510 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.503+0000 7f9528b9a700 1 -- 172.21.15.178:0/1645358533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739881fb0 msgr2=0x5637398823d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:40:26.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.503+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739881fb0 0x5637398823d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.511 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.503+0000 7f9528b9a700 1 -- 172.21.15.178:0/1645358533 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637397923f0 con 0x563739882910 2023-10-16T14:40:26.513 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.510+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9514009cd0 tx=0x7f951400e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec466c4055ec11d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:26.516 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.510+0000 7f950f7fe700 1 -- 172.21.15.178:0/1645358533 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951400ec80 con 0x563739882910 2023-10-16T14:40:26.519 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.516+0000 7f950f7fe700 1 -- 172.21.15.178:0/1645358533 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951400ede0 con 0x563739882910 2023-10-16T14:40:26.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.518+0000 7f953ef31b80 1 -- 172.21.15.178:0/1645358533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 msgr2=0x563739883220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:26.528 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.518+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9514009cd0 tx=0x7f951400e350 comp rx=0 tx=0).stop 2023-10-16T14:40:26.530 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.527+0000 7f953ef31b80 1 -- 172.21.15.178:0/1645358533 shutdown_connections 2023-10-16T14:40:26.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.527+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563739883800 0x563739883c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.527+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739881fb0 0x5637398823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.527+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1645358533 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x563739882910 0x563739883220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.532 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.527+0000 7f953ef31b80 1 -- 172.21.15.178:0/1645358533 >> 172.21.15.178:0/1645358533 conn(0x5637397afdd0 msgr2=0x56373987e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:26.572 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.567+0000 7f953ef31b80 1 -- 172.21.15.178:0/1645358533 shutdown_connections 2023-10-16T14:40:26.573 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.569+0000 7f953ef31b80 1 -- 172.21.15.178:0/1645358533 wait complete. 2023-10-16T14:40:26.581 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.574+0000 7f953ef31b80 1 -- start start 2023-10-16T14:40:26.582 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.574+0000 7f953ef31b80 1 Processor -- start 2023-10-16T14:40:26.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.593+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.596+0000 7f952939b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.612 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.596+0000 7f952939b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:52984/0 (socket says 172.21.15.178:52984) 2023-10-16T14:40:26.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.609+0000 7f952939b700 1 -- 172.21.15.178:0/1828103635 learned_addr learned my addr 172.21.15.178:0/1828103635 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:26.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.611+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 0x563739891280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.611+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739893830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.611+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637397d4e70 con 0x563739883800 2023-10-16T14:40:26.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.611+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x563739771710 con 0x56373988c250 2023-10-16T14:40:26.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.612+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637397713b0 con 0x56373988c6d0 2023-10-16T14:40:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.616+0000 7f953ef31b80 10 client.0 _ll_register_callbacks cb 0x5637395ce800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:40:26.619 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.616+0000 7f952939b700 1 -- 172.21.15.178:0/1828103635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 msgr2=0x563739893830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:40:26.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.617+0000 7f952939b700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739893830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.618+0000 7f952939b700 1 -- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 msgr2=0x563739891280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:40:26.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.621+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 0x563739891280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.625+0000 7f952939b700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 0x563739891280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.626+0000 7f952939b700 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9514009980 con 0x563739883800 2023-10-16T14:40:26.634 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.629+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 0x563739891280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:40:26.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.632+0000 7f952939b700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f952000ff30 tx=0x7f9520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62533f108557595f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.657+0000 7f950cff9700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:40:26.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.657+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952001e450 con 0x563739883800 2023-10-16T14:40:26.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.657+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f952001e5b0 con 0x563739883800 2023-10-16T14:40:26.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.657+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563739851c10 con 0x563739883800 2023-10-16T14:40:26.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.658+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f952001e770 con 0x563739883800 2023-10-16T14:40:26.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.658+0000 7f953ef31b80 10 client.16188 fetch_fsmap learned FSMap version 36 2023-10-16T14:40:26.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.658+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5637395ffde0 con 0x563739883800 2023-10-16T14:40:26.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.660+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563739851c10 con 0x563739883800 2023-10-16T14:40:26.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f9520026a10 con 0x563739883800 2023-10-16T14:40:26.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 10 client.16188 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:40:26.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:40:26.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 20 client.16188 populate_metadata read hostname 'smithi178' 2023-10-16T14:40:26.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563739898420 0x56373989a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:26.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 4 client.16188 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:40:26.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.664+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637397cde50 con 0x563739898420 2023-10-16T14:40:26.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.666+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563739898420 0x56373989a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:26.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.669+0000 7f950cff9700 10 client.16188 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:26.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.669+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563739898420 0x56373989a840 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168f95697ef685ca server_cookie=9baeb5976772001 in_seq=0 out_seq=0 2023-10-16T14:40:26.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.672+0000 7f950cff9700 1 -- 172.21.15.178:0/1828103635 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5637397cde50 con 0x563739898420 2023-10-16T14:40:26.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.672+0000 7f950cff9700 10 client.16188 handle_command_reply: tid=0 2023-10-16T14:40:26.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.673+0000 7f953ef31b80 1 client.16188 shutdown 2023-10-16T14:40:26.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.675+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 msgr2=0x56373988bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:26.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.675+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f952000ff30 tx=0x7f9520004750 comp rx=0 tx=0).stop 2023-10-16T14:40:26.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 shutdown_connections 2023-10-16T14:40:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x563739898420 0x56373989a840 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739893830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x563739883800 0x56373988bd10 secure :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f952000ff30 tx=0x7f9520004750 comp rx=0 tx=0).stop 2023-10-16T14:40:26.719 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1828103635 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373988c250 0x563739891280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:26.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.691+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 >> 172.21.15.178:0/1828103635 conn(0x5637397afdd0 msgr2=0x56373987e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:26.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.725+0000 7f950cff9700 0 client.16188 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:26.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.812+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 shutdown_connections 2023-10-16T14:40:26.819 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.812+0000 7f953ef31b80 1 -- 172.21.15.178:0/1828103635 wait complete. 2023-10-16T14:40:26.942 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:26.931+0000 7f953ef31b80 20 client.16188 trim_cache size 0 max 16384 2023-10-16T14:40:27.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.758+0000 7f953ef31b80 1 Processor -- start 2023-10-16T14:40:27.811 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 -- start start 2023-10-16T14:40:27.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373988c250 0x5637398a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739894350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x563739655920 con 0x56373988c250 2023-10-16T14:40:27.812 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56373988a790 con 0x56373987edf0 2023-10-16T14:40:27.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.802+0000 7f953ef31b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x563739881bf0 con 0x56373988c6d0 2023-10-16T14:40:27.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:27.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:54472/0 (socket says 172.21.15.178:54472) 2023-10-16T14:40:27.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 -- 172.21.15.178:0/1853859200 learned_addr learned my addr 172.21.15.178:0/1853859200 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:27.813 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 -- 172.21.15.178:0/1853859200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 msgr2=0x563739894350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:40:27.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739894350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 -- 172.21.15.178:0/1853859200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373988c250 msgr2=0x5637398a8060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T14:40:27.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373988c250 0x5637398a8060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.803+0000 7f9529b9c700 1 -- 172.21.15.178:0/1853859200 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637395e8c60 con 0x56373987edf0 2023-10-16T14:40:27.814 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.804+0000 7f9529b9c700 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f951c0062a0 tx=0x7f951c00bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182c217691e1aa33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:27.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.804+0000 7f950ffff700 1 -- 172.21.15.178:0/1853859200 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c09ac80 con 0x56373987edf0 2023-10-16T14:40:27.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.804+0000 7f950ffff700 1 -- 172.21.15.178:0/1853859200 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f951c09ba20 con 0x56373987edf0 2023-10-16T14:40:27.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.804+0000 7f950ffff700 1 -- 172.21.15.178:0/1853859200 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c099750 con 0x56373987edf0 2023-10-16T14:40:27.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.805+0000 7f953ef31b80 1 -- 172.21.15.178:0/1853859200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 msgr2=0x563739882390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:27.815 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.805+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f951c0062a0 tx=0x7f951c00bd80 comp rx=0 tx=0).stop 2023-10-16T14:40:27.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.806+0000 7f953ef31b80 1 -- 172.21.15.178:0/1853859200 shutdown_connections 2023-10-16T14:40:27.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.806+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56373987edf0 0x563739882390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.806+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373988c250 0x5637398a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.806+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1853859200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c6d0 0x563739894350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.816 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.806+0000 7f953ef31b80 1 -- 172.21.15.178:0/1853859200 >> 172.21.15.178:0/1853859200 conn(0x5637397afdd0 msgr2=0x563739883010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:27.820 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.816+0000 7f953ef31b80 1 -- 172.21.15.178:0/1853859200 shutdown_connections 2023-10-16T14:40:27.821 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.816+0000 7f953ef31b80 1 -- 172.21.15.178:0/1853859200 wait complete. 2023-10-16T14:40:27.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.818+0000 7f953ef31b80 1 -- start start 2023-10-16T14:40:27.822 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.819+0000 7f953ef31b80 1 Processor -- start 2023-10-16T14:40:27.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.833+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373987edf0 0x5637398a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.833+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f953ef31b80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637398a68b0 0x5637398a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f953ef31b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637398a83d0 con 0x56373987edf0 2023-10-16T14:40:27.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f953ef31b80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637398ec9c0 con 0x5637398a68b0 2023-10-16T14:40:27.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f953ef31b80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637398ecb30 con 0x56373988c250 2023-10-16T14:40:27.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f953ef31b80 10 client.0 _ll_register_callbacks cb 0x7f952000b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:40:27.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f9528b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:27.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f9528b9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:36416/0 (socket says 172.21.15.178:36416) 2023-10-16T14:40:27.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.834+0000 7f9528b9a700 1 -- 172.21.15.178:0/1688237209 learned_addr learned my addr 172.21.15.178:0/1688237209 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:27.874 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.836+0000 7f9529b9c700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637398a68b0 0x5637398a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:27.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.852+0000 7f9528b9a700 1 -- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637398a68b0 msgr2=0x5637398a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:27.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.852+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637398a68b0 0x5637398a6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.852+0000 7f9528b9a700 1 -- 172.21.15.178:0/1688237209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373987edf0 msgr2=0x5637398a7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:27.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.852+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373987edf0 0x5637398a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.875 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.852+0000 7f9528b9a700 1 -- 172.21.15.178:0/1688237209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9524072d50 con 0x56373988c250 2023-10-16T14:40:27.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.871+0000 7f950cff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:40:27.876 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.873+0000 7f9528b9a700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9514009ca0 tx=0x7f9514005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d59cc5a0fab356 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:27.879 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.876+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514025d60 con 0x56373988c250 2023-10-16T14:40:27.883 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.878+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9514014070 con 0x56373988c250 2023-10-16T14:40:27.884 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.881+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637397d9ef0 con 0x56373988c250 2023-10-16T14:40:27.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.885+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9514025d60 con 0x56373988c250 2023-10-16T14:40:27.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.885+0000 7f953ef31b80 10 client.26230 fetch_fsmap learned FSMap version 36 2023-10-16T14:40:27.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.887+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x563739626250 con 0x56373988c250 2023-10-16T14:40:27.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.891+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637397d9ef0 con 0x56373988c250 2023-10-16T14:40:27.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.901+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f95140041c0 con 0x56373988c250 2023-10-16T14:40:27.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.901+0000 7f953ef31b80 10 client.26230 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:40:27.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.901+0000 7f953ef31b80 10 client.26230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:40:27.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.901+0000 7f953ef31b80 20 client.26230 populate_metadata read hostname 'smithi178' 2023-10-16T14:40:27.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.901+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5637398ee5b0 0x5637398f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:27.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.905+0000 7f953ef31b80 4 client.26230 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:40:27.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.905+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5637397cdb70 con 0x5637398ee5b0 2023-10-16T14:40:27.918 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:40:27.918 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "IDLE (waiting for more scrubs)", 2023-10-16T14:40:27.919 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": { 2023-10-16T14:40:27.919 INFO:teuthology.orchestra.run.smithi178.stdout: "fca1b280-3016-4a9c-8fd4-d0bd2508c9a7": { 2023-10-16T14:40:27.919 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/", 2023-10-16T14:40:27.919 INFO:teuthology.orchestra.run.smithi178.stdout: "tag": "fca1b280-3016-4a9c-8fd4-d0bd2508c9a7", 2023-10-16T14:40:27.919 INFO:teuthology.orchestra.run.smithi178.stdout: "options": "recursive,force" 2023-10-16T14:40:27.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:40:27.920 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:40:27.920 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:40:27.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.912+0000 7f952939b700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5637398ee5b0 0x5637398f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:27.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.912+0000 7f952939b700 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5637398ee5b0 0x5637398f09d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d1c230190308bc server_cookie=9a421e28ec8e1ca0 in_seq=0 out_seq=0 2023-10-16T14:40:27.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.912+0000 7f950cff9700 10 client.26230 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:27.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f950cff9700 1 -- 172.21.15.178:0/1688237209 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5637397cdb70 con 0x5637398ee5b0 2023-10-16T14:40:27.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f950cff9700 10 client.26230 handle_command_reply: tid=0 2023-10-16T14:40:27.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 client.26230 shutdown 2023-10-16T14:40:27.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 msgr2=0x5637398a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:27.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9514009ca0 tx=0x7f9514005990 comp rx=0 tx=0).stop 2023-10-16T14:40:27.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 shutdown_connections 2023-10-16T14:40:27.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5637398ee5b0 0x5637398f09d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5637398a68b0 0x5637398a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56373987edf0 0x5637398a7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 --2- 172.21.15.178:0/1688237209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56373988c250 0x5637398a6370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 >> 172.21.15.178:0/1688237209 conn(0x5637397afdd0 msgr2=0x5637398898d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:27.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.913+0000 7f950cff9700 0 client.26230 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:27.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.914+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 shutdown_connections 2023-10-16T14:40:27.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.914+0000 7f953ef31b80 1 -- 172.21.15.178:0/1688237209 wait complete. 2023-10-16T14:40:27.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.914+0000 7f953ef31b80 20 client.26230 trim_cache size 0 max 16384 2023-10-16T14:40:27.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.916+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 msgr2=0x7f95100a3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:27.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.916+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 0x7f95100a3c20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f952000bcd0 tx=0x7f952000b2c0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.916+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 msgr2=0x7f9524071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:27.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.916+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 0x7f9524071da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f951c00d010 tx=0x7f951c003ce0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 shutdown_connections 2023-10-16T14:40:27.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f95100a1760 0x7f95100a3c20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9524108ea0 0x7f9524071da0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f9524107310 0x7f9524070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 --2- 172.21.15.178:0/3440002032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9524108500 0x7f9524070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:27.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 >> 172.21.15.178:0/3440002032 conn(0x7f95240762e0 msgr2=0x7f952410b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:27.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.917+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 shutdown_connections 2023-10-16T14:40:27.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:27.919+0000 7f952a39d700 1 -- 172.21.15.178:0/3440002032 wait complete. 2023-10-16T14:40:28.325 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:27 smithi178 ceph-mon[98495]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 719 op/s 2023-10-16T14:40:28.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:27 smithi186 ceph-mon[134892]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 719 op/s 2023-10-16T14:40:28.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:27 smithi187 ceph-mon[135297]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 719 op/s 2023-10-16T14:40:28.709 INFO:tasks.cephfs.filesystem:scrub status for tag:fca1b280-3016-4a9c-8fd4-d0bd2508c9a7 - {'path': '/', 'tag': 'fca1b280-3016-4a9c-8fd4-d0bd2508c9a7', 'options': 'recursive,force'} 2023-10-16T14:40:30.310 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:29 smithi178 ceph-mon[98495]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 684 op/s 2023-10-16T14:40:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:29 smithi186 ceph-mon[134892]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 684 op/s 2023-10-16T14:40:30.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:29 smithi187 ceph-mon[135297]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 684 op/s 2023-10-16T14:40:31.105 INFO:tasks.workunit.client.0.smithi178.stdout: 400 504409 307 350969 205 249575 830 2023-10-16T14:40:31.805 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:31 smithi178 ceph-mon[98495]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-16T14:40:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:31 smithi186 ceph-mon[134892]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-16T14:40:31.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:31 smithi187 ceph-mon[135297]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 779 op/s 2023-10-16T14:40:34.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:33 smithi187 ceph-mon[135297]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 676 op/s 2023-10-16T14:40:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:33 smithi178 ceph-mon[98495]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 676 op/s 2023-10-16T14:40:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:33 smithi186 ceph-mon[134892]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 676 op/s 2023-10-16T14:40:35.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:35.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:34 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:34 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:35.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:35.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:34 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:35 smithi187 ceph-mon[135297]: scrub summary: active paths [/] 2023-10-16T14:40:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:35 smithi187 ceph-mon[135297]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:36.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:35 smithi187 ceph-mon[135297]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-16T14:40:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:35 smithi178 ceph-mon[98495]: scrub summary: active paths [/] 2023-10-16T14:40:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:35 smithi178 ceph-mon[98495]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:35 smithi178 ceph-mon[98495]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-16T14:40:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:35 smithi186 ceph-mon[134892]: scrub summary: active paths [/] 2023-10-16T14:40:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:35 smithi186 ceph-mon[134892]: scrub summary: idle+waiting paths [/] 2023-10-16T14:40:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:35 smithi186 ceph-mon[134892]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 670 op/s 2023-10-16T14:40:38.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:37 smithi187 ceph-mon[135297]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 684 op/s 2023-10-16T14:40:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:37 smithi178 ceph-mon[98495]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 684 op/s 2023-10-16T14:40:38.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:37 smithi186 ceph-mon[134892]: pgmap v546: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 684 op/s 2023-10-16T14:40:40.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:39 smithi187 ceph-mon[135297]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 444 op/s 2023-10-16T14:40:40.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:39 smithi178 ceph-mon[98495]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 444 op/s 2023-10-16T14:40:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:39 smithi186 ceph-mon[134892]: pgmap v547: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 444 op/s 2023-10-16T14:40:41.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:41 smithi178 ceph-mon[98495]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 443 op/s 2023-10-16T14:40:41.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:41 smithi186 ceph-mon[134892]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 443 op/s 2023-10-16T14:40:41.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:41 smithi187 ceph-mon[135297]: pgmap v548: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 443 op/s 2023-10-16T14:40:44.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:43 smithi187 ceph-mon[135297]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-10-16T14:40:44.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:43 smithi178 ceph-mon[98495]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-10-16T14:40:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:43 smithi186 ceph-mon[134892]: pgmap v549: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2023-10-16T14:40:46.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:45 smithi186 ceph-mon[134892]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2023-10-16T14:40:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:45 smithi187 ceph-mon[135297]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2023-10-16T14:40:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:45 smithi178 ceph-mon[98495]: pgmap v550: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2023-10-16T14:40:47.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:46 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:40:47.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:46 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:40:47.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:46 smithi187 ceph-mon[135297]: scrub summary: idle 2023-10-16T14:40:47.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:46 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:40:47.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:46 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:40:47.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:46 smithi178 ceph-mon[98495]: scrub summary: idle 2023-10-16T14:40:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:46 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:40:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:46 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:40:47.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:46 smithi186 ceph-mon[134892]: scrub summary: idle 2023-10-16T14:40:48.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:47 smithi187 ceph-mon[135297]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2023-10-16T14:40:48.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:47 smithi178 ceph-mon[98495]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2023-10-16T14:40:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:47 smithi186 ceph-mon[134892]: pgmap v551: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2023-10-16T14:40:50.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:49 smithi178 ceph-mon[98495]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 134 KiB/s wr, 48 op/s 2023-10-16T14:40:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:49 smithi186 ceph-mon[134892]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 134 KiB/s wr, 48 op/s 2023-10-16T14:40:50.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:49 smithi187 ceph-mon[135297]: pgmap v552: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 134 KiB/s wr, 48 op/s 2023-10-16T14:40:51.051 INFO:tasks.workunit.client.0.smithi178.stdout: 2023-10-16T14:40:51.052 INFO:tasks.workunit.client.0.smithi178.stdout:Final score for writes: 400 2023-10-16T14:40:51.052 INFO:tasks.workunit.client.0.smithi178.stdout:Final score for reads : 311846 2023-10-16T14:40:51.052 INFO:tasks.workunit.client.0.smithi178.stdout: 2023-10-16T14:40:51.053 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-10-16T14:40:51.054 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-16T14:40:51.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:51 smithi178 ceph-mon[98495]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 321 KiB/s wr, 54 op/s 2023-10-16T14:40:51.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:51 smithi186 ceph-mon[134892]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 321 KiB/s wr, 54 op/s 2023-10-16T14:40:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:51 smithi187 ceph-mon[135297]: pgmap v553: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 321 KiB/s wr, 54 op/s 2023-10-16T14:40:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:53 smithi178 ceph-mon[98495]: pgmap v554: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 363 KiB/s wr, 39 op/s 2023-10-16T14:40:54.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:53 smithi186 ceph-mon[134892]: pgmap v554: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 363 KiB/s wr, 39 op/s 2023-10-16T14:40:54.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:53 smithi187 ceph-mon[135297]: pgmap v554: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 363 KiB/s wr, 39 op/s 2023-10-16T14:40:56.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:55 smithi178 ceph-mon[98495]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 858 KiB/s wr, 59 op/s 2023-10-16T14:40:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:55 smithi186 ceph-mon[134892]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 858 KiB/s wr, 59 op/s 2023-10-16T14:40:56.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:55 smithi187 ceph-mon[135297]: pgmap v555: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 858 KiB/s wr, 59 op/s 2023-10-16T14:40:58.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:57 smithi178 ceph-mon[98495]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-10-16T14:40:58.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:57 smithi186 ceph-mon[134892]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-10-16T14:40:58.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:57 smithi187 ceph-mon[135297]: pgmap v556: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 2.0 MiB/s wr, 184 op/s 2023-10-16T14:40:58.710 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-10-16T14:40:59.103 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 -- 172.21.15.178:0/2139553181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 msgr2=0x7fe9fc104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 --2- 172.21.15.178:0/2139553181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc104740 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00099c0 tx=0x7fe9f0009cd0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 -- 172.21.15.178:0/2139553181 shutdown_connections 2023-10-16T14:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 --2- 172.21.15.178:0/2139553181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc10b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.104 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 --2- 172.21.15.178:0/2139553181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 0x7fe9fc103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 --2- 172.21.15.178:0/2139553181 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc104740 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.100+0000 7fea037ff700 1 -- 172.21.15.178:0/2139553181 >> 172.21.15.178:0/2139553181 conn(0x7fe9fc0fe620 msgr2=0x7fe9fc100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.101+0000 7fea037ff700 1 -- 172.21.15.178:0/2139553181 shutdown_connections 2023-10-16T14:40:59.105 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.101+0000 7fea037ff700 1 -- 172.21.15.178:0/2139553181 wait complete. 2023-10-16T14:40:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.102+0000 7fea037ff700 1 Processor -- start 2023-10-16T14:40:59.106 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.102+0000 7fea037ff700 1 -- start start 2023-10-16T14:40:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 0x7fe9fc198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.107 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1a0a00 con 0x7fe9fc1030f0 2023-10-16T14:40:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1a0b40 con 0x7fe9fc1042e0 2023-10-16T14:40:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea037ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1a0cd0 con 0x7fe9fc104c80 2023-10-16T14:40:59.108 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52930/0 (socket says 172.21.15.178:52930) 2023-10-16T14:40:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea01d9c700 1 -- 172.21.15.178:0/2896597702 learned_addr learned my addr 172.21.15.178:0/2896597702 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.104+0000 7fea00d9a700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.109 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea0159b700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 0x7fe9fc198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea00d9a700 1 -- 172.21.15.178:0/2896597702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 msgr2=0x7fe9fc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea00d9a700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.110 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea00d9a700 1 -- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 msgr2=0x7fe9fc198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea00d9a700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 0x7fe9fc198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.105+0000 7fea00d9a700 1 -- 172.21.15.178:0/2896597702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00095f0 con 0x7fe9fc1042e0 2023-10-16T14:40:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fea00d9a700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc1993c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098e0 tx=0x7fe9f00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.111 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fe9fa7fc700 1 -- 172.21.15.178:0/2896597702 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f001d070 con 0x7fe9fc1042e0 2023-10-16T14:40:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fe9fa7fc700 1 -- 172.21.15.178:0/2896597702 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f0003e80 con 0x7fe9fc1042e0 2023-10-16T14:40:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fe9fa7fc700 1 -- 172.21.15.178:0/2896597702 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000fc80 con 0x7fe9fc1042e0 2023-10-16T14:40:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc1a0f50 con 0x7fe9fc1042e0 2023-10-16T14:40:59.112 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.106+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc1a12c0 con 0x7fe9fc1042e0 2023-10-16T14:40:59.115 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.112+0000 7fe9fa7fc700 1 -- 172.21.15.178:0/2896597702 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fe9f000f460 con 0x7fe9fc1042e0 2023-10-16T14:40:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.120+0000 7fea171bbb80 1 Processor -- start 2023-10-16T14:40:59.123 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.121+0000 7fe9fa7fc700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 0x7fe9e40a3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.121+0000 7fe9fa7fc700 1 -- 172.21.15.178:0/2896597702 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9f00ce000 con 0x7fe9fc1042e0 2023-10-16T14:40:59.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 -- start start 2023-10-16T14:40:59.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db484ac70 con 0x561db48def90 2023-10-16T14:40:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.122+0000 7fea171bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db482fc60 con 0x561db48dfe30 2023-10-16T14:40:59.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea171bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db482d580 con 0x561db48de630 2023-10-16T14:40:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52934/0 (socket says 172.21.15.178:52934) 2023-10-16T14:40:59.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 -- 172.21.15.178:0/3181707110 learned_addr learned my addr 172.21.15.178:0/3181707110 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 -- 172.21.15.178:0/3181707110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 msgr2=0x561db48e02b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:40:59.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48e02b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.128 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 -- 172.21.15.178:0/3181707110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 msgr2=0x561db48df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.123+0000 7fea0159b700 1 -- 172.21.15.178:0/3181707110 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db47e63f0 con 0x561db48de630 2023-10-16T14:40:59.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.124+0000 7fea01d9c700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 0x7fe9e40a3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.124+0000 7fea0159b700 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400a950 tx=0x7fe9f400ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a39caf61790c4bb3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.129 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.125+0000 7fe9e37fe700 1 -- 172.21.15.178:0/3181707110 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400aea0 con 0x561db48de630 2023-10-16T14:40:59.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.126+0000 7fe9e37fe700 1 -- 172.21.15.178:0/3181707110 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f40113f0 con 0x561db48de630 2023-10-16T14:40:59.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.126+0000 7fe9e37fe700 1 -- 172.21.15.178:0/3181707110 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f40185c0 con 0x561db48de630 2023-10-16T14:40:59.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.127+0000 7fea171bbb80 1 -- 172.21.15.178:0/3181707110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 msgr2=0x561db48dea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.127+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400a950 tx=0x7fe9f400ac60 comp rx=0 tx=0).stop 2023-10-16T14:40:59.131 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.127+0000 7fea01d9c700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 0x7fe9e40a3b10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x561db48df610 tx=0x7fe9ec00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.129+0000 7fea171bbb80 1 -- 172.21.15.178:0/3181707110 shutdown_connections 2023-10-16T14:40:59.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.129+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48e02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.129+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48de630 0x561db48dea50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.129+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3181707110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.129+0000 7fea171bbb80 1 -- 172.21.15.178:0/3181707110 >> 172.21.15.178:0/3181707110 conn(0x561db4803dd0 msgr2=0x561db48d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.130+0000 7fea171bbb80 1 -- 172.21.15.178:0/3181707110 shutdown_connections 2023-10-16T14:40:59.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.130+0000 7fea171bbb80 1 -- 172.21.15.178:0/3181707110 wait complete. 2023-10-16T14:40:59.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.131+0000 7fea171bbb80 1 -- start start 2023-10-16T14:40:59.135 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.131+0000 7fea171bbb80 1 Processor -- start 2023-10-16T14:40:59.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db48d73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db48d7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.136 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db4828e70 con 0x561db48de630 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db47c5710 con 0x561db48dfe30 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db47c53b0 con 0x561db48def90 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53072/0 (socket says 172.21.15.178:53072) 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea01d9c700 1 -- 172.21.15.178:0/1487219985 learned_addr learned my addr 172.21.15.178:0/1487219985 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.132+0000 7fea171bbb80 10 client.0 _ll_register_callbacks cb 0x561db4622800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:40:59.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 -- 172.21.15.178:0/1487219985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 msgr2=0x561db48d7920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T14:40:59.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db48d7920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 -- 172.21.15.178:0/1487219985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 msgr2=0x561db48d73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db48d73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 -- 172.21.15.178:0/1487219985 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f400a5b0 con 0x561db48dfe30 2023-10-16T14:40:59.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fea01d9c700 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec008ac0 tx=0x7fe9ec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aa4c5d1c4dd6ac0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.133+0000 7fe9e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:40:59.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.134+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec01c040 con 0x561db48dfe30 2023-10-16T14:40:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.134+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561db482def0 con 0x561db48dfe30 2023-10-16T14:40:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.134+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9ec017ce0 con 0x561db48dfe30 2023-10-16T14:40:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.135+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9ec01c040 con 0x561db48dfe30 2023-10-16T14:40:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.135+0000 7fea171bbb80 10 client.26267 fetch_fsmap learned FSMap version 36 2023-10-16T14:40:59.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.135+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561db4653de0 con 0x561db48dfe30 2023-10-16T14:40:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.135+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561db482def0 con 0x561db48dfe30 2023-10-16T14:40:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.135+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe9ec015480 con 0x561db48dfe30 2023-10-16T14:40:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 10 client.26267 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:40:59.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 10 client.26267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:40:59.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 20 client.26267 populate_metadata read hostname 'smithi178' 2023-10-16T14:40:59.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db48fcc70 0x561db48ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 4 client.26267 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:40:59.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.136+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561db48a5c10 con 0x561db48fcc70 2023-10-16T14:40:59.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.137+0000 7fea0159b700 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db48fcc70 0x561db48ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.137+0000 7fe9e0ff9700 10 client.26267 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:59.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.139+0000 7fea0159b700 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db48fcc70 0x561db48ff090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d7a9866e18405d server_cookie=24afeb2f2033b184 in_seq=0 out_seq=0 2023-10-16T14:40:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.143+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1487219985 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561db48a5c10 con 0x561db48fcc70 2023-10-16T14:40:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.143+0000 7fe9e0ff9700 10 client.26267 handle_command_reply: tid=0 2023-10-16T14:40:59.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.143+0000 7fea171bbb80 1 client.26267 shutdown 2023-10-16T14:40:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 msgr2=0x561db48dc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.147 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec008ac0 tx=0x7fe9ec004ad0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 shutdown_connections 2023-10-16T14:40:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db48fcc70 0x561db48ff090 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db48dc950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db48d73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.148 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1487219985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db48d7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 >> 172.21.15.178:0/1487219985 conn(0x561db4803dd0 msgr2=0x561db48d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.144+0000 7fe9e0ff9700 0 client.26267 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.145+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 shutdown_connections 2023-10-16T14:40:59.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.145+0000 7fea171bbb80 1 -- 172.21.15.178:0/1487219985 wait complete. 2023-10-16T14:40:59.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.145+0000 7fea171bbb80 20 client.26267 trim_cache size 0 max 16384 2023-10-16T14:40:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.190+0000 7fea171bbb80 1 Processor -- start 2023-10-16T14:40:59.193 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.190+0000 7fea171bbb80 1 -- start start 2023-10-16T14:40:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.190+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 0x561db490c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.190+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.190+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 0x561db48d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.191+0000 7fea171bbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db46a9920 con 0x561db48def90 2023-10-16T14:40:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.191+0000 7fea171bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db482def0 con 0x561db48de630 2023-10-16T14:40:59.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.191+0000 7fea171bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db48d5dd0 con 0x561db48dfe30 2023-10-16T14:40:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.191+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 0x561db48d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea01d9c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 0x561db48d0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52960/0 (socket says 172.21.15.178:52960) 2023-10-16T14:40:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea01d9c700 1 -- 172.21.15.178:0/3021907960 learned_addr learned my addr 172.21.15.178:0/3021907960 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea00d9a700 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.196 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea0159b700 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 0x561db490c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea0159b700 1 -- 172.21.15.178:0/3021907960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 msgr2=0x561db48d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.197 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea0159b700 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 0x561db48d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.192+0000 7fea0159b700 1 -- 172.21.15.178:0/3021907960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 msgr2=0x561db48d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.193+0000 7fea0159b700 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.198 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.193+0000 7fea0159b700 1 -- 172.21.15.178:0/3021907960 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db463cc60 con 0x561db48de630 2023-10-16T14:40:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.196+0000 7fea0159b700 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 0x561db490c870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400a1a0 tx=0x7fe9f4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c24456051fe2691 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.196+0000 7fe9e3fff700 1 -- 172.21.15.178:0/3021907960 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f401f040 con 0x561db48de630 2023-10-16T14:40:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.196+0000 7fe9e3fff700 1 -- 172.21.15.178:0/3021907960 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f402a070 con 0x561db48de630 2023-10-16T14:40:59.199 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.196+0000 7fe9e3fff700 1 -- 172.21.15.178:0/3021907960 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400ce10 con 0x561db48de630 2023-10-16T14:40:59.202 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.200+0000 7fea171bbb80 1 -- 172.21.15.178:0/3021907960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 msgr2=0x561db490c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.200+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 0x561db490c870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400a1a0 tx=0x7fe9f4000bc0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.203 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 -- 172.21.15.178:0/3021907960 shutdown_connections 2023-10-16T14:40:59.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48dfe30 0x561db48d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48de630 0x561db490c870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 --2- 172.21.15.178:0/3021907960 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48def90 0x561db48d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.204 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 -- 172.21.15.178:0/3021907960 >> 172.21.15.178:0/3021907960 conn(0x561db4803dd0 msgr2=0x561db48dd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 -- 172.21.15.178:0/3021907960 shutdown_connections 2023-10-16T14:40:59.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.201+0000 7fea171bbb80 1 -- 172.21.15.178:0/3021907960 wait complete. 2023-10-16T14:40:59.205 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.202+0000 7fea171bbb80 1 -- start start 2023-10-16T14:40:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.202+0000 7fea171bbb80 1 Processor -- start 2023-10-16T14:40:59.206 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db49493e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db4950300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db4950840 con 0x561db48de630 2023-10-16T14:40:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db49509b0 con 0x561db48dfe30 2023-10-16T14:40:59.207 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db4950b70 con 0x561db48def90 2023-10-16T14:40:59.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea171bbb80 10 client.0 _ll_register_callbacks cb 0x7fe9ec009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:40:59.208 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea00d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea00d9a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52974/0 (socket says 172.21.15.178:52974) 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.204+0000 7fea00d9a700 1 -- 172.21.15.178:0/1058107329 learned_addr learned my addr 172.21.15.178:0/1058107329 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea0159b700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db49493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea00d9a700 1 -- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 msgr2=0x561db4950300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea00d9a700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db4950300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.209 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea00d9a700 1 -- 172.21.15.178:0/1058107329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 msgr2=0x561db49493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:40:59.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea00d9a700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db49493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea00d9a700 1 -- 172.21.15.178:0/1058107329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561db47e63f0 con 0x561db48def90 2023-10-16T14:40:59.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.205+0000 7fea0159b700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db49493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:40:59.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.206+0000 7fe9e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2023-10-16T14:40:59.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.206+0000 7fea00d9a700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00cb730 tx=0x7fe9f009b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b1ab3061f5587d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.212 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.210+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f009f130 con 0x561db48def90 2023-10-16T14:40:59.213 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.210+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mon.1 v2:172.21.15.187:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe9f00a32e0 con 0x561db48def90 2023-10-16T14:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.210+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9f40113f0 con 0x561db48def90 2023-10-16T14:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.211+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9f009a4d0 con 0x561db48def90 2023-10-16T14:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.211+0000 7fea171bbb80 10 client.26245 fetch_fsmap learned FSMap version 36 2023-10-16T14:40:59.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.211+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x561db467a250 con 0x561db48def90 2023-10-16T14:40:59.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mon.1 v2:172.21.15.187:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9f40113f0 con 0x561db48def90 2023-10-16T14:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mon.1 v2:172.21.15.187:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fe9f0096dd0 con 0x561db48def90 2023-10-16T14:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 10 client.26245 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 10 client.26245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:40:59.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 20 client.26245 populate_metadata read hostname 'smithi178' 2023-10-16T14:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db49522c0 0x561db49546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 4 client.26245 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-10-16T14:40:59.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.214+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561db4821e50 con 0x561db49522c0 2023-10-16T14:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.215+0000 7fea0159b700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db49522c0 0x561db49546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.217+0000 7fea0159b700 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db49522c0 0x561db49546e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d8b6d86c35fb1e server_cookie=62b3120e2a01cdf in_seq=0 out_seq=0 2023-10-16T14:40:59.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.217+0000 7fe9e0ff9700 10 client.26245 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.220+0000 7fe9e0ff9700 1 -- 172.21.15.178:0/1058107329 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561db4821e50 con 0x561db49522c0 2023-10-16T14:40:59.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.220+0000 7fe9e0ff9700 10 client.26245 handle_command_reply: tid=0 2023-10-16T14:40:59.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.220+0000 7fea171bbb80 1 client.26245 shutdown 2023-10-16T14:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 msgr2=0x561db4949920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00cb730 tx=0x7fe9f009b790 comp rx=0 tx=0).stop 2023-10-16T14:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 shutdown_connections 2023-10-16T14:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x561db49522c0 0x561db49546e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x561db48dfe30 0x561db4950300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x561db48de630 0x561db49493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 --2- 172.21.15.178:0/1058107329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561db48def90 0x561db4949920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fe9e0ff9700 0 client.26245 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:40:59.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 >> 172.21.15.178:0/1058107329 conn(0x561db4803dd0 msgr2=0x561db48d5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.221+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 shutdown_connections 2023-10-16T14:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.222+0000 7fea171bbb80 1 -- 172.21.15.178:0/1058107329 wait complete. 2023-10-16T14:40:59.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.223+0000 7fea171bbb80 20 client.26245 trim_cache size 0 max 16384 2023-10-16T14:40:59.228 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:40:59.228 INFO:teuthology.orchestra.run.smithi178.stdout: "status": "no active scrubs running", 2023-10-16T14:40:59.228 INFO:teuthology.orchestra.run.smithi178.stdout: "scrubs": {} 2023-10-16T14:40:59.228 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.223+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 msgr2=0x7fe9e40a3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.223+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 0x7fe9e40a3b10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x561db48df610 tx=0x7fe9ec00b040 comp rx=0 tx=0).stop 2023-10-16T14:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.223+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 msgr2=0x7fe9fc1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.223+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc1993c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098e0 tx=0x7fe9f00045e0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 shutdown_connections 2023-10-16T14:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fe9e40a1650 0x7fe9e40a3b10 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9fc104c80 0x7fe9fc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9fc1030f0 0x7fe9fc198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 --2- 172.21.15.178:0/2896597702 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fe9fc1042e0 0x7fe9fc1993c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.225+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 >> 172.21.15.178:0/2896597702 conn(0x7fe9fc0fe620 msgr2=0x7fe9fc1077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.228+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 shutdown_connections 2023-10-16T14:40:59.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.228+0000 7fea037ff700 1 -- 172.21.15.178:0/2896597702 wait complete. 2023-10-16T14:40:59.276 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:58 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:40:59.281 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-10-16T14:40:59.281 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:40:59.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:58 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:40:59.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:58 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:40:59.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 -- 172.21.15.178:0/692387509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 msgr2=0x7fc9dc0732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.621 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/692387509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc0732e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009990 tx=0x7fc9cc009ca0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 -- 172.21.15.178:0/692387509 shutdown_connections 2023-10-16T14:40:59.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/692387509 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 0x7fc9dc10b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/692387509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc0732e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/692387509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 -- 172.21.15.178:0/692387509 >> 172.21.15.178:0/692387509 conn(0x7fc9dc0fc430 msgr2=0x7fc9dc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 -- 172.21.15.178:0/692387509 shutdown_connections 2023-10-16T14:40:59.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.618+0000 7fc9e27ff700 1 -- 172.21.15.178:0/692387509 wait complete. 2023-10-16T14:40:59.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.620+0000 7fc9e27ff700 1 Processor -- start 2023-10-16T14:40:59.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.620+0000 7fc9e27ff700 1 -- start start 2023-10-16T14:40:59.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.620+0000 7fc9e27ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e27ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e27ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 0x7fc9dc1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e27ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1a0880 con 0x7fc9dc074e90 2023-10-16T14:40:59.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e27ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1a09f0 con 0x7fc9dc108db0 2023-10-16T14:40:59.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e27ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc1a0bb0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:52988/0 (socket says 172.21.15.178:52988) 2023-10-16T14:40:59.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 -- 172.21.15.178:0/1415734595 learned_addr learned my addr 172.21.15.178:0/1415734595 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:40:59.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9e0d9c700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 0x7fc9dc1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9db7fe700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 -- 172.21.15.178:0/1415734595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 msgr2=0x7fc9dc1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 0x7fc9dc1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 -- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 msgr2=0x7fc9dc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 -- 172.21.15.178:0/1415734595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009520 con 0x7fc9dc0738b0 2023-10-16T14:40:59.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.621+0000 7fc9dbfff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00b680 tx=0x7fc9cc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.622+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc01d070 con 0x7fc9dc0738b0 2023-10-16T14:40:59.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.622+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9cc022470 con 0x7fc9dc0738b0 2023-10-16T14:40:59.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.622+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc017650 con 0x7fc9dc0738b0 2023-10-16T14:40:59.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.622+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0095f0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.622+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc1a10c0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.623+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc1074b0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.625+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fc9cc0225e0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.626+0000 7fc9d97fa700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 0x7fc9c40a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:40:59.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.626+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc9cc0ce4f0 con 0x7fc9dc0738b0 2023-10-16T14:40:59.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.627+0000 7fc9db7fe700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 0x7fc9c40a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:40:59.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.627+0000 7fc9db7fe700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 0x7fc9c40a3a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009df0 tx=0x7fc9d0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:40:59.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.630+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fc9cc097810 con 0x7fc9dc0738b0 2023-10-16T14:40:59.981 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.979+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9dc10e920 con 0x7fc9dc0738b0 2023-10-16T14:40:59.982 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.980+0000 7fc9d97fa700 1 -- 172.21.15.178:0/1415734595 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+11350 (secure 0 0 0) 0x7fc9cc027090 con 0x7fc9dc0738b0 2023-10-16T14:40:59.983 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:40:59.984 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:39:52.484857+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":128,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:40:59.987 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.985+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 msgr2=0x7fc9c40a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.985+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 0x7fc9c40a3a60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009df0 tx=0x7fc9d0009260 comp rx=0 tx=0).stop 2023-10-16T14:40:59.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.985+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 msgr2=0x7fc9dc198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:40:59.988 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.985+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00b680 tx=0x7fc9cc005740 comp rx=0 tx=0).stop 2023-10-16T14:40:59.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 shutdown_connections 2023-10-16T14:40:59.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fc9c40a15a0 0x7fc9c40a3a60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fc9dc108db0 0x7fc9dc1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9dc0738b0 0x7fc9dc198d40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 --2- 172.21.15.178:0/1415734595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9dc074e90 0x7fc9dc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:40:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 >> 172.21.15.178:0/1415734595 conn(0x7fc9dc0fc430 msgr2=0x7fc9dc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:40:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 shutdown_connections 2023-10-16T14:40:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:40:59.988+0000 7fc9e27ff700 1 -- 172.21.15.178:0/1415734595 wait complete. 2023-10-16T14:40:59.992 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 36 2023-10-16T14:41:00.028 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-10-16T14:41:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: pgmap v557: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 194 op/s 2023-10-16T14:41:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:40:59 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: pgmap v557: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 194 op/s 2023-10-16T14:41:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:40:59 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/1446342712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d641030f0 msgr2=0x7f8d64103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/1446342712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d641030f0 0x7f8d64103510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009990 tx=0x7f8d54009ca0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/1446342712 shutdown_connections 2023-10-16T14:41:00.333 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/1446342712 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d64104c80 0x7f8d6410b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/1446342712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d641030f0 0x7f8d64103510 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/1446342712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641042e0 0x7f8d64104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/1446342712 >> 172.21.15.178:0/1446342712 conn(0x7f8d640fe620 msgr2=0x7f8d64100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/1446342712 shutdown_connections 2023-10-16T14:41:00.334 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.330+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/1446342712 wait complete. 2023-10-16T14:41:00.336 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 Processor -- start 2023-10-16T14:41:00.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 -- start start 2023-10-16T14:41:00.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 0x7f8d64198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.337 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 0x7f8d641a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a09e0 con 0x7f8d641030f0 2023-10-16T14:41:00.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.331+0000 7f8d6a7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a0b20 con 0x7f8d641042e0 2023-10-16T14:41:00.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d6a7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d641a0cb0 con 0x7f8d64104c80 2023-10-16T14:41:00.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.338 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53110/0 (socket says 172.21.15.178:53110) 2023-10-16T14:41:00.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 -- 172.21.15.178:0/4204623127 learned_addr learned my addr 172.21.15.178:0/4204623127 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d63fff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 0x7f8d64198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 0x7f8d641a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 -- 172.21.15.178:0/4204623127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 msgr2=0x7f8d641a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.339 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 0x7f8d641a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 -- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 msgr2=0x7f8d64198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 0x7f8d64198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d637fe700 1 -- 172.21.15.178:0/4204623127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c009c40 con 0x7f8d641042e0 2023-10-16T14:41:00.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 0x7f8d641a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:41:00.340 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.332+0000 7f8d63fff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 0x7f8d64198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:41:00.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d637fe700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800e9a0 tx=0x7f8d5800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d617fa700 1 -- 172.21.15.178:0/4204623127 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58009790 con 0x7f8d641042e0 2023-10-16T14:41:00.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d617fa700 1 -- 172.21.15.178:0/4204623127 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5800cbd0 con 0x7f8d641042e0 2023-10-16T14:41:00.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d617fa700 1 -- 172.21.15.178:0/4204623127 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58018680 con 0x7f8d641042e0 2023-10-16T14:41:00.341 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540095f0 con 0x7f8d641042e0 2023-10-16T14:41:00.342 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.333+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d641a1230 con 0x7f8d641042e0 2023-10-16T14:41:00.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.337+0000 7f8d617fa700 1 -- 172.21.15.178:0/4204623127 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8d5800cd40 con 0x7f8d641042e0 2023-10-16T14:41:00.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.338+0000 7f8d7e20bb80 1 Processor -- start 2023-10-16T14:41:00.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.338+0000 7f8d617fa700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 0x7f8d4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.338+0000 7f8d7e20bb80 1 -- start start 2023-10-16T14:41:00.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d617fa700 1 -- 172.21.15.178:0/4204623127 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d58014070 con 0x7f8d641042e0 2023-10-16T14:41:00.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 0x7f8d4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb34ec70 con 0x55b8fb3db620 2023-10-16T14:41:00.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb333c60 con 0x55b8fb3d9e20 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb331580 con 0x55b8fb3da780 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53112/0 (socket says 172.21.15.178:53112) 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 -- 172.21.15.178:0/78768531 learned_addr learned my addr 172.21.15.178:0/78768531 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d637fe700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.347 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 -- 172.21.15.178:0/78768531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 msgr2=0x55b8fb3db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 -- 172.21.15.178:0/78768531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 msgr2=0x55b8fb3dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.339+0000 7f8d63fff700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d63fff700 1 -- 172.21.15.178:0/78768531 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8fb2ea3f0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:41:00.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d63fff700 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400b2e0 tx=0x7f8d54005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cd3ce29b7f42ee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 0x7f8d4c0a3940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55b8fb3dae00 tx=0x7f8d5c009920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/78768531 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5400beb0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/78768531 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5401e040 con 0x55b8fb3d9e20 2023-10-16T14:41:00.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.340+0000 7f8d4a7fc700 1 -- 172.21.15.178:0/78768531 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54004b70 con 0x55b8fb3d9e20 2023-10-16T14:41:00.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/78768531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 msgr2=0x55b8fb3da240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400b2e0 tx=0x7f8d54005c00 comp rx=0 tx=0).stop 2023-10-16T14:41:00.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/78768531 shutdown_connections 2023-10-16T14:41:00.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3da240 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/78768531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/78768531 >> 172.21.15.178:0/78768531 conn(0x55b8fb307dd0 msgr2=0x55b8fb3d4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/78768531 shutdown_connections 2023-10-16T14:41:00.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.341+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/78768531 wait complete. 2023-10-16T14:41:00.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.342+0000 7f8d7e20bb80 1 -- start start 2023-10-16T14:41:00.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.342+0000 7f8d7e20bb80 1 Processor -- start 2023-10-16T14:41:00.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb32ce70 con 0x55b8fb3db620 2023-10-16T14:41:00.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb2c9710 con 0x55b8fb3da780 2023-10-16T14:41:00.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb2c93b0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d7e20bb80 10 client.0 _ll_register_callbacks cb 0x55b8fb126800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:00.356 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53120/0 (socket says 172.21.15.178:53120) 2023-10-16T14:41:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d637fe700 1 -- 172.21.15.178:0/3996423425 learned_addr learned my addr 172.21.15.178:0/3996423425 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.343+0000 7f8d63fff700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.357 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 -- 172.21.15.178:0/3996423425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 msgr2=0x55b8fb3e3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 -- 172.21.15.178:0/3996423425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 msgr2=0x55b8fb3eb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3eb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 -- 172.21.15.178:0/3996423425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6410be80 con 0x55b8fb3da780 2023-10-16T14:41:00.358 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d637fe700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8d58096820 tx=0x7f8d5809a860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a131f2ad27b666c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:41:00.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d580a26b0 con 0x55b8fb3da780 2023-10-16T14:41:00.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8fb331ef0 con 0x55b8fb3da780 2023-10-16T14:41:00.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d5809e170 con 0x55b8fb3da780 2023-10-16T14:41:00.359 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d580a2220 con 0x55b8fb3da780 2023-10-16T14:41:00.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d7e20bb80 10 client.26288 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:00.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.344+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b8fb157de0 con 0x55b8fb3da780 2023-10-16T14:41:00.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8fb331ef0 con 0x55b8fb3da780 2023-10-16T14:41:00.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8d58065080 con 0x55b8fb3da780 2023-10-16T14:41:00.360 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 10 client.26288 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:00.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 10 client.26288 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:41:00.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 20 client.26288 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:00.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb3efa00 0x55b8fb3f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 4 client.26288 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:41:00.361 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8fb3a9c10 con 0x55b8fb3efa00 2023-10-16T14:41:00.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.345+0000 7f8d63fff700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb3efa00 0x55b8fb3f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.346+0000 7f8d437fe700 10 client.26288 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:41:00.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.346+0000 7f8d63fff700 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb3efa00 0x55b8fb3f1e20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b06d68b0fc3e506 server_cookie=1cb2c52ac6925e85 in_seq=0 out_seq=0 2023-10-16T14:41:00.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d437fe700 1 -- 172.21.15.178:0/3996423425 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b8fb3a9c10 con 0x55b8fb3efa00 2023-10-16T14:41:00.362 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d437fe700 10 client.26288 handle_command_reply: tid=0 2023-10-16T14:41:00.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 client.26288 shutdown 2023-10-16T14:41:00.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 msgr2=0x55b8fb3e4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.363 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8d58096820 tx=0x7f8d5809a860 comp rx=0 tx=0).stop 2023-10-16T14:41:00.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 shutdown_connections 2023-10-16T14:41:00.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb3efa00 0x55b8fb3f1e20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3eb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d437fe700 0 client.26288 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:41:00.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/3996423425 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3e4180 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 >> 172.21.15.178:0/3996423425 conn(0x55b8fb307dd0 msgr2=0x55b8fb3d4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.366 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 shutdown_connections 2023-10-16T14:41:00.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/3996423425 wait complete. 2023-10-16T14:41:00.367 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.347+0000 7f8d7e20bb80 20 client.26288 trim_cache size 0 max 16384 2023-10-16T14:41:00.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 Processor -- start 2023-10-16T14:41:00.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 -- start start 2023-10-16T14:41:00.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3dba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb3e3810 con 0x55b8fb3db620 2023-10-16T14:41:00.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb1ad920 con 0x55b8fb3da780 2023-10-16T14:41:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb3e2710 con 0x55b8fb3d9e20 2023-10-16T14:41:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d637fe700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53134/0 (socket says 172.21.15.178:53134) 2023-10-16T14:41:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d637fe700 1 -- 172.21.15.178:0/828211331 learned_addr learned my addr 172.21.15.178:0/828211331 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.371 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d63fff700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3dba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.363+0000 7f8d68d9c700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 -- 172.21.15.178:0/828211331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 msgr2=0x55b8fb3dba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3dba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.372 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 -- 172.21.15.178:0/828211331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 msgr2=0x55b8fb3d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 -- 172.21.15.178:0/828211331 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8fb140c60 con 0x55b8fb3da780 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d63fff700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d637fe700 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8d58096d40 tx=0x7f8d58099af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2243aed38a076a15 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d4affd700 1 -- 172.21.15.178:0/828211331 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5809f1b0 con 0x55b8fb3da780 2023-10-16T14:41:00.373 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d4affd700 1 -- 172.21.15.178:0/828211331 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d58062050 con 0x55b8fb3da780 2023-10-16T14:41:00.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.364+0000 7f8d4affd700 1 -- 172.21.15.178:0/828211331 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5809f3f0 con 0x55b8fb3da780 2023-10-16T14:41:00.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/828211331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 msgr2=0x55b8fb3eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8d58096d40 tx=0x7f8d58099af0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/828211331 shutdown_connections 2023-10-16T14:41:00.374 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3db620 0x55b8fb3d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3dba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/828211331 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3da780 0x55b8fb3eb180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/828211331 >> 172.21.15.178:0/828211331 conn(0x55b8fb307dd0 msgr2=0x55b8fb3ebab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/828211331 shutdown_connections 2023-10-16T14:41:00.375 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.365+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/828211331 wait complete. 2023-10-16T14:41:00.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.366+0000 7f8d7e20bb80 1 -- start start 2023-10-16T14:41:00.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.366+0000 7f8d7e20bb80 1 Processor -- start 2023-10-16T14:41:00.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.376 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 0x55b8fb3ea330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb3ea870 con 0x55b8fb3d9e20 2023-10-16T14:41:00.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb3ea9e0 con 0x55b8fb3db620 2023-10-16T14:41:00.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8fb3eaba0 con 0x55b8fb3da780 2023-10-16T14:41:00.377 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d7e20bb80 10 client.0 _ll_register_callbacks cb 0x7f8d58099d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:00.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d68d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 0x55b8fb3ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d63fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d68d9c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 0x55b8fb3ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53150/0 (socket says 172.21.15.178:53150) 2023-10-16T14:41:00.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d68d9c700 1 -- 172.21.15.178:0/314583763 learned_addr learned my addr 172.21.15.178:0/314583763 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.367+0000 7f8d637fe700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 -- 172.21.15.178:0/314583763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 msgr2=0x55b8fb3e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3e5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 -- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 msgr2=0x55b8fb3ea330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 0x55b8fb3ea330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.381 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 -- 172.21.15.178:0/314583763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8fb2ea3f0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d63fff700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e5300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d54005b40 tx=0x7f8d5400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1d8cc9739787f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d437fe700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:41:00.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54031350 con 0x55b8fb3d9e20 2023-10-16T14:41:00.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d5401e040 con 0x55b8fb3d9e20 2023-10-16T14:41:00.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8d540314b0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.368+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d54005040 con 0x55b8fb3d9e20 2023-10-16T14:41:00.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 10 client.16215 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:00.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55b8fb157de0 con 0x55b8fb3d9e20 2023-10-16T14:41:00.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5401e040 con 0x55b8fb3d9e20 2023-10-16T14:41:00.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7f8d54004c60 con 0x55b8fb3d9e20 2023-10-16T14:41:00.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 10 client.16215 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:00.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 10 client.16215 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2023-10-16T14:41:00.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 20 client.16215 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:00.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb455280 0x55b8fb4576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 4 client.16215 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 --> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b8fb325e50 con 0x55b8fb455280 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.369+0000 7f8d637fe700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb455280 0x55b8fb4576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.370+0000 7f8d637fe700 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb455280 0x55b8fb4576a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63d2eff1cbc5ac3 server_cookie=47ca1e14add27fa9 in_seq=0 out_seq=0 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.370+0000 7f8d437fe700 10 client.16215 ms_handle_connect on v2:172.21.15.186:6836/1524095342 2023-10-16T14:41:00.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d437fe700 1 -- 172.21.15.178:0/314583763 <== mds.2 v2:172.21.15.186:6836/1524095342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b8fb325e50 con 0x55b8fb455280 2023-10-16T14:41:00.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d437fe700 10 client.16215 handle_command_reply: tid=0 2023-10-16T14:41:00.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 client.16215 shutdown 2023-10-16T14:41:00.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 msgr2=0x55b8fb3e5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e5300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8d54005b40 tx=0x7f8d5400bd40 comp rx=0 tx=0).stop 2023-10-16T14:41:00.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 shutdown_connections 2023-10-16T14:41:00.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:6836/1524095342,v1:172.21.15.186:6837/1524095342] conn(0x55b8fb455280 0x55b8fb4576a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55b8fb3db620 0x55b8fb3ea330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55b8fb3d9e20 0x55b8fb3e5300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d437fe700 0 client.16215 ms_handle_reset on v2:172.21.15.186:6836/1524095342 2023-10-16T14:41:00.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 --2- 172.21.15.178:0/314583763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b8fb3da780 0x55b8fb3e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 >> 172.21.15.178:0/314583763 conn(0x55b8fb307dd0 msgr2=0x55b8fb3d4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 shutdown_connections 2023-10-16T14:41:00.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.371+0000 7f8d7e20bb80 1 -- 172.21.15.178:0/314583763 wait complete. 2023-10-16T14:41:00.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d7e20bb80 20 client.16215 trim_cache size 0 max 16384 2023-10-16T14:41:00.389 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:41:00.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 msgr2=0x7f8d4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 0x7f8d4c0a3940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55b8fb3dae00 tx=0x7f8d5c009920 comp rx=0 tx=0).stop 2023-10-16T14:41:00.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 msgr2=0x7f8d641993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800e9a0 tx=0x7f8d5800ed60 comp rx=0 tx=0).stop 2023-10-16T14:41:00.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 shutdown_connections 2023-10-16T14:41:00.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8d4c0a1480 0x7f8d4c0a3940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8d64104c80 0x7f8d641a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d641030f0 0x7f8d64198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 --2- 172.21.15.178:0/4204623127 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8d641042e0 0x7f8d641993a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 >> 172.21.15.178:0/4204623127 conn(0x7f8d640fe620 msgr2=0x7f8d64100970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 shutdown_connections 2023-10-16T14:41:00.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.372+0000 7f8d6a7ff700 1 -- 172.21.15.178:0/4204623127 wait complete. 2023-10-16T14:41:00.395 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-10-16T14:41:00.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: pgmap v557: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 194 op/s 2023-10-16T14:41:00.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:40:59 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:00.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 -- 172.21.15.178:0/2202660346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd181028a0 msgr2=0x7efd18109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2202660346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd181028a0 0x7efd18109150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efd10009fc0 tx=0x7efd1000b330 comp rx=0 tx=0).stop 2023-10-16T14:41:00.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 -- 172.21.15.178:0/2202660346 shutdown_connections 2023-10-16T14:41:00.610 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2202660346 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd181028a0 0x7efd18109150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2202660346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18100d10 0x7efd18101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2202660346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd18101f00 0x7efd18102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 -- 172.21.15.178:0/2202660346 >> 172.21.15.178:0/2202660346 conn(0x7efd180fc280 msgr2=0x7efd180fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 -- 172.21.15.178:0/2202660346 shutdown_connections 2023-10-16T14:41:00.611 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.607+0000 7efd1f19d700 1 -- 172.21.15.178:0/2202660346 wait complete. 2023-10-16T14:41:00.613 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 Processor -- start 2023-10-16T14:41:00.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 -- start start 2023-10-16T14:41:00.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 0x7efd180732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd181028a0 0x7efd180737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd18073f80 con 0x7efd18101f00 2023-10-16T14:41:00.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180740f0 con 0x7efd18100d10 2023-10-16T14:41:00.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1f19d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181a0a30 con 0x7efd181028a0 2023-10-16T14:41:00.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1e19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1e19b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53166/0 (socket says 172.21.15.178:53166) 2023-10-16T14:41:00.615 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1e19b700 1 -- 172.21.15.178:0/2440220271 learned_addr learned my addr 172.21.15.178:0/2440220271 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.608+0000 7efd1d99a700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 0x7efd180732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e99c700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd181028a0 0x7efd180737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 -- 172.21.15.178:0/2440220271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd181028a0 msgr2=0x7efd180737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd181028a0 0x7efd180737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 -- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 msgr2=0x7efd180732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.616 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 0x7efd180732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 -- 172.21.15.178:0/2440220271 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009520 con 0x7efd18100d10 2023-10-16T14:41:00.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1d99a700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 0x7efd180732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:00.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1e19b700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd1400ea50 tx=0x7efd1400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd0f7fe700 1 -- 172.21.15.178:0/2440220271 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400cbd0 con 0x7efd18100d10 2023-10-16T14:41:00.617 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd0f7fe700 1 -- 172.21.15.178:0/2440220271 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd1400cd30 con 0x7efd18100d10 2023-10-16T14:41:00.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd080095f0 con 0x7efd18100d10 2023-10-16T14:41:00.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.609+0000 7efd0f7fe700 1 -- 172.21.15.178:0/2440220271 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14018680 con 0x7efd18100d10 2023-10-16T14:41:00.618 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.610+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10009c40 con 0x7efd18100d10 2023-10-16T14:41:00.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.613+0000 7efd0f7fe700 1 -- 172.21.15.178:0/2440220271 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7efd14010430 con 0x7efd18100d10 2023-10-16T14:41:00.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.614+0000 7efd0f7fe700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 0x7efd040a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.614+0000 7efd33decb80 1 Processor -- start 2023-10-16T14:41:00.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.614+0000 7efd0f7fe700 1 -- 172.21.15.178:0/2440220271 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd14014070 con 0x7efd18100d10 2023-10-16T14:41:00.622 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.614+0000 7efd33decb80 1 -- start start 2023-10-16T14:41:00.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.614+0000 7efd1d99a700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 0x7efd040a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300db943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300db95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.623 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300db08c70 con 0x56300db94910 2023-10-16T14:41:00.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300daedc60 con 0x56300db95800 2023-10-16T14:41:00.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd33decb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300daeb580 con 0x56300db93fb0 2023-10-16T14:41:00.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53168/0 (socket says 172.21.15.178:53168) 2023-10-16T14:41:00.624 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 -- 172.21.15.178:0/2884486172 learned_addr learned my addr 172.21.15.178:0/2884486172 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1d99a700 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300db95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e19b700 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300db943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 -- 172.21.15.178:0/2884486172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 msgr2=0x56300db943d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300db943d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 -- 172.21.15.178:0/2884486172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 msgr2=0x56300db95220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.625 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300db95220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1d99a700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 0x7efd040a3a60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efd08005fd0 tx=0x7efd0800fa90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.615+0000 7efd1e99c700 1 -- 172.21.15.178:0/2884486172 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300daa43f0 con 0x56300db95800 2023-10-16T14:41:00.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd1e99c700 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd10009bb0 tx=0x7efd10009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4241bcc74a9bb115 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd0cbfe700 1 -- 172.21.15.178:0/2884486172 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000e040 con 0x56300db95800 2023-10-16T14:41:00.626 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd0cbfe700 1 -- 172.21.15.178:0/2884486172 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd100073e0 con 0x56300db95800 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd0cbfe700 1 -- 172.21.15.178:0/2884486172 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000bc50 con 0x56300db95800 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd33decb80 1 -- 172.21.15.178:0/2884486172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 msgr2=0x56300db95c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.616+0000 7efd33decb80 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd10009bb0 tx=0x7efd10009320 comp rx=0 tx=0).stop 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 -- 172.21.15.178:0/2884486172 shutdown_connections 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300db95c80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.627 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300db943d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 --2- 172.21.15.178:0/2884486172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300db95220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 -- 172.21.15.178:0/2884486172 >> 172.21.15.178:0/2884486172 conn(0x56300dac1dd0 msgr2=0x56300db90960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 -- 172.21.15.178:0/2884486172 shutdown_connections 2023-10-16T14:41:00.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 -- 172.21.15.178:0/2884486172 wait complete. 2023-10-16T14:41:00.628 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 -- start start 2023-10-16T14:41:00.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.617+0000 7efd33decb80 1 Processor -- start 2023-10-16T14:41:00.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db9ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.629 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300dae6e70 con 0x56300db93fb0 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300da83710 con 0x56300db94910 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300da833b0 con 0x56300db95800 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd33decb80 10 client.0 _ll_register_callbacks cb 0x56300d8e0800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd1d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd1d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53178/0 (socket says 172.21.15.178:53178) 2023-10-16T14:41:00.630 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd1d99a700 1 -- 172.21.15.178:0/3924625370 learned_addr learned my addr 172.21.15.178:0/3924625370 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd1e19b700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db9ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.618+0000 7efd1e99c700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 -- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 msgr2=0x56300dba5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.631 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 -- 172.21.15.178:0/3924625370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 msgr2=0x56300db9ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db9ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 -- 172.21.15.178:0/3924625370 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd18109aa0 con 0x56300db94910 2023-10-16T14:41:00.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1e19b700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:41:00.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd1d99a700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd08011c10 tx=0x7efd080047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=207e09ba6369a264 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.632 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:41:00.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0801ec40 con 0x56300db94910 2023-10-16T14:41:00.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56300daebef0 con 0x56300db94910 2023-10-16T14:41:00.633 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.619+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd0801eda0 con 0x56300db94910 2023-10-16T14:41:00.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd0802d5b0 con 0x56300db94910 2023-10-16T14:41:00.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 10 client.26306 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56300d911de0 con 0x56300db94910 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56300daebef0 con 0x56300db94910 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7efd08026590 con 0x56300db94910 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 10 client.26306 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 10 client.26306 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:41:00.636 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 20 client.26306 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:00.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dba9ce0 0x56300dbac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 4 client.26306 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:41:00.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56300db63c10 con 0x56300dba9ce0 2023-10-16T14:41:00.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.620+0000 7efd1e19b700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dba9ce0 0x56300dbac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.637 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.621+0000 7efd01ffb700 10 client.26306 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:41:00.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.621+0000 7efd1e19b700 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dba9ce0 0x56300dbac100 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e2256e36866482 server_cookie=bb87464a6e153e95 in_seq=0 out_seq=0 2023-10-16T14:41:00.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.621+0000 7efd01ffb700 1 -- 172.21.15.178:0/3924625370 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56300db63c10 con 0x56300dba9ce0 2023-10-16T14:41:00.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.621+0000 7efd01ffb700 10 client.26306 handle_command_reply: tid=0 2023-10-16T14:41:00.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 client.26306 shutdown 2023-10-16T14:41:00.638 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 msgr2=0x56300db9e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd08011c10 tx=0x7efd080047c0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 shutdown_connections 2023-10-16T14:41:00.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dba9ce0 0x56300dbac100 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db9ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd01ffb700 0 client.26306 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 --2- 172.21.15.178:0/3924625370 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300db9e330 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 >> 172.21.15.178:0/3924625370 conn(0x56300dac1dd0 msgr2=0x56300db8ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 shutdown_connections 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 1 -- 172.21.15.178:0/3924625370 wait complete. 2023-10-16T14:41:00.640 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.622+0000 7efd33decb80 20 client.26306 trim_cache size 0 max 16384 2023-10-16T14:41:00.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.639+0000 7efd33decb80 1 Processor -- start 2023-10-16T14:41:00.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 -- start start 2023-10-16T14:41:00.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db95be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300d967920 con 0x56300db93fb0 2023-10-16T14:41:00.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300db95010 con 0x56300db94910 2023-10-16T14:41:00.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd33decb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300db9b960 con 0x56300db95800 2023-10-16T14:41:00.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1e19b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db95be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53180/0 (socket says 172.21.15.178:53180) 2023-10-16T14:41:00.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 -- 172.21.15.178:0/1894065433 learned_addr learned my addr 172.21.15.178:0/1894065433 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1e99c700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 -- 172.21.15.178:0/1894065433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 msgr2=0x56300dba5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 -- 172.21.15.178:0/1894065433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 msgr2=0x56300db95be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db95be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1d99a700 1 -- 172.21.15.178:0/1894065433 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56300d8fac60 con 0x56300db94910 2023-10-16T14:41:00.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1e99c700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:00.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.640+0000 7efd1e19b700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:00.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.641+0000 7efd1d99a700 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efd0801e7b0 tx=0x7efd0800fe90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7bb0d13f3c66125 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.641+0000 7efd0d3ff700 1 -- 172.21.15.178:0/1894065433 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08026d10 con 0x56300db94910 2023-10-16T14:41:00.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.641+0000 7efd0d3ff700 1 -- 172.21.15.178:0/1894065433 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd0801ea40 con 0x56300db94910 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.641+0000 7efd0d3ff700 1 -- 172.21.15.178:0/1894065433 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08036a70 con 0x56300db94910 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- 172.21.15.178:0/1894065433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 msgr2=0x56300dba5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efd0801e7b0 tx=0x7efd0800fe90 comp rx=0 tx=0).stop 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- 172.21.15.178:0/1894065433 shutdown_connections 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db95800 0x56300dba5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db93fb0 0x56300db95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 --2- 172.21.15.178:0/1894065433 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db94910 0x56300dba5290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- 172.21.15.178:0/1894065433 >> 172.21.15.178:0/1894065433 conn(0x56300dac1dd0 msgr2=0x56300db9cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- 172.21.15.178:0/1894065433 shutdown_connections 2023-10-16T14:41:00.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- 172.21.15.178:0/1894065433 wait complete. 2023-10-16T14:41:00.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 -- start start 2023-10-16T14:41:00.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.642+0000 7efd33decb80 1 Processor -- start 2023-10-16T14:41:00.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300dba4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300dba34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.651 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300dba3f70 con 0x56300db94910 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300dba40e0 con 0x56300db95800 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x56300dba42a0 con 0x56300db93fb0 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd33decb80 10 client.0 _ll_register_callbacks cb 0x7efd08011a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1d99a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300dba34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.652 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53192/0 (socket says 172.21.15.178:53192) 2023-10-16T14:41:00.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 -- 172.21.15.178:0/535216225 learned_addr learned my addr 172.21.15.178:0/535216225 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:00.653 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e19b700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300dba4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.655 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 -- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 msgr2=0x56300dba4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300dba4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 -- 172.21.15.178:0/535216225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 msgr2=0x56300dba34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:00.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300dba34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.643+0000 7efd1e99c700 1 -- 172.21.15.178:0/535216225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10009c40 con 0x56300db95800 2023-10-16T14:41:00.656 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd1e99c700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efd1000b370 tx=0x7efd10003f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdd8fef80b35ff53 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:00.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:41:00.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10007ea0 con 0x56300db95800 2023-10-16T14:41:00.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd1000b9f0 con 0x56300db95800 2023-10-16T14:41:00.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd100073e0 con 0x56300db95800 2023-10-16T14:41:00.657 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd1000e040 con 0x56300db95800 2023-10-16T14:41:00.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd33decb80 10 client.26312 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:00.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56300daa43f0 con 0x56300db95800 2023-10-16T14:41:00.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.644+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd100073e0 con 0x56300db95800 2023-10-16T14:41:00.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7efd10019070 con 0x56300db95800 2023-10-16T14:41:00.658 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 10 client.26312 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:00.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 10 client.26312 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-10-16T14:41:00.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 20 client.26312 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:00.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dbfef50 0x56300dc01370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:00.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 4 client.26312 mds_command: new command op to 24550 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:41:00.659 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 --> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56300dadfe50 con 0x56300dbfef50 2023-10-16T14:41:00.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd1e19b700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dbfef50 0x56300dc01370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:00.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd1e19b700 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dbfef50 0x56300dc01370 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd9f9a67fc29b05a server_cookie=e0e305532d5cbbce in_seq=0 out_seq=0 2023-10-16T14:41:00.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.645+0000 7efd01ffb700 10 client.26312 ms_handle_connect on v2:172.21.15.187:6838/397706812 2023-10-16T14:41:00.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.647+0000 7efd01ffb700 1 -- 172.21.15.178:0/535216225 <== mds.1 v2:172.21.15.187:6838/397706812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+92550 (crc 0 0 0) 0x56300dadfe50 con 0x56300dbfef50 2023-10-16T14:41:00.660 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.647+0000 7efd01ffb700 10 client.26312 handle_command_reply: tid=0 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 client.26312 shutdown 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 msgr2=0x56300dba3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efd1000b370 tx=0x7efd10003f70 comp rx=0 tx=0).stop 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 shutdown_connections 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:6838/397706812,v1:172.21.15.187:6839/397706812] conn(0x56300dbfef50 0x56300dc01370 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.661 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x56300db95800 0x56300dba3a30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56300db93fb0 0x56300dba4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 --2- 172.21.15.178:0/535216225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x56300db94910 0x56300dba34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:00.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd01ffb700 0 client.26312 ms_handle_reset on v2:172.21.15.187:6838/397706812 2023-10-16T14:41:00.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 >> 172.21.15.178:0/535216225 conn(0x56300dac1dd0 msgr2=0x56300db90540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:00.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 shutdown_connections 2023-10-16T14:41:00.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 1 -- 172.21.15.178:0/535216225 wait complete. 2023-10-16T14:41:00.663 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.648+0000 7efd33decb80 20 client.26312 trim_cache size 0 max 16384 2023-10-16T14:41:00.664 INFO:teuthology.orchestra.run.smithi178.stdout:[ 2023-10-16T14:41:00.664 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.665 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.665 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2319878, 2023-10-16T14:41:00.665 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629966, 2023-10-16T14:41:00.665 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml" 2023-10-16T14:41:00.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.666 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.666 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.666 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 7835518, 2023-10-16T14:41:00.666 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629345, 2023-10-16T14:41:00.666 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg" 2023-10-16T14:41:00.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.667 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.667 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.667 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 12837385, 2023-10-16T14:41:00.667 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629672, 2023-10-16T14:41:00.668 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml" 2023-10-16T14:41:00.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.668 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.668 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.668 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24322276, 2023-10-16T14:41:00.669 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629241, 2023-10-16T14:41:00.669 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg" 2023-10-16T14:41:00.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.669 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.670 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.670 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 26658060, 2023-10-16T14:41:00.670 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629998, 2023-10-16T14:41:00.670 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml" 2023-10-16T14:41:00.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 67181017, 2023-10-16T14:41:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629304, 2023-10-16T14:41:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg" 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 70917158, 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629992, 2023-10-16T14:41:00.672 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml" 2023-10-16T14:41:00.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.673 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.673 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.673 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 81342961, 2023-10-16T14:41:00.673 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630309, 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg" 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 94428883, 2023-10-16T14:41:00.674 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511632676, 2023-10-16T14:41:00.675 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg" 2023-10-16T14:41:00.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.675 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.675 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 106156555, 2023-10-16T14:41:00.676 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629150, 2023-10-16T14:41:00.676 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml" 2023-10-16T14:41:00.676 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.676 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.676 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 126737440, 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630236, 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg" 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.677 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.678 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 143231450, 2023-10-16T14:41:00.678 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635443, 2023-10-16T14:41:00.678 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg" 2023-10-16T14:41:00.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.678 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 144891685, 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635322, 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml" 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.679 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.680 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.680 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 154245988, 2023-10-16T14:41:00.680 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630192, 2023-10-16T14:41:00.680 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg" 2023-10-16T14:41:00.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 170618326, 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635331, 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml" 2023-10-16T14:41:00.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.682 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.682 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.682 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 193400914, 2023-10-16T14:41:00.682 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635229, 2023-10-16T14:41:00.682 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml" 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 205955252, 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629158, 2023-10-16T14:41:00.683 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml" 2023-10-16T14:41:00.684 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.684 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.684 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.684 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 208984527, 2023-10-16T14:41:00.684 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635472, 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg" 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 223304131, 2023-10-16T14:41:00.685 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629744, 2023-10-16T14:41:00.686 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml" 2023-10-16T14:41:00.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.686 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.686 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.686 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 243091576, 2023-10-16T14:41:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630245, 2023-10-16T14:41:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg" 2023-10-16T14:41:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.687 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 253623843, 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629609, 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10" 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.688 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.689 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 268762640, 2023-10-16T14:41:00.689 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629364, 2023-10-16T14:41:00.689 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg" 2023-10-16T14:41:00.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.690 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.690 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 284056542, 2023-10-16T14:41:00.690 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635337, 2023-10-16T14:41:00.690 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml" 2023-10-16T14:41:00.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 284547879, 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629181, 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml" 2023-10-16T14:41:00.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 288645309, 2023-10-16T14:41:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635457, 2023-10-16T14:41:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg" 2023-10-16T14:41:00.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.693 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.693 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.693 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 291924694, 2023-10-16T14:41:00.693 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635282, 2023-10-16T14:41:00.694 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml" 2023-10-16T14:41:00.694 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.694 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.694 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.694 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 296312784, 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629836, 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml" 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.695 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 301713849, 2023-10-16T14:41:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635375, 2023-10-16T14:41:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg" 2023-10-16T14:41:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.696 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 310851966, 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629349, 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg" 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.697 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.698 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 320244501, 2023-10-16T14:41:00.698 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635197, 2023-10-16T14:41:00.698 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml" 2023-10-16T14:41:00.698 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.698 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 334318390, 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629273, 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg" 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.699 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 346425679, 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635451, 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg" 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.700 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.701 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.701 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 349497018, 2023-10-16T14:41:00.701 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629254, 2023-10-16T14:41:00.701 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg" 2023-10-16T14:41:00.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.702 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.702 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.702 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 351050429, 2023-10-16T14:41:00.702 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629168, 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml" 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 352636356, 2023-10-16T14:41:00.703 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635605, 2023-10-16T14:41:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg" 2023-10-16T14:41:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.704 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 363016705, 2023-10-16T14:41:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635548, 2023-10-16T14:41:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg" 2023-10-16T14:41:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.705 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 363171402, 2023-10-16T14:41:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635563, 2023-10-16T14:41:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg" 2023-10-16T14:41:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.706 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.707 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.707 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 363691499, 2023-10-16T14:41:00.707 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635518, 2023-10-16T14:41:00.707 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg" 2023-10-16T14:41:00.707 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 372422872, 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635205, 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml" 2023-10-16T14:41:00.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.709 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.709 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.709 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 375625430, 2023-10-16T14:41:00.709 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635557, 2023-10-16T14:41:00.709 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg" 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 384543122, 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629327, 2023-10-16T14:41:00.710 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg" 2023-10-16T14:41:00.711 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.711 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.711 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.711 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 402109119, 2023-10-16T14:41:00.711 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629283, 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg" 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 436411263, 2023-10-16T14:41:00.712 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635189, 2023-10-16T14:41:00.713 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml" 2023-10-16T14:41:00.713 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.713 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.713 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.713 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 437040640, 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630298, 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg" 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.714 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 474633632, 2023-10-16T14:41:00.715 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629396, 2023-10-16T14:41:00.715 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg" 2023-10-16T14:41:00.715 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.715 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.715 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.716 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 476136294, 2023-10-16T14:41:00.716 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629282, 2023-10-16T14:41:00.716 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg" 2023-10-16T14:41:00.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.716 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 478921388, 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629687, 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml" 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.717 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 479002747, 2023-10-16T14:41:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629384, 2023-10-16T14:41:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg" 2023-10-16T14:41:00.718 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 480336587, 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629155, 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml" 2023-10-16T14:41:00.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 485562124, 2023-10-16T14:41:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631649, 2023-10-16T14:41:00.720 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml" 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 489044723, 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629280, 2023-10-16T14:41:00.721 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg" 2023-10-16T14:41:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 490146238, 2023-10-16T14:41:00.722 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629337, 2023-10-16T14:41:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg" 2023-10-16T14:41:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.723 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 503436537, 2023-10-16T14:41:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635233, 2023-10-16T14:41:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml" 2023-10-16T14:41:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.724 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 507358548, 2023-10-16T14:41:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629682, 2023-10-16T14:41:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml" 2023-10-16T14:41:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.725 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 508942911, 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629342, 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg" 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.726 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 519797832, 2023-10-16T14:41:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635203, 2023-10-16T14:41:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml" 2023-10-16T14:41:00.727 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 541904875, 2023-10-16T14:41:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629950, 2023-10-16T14:41:00.728 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml" 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 572675621, 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629300, 2023-10-16T14:41:00.729 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg" 2023-10-16T14:41:00.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.730 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.730 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.730 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 589455213, 2023-10-16T14:41:00.730 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629927, 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml" 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 594668103, 2023-10-16T14:41:00.731 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635554, 2023-10-16T14:41:00.732 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg" 2023-10-16T14:41:00.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.732 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.732 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.732 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 654339824, 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511632683, 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml" 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.733 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 687501352, 2023-10-16T14:41:00.734 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629382, 2023-10-16T14:41:00.734 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg" 2023-10-16T14:41:00.734 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.734 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.734 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 688171571, 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629618, 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml" 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.735 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.736 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 700224530, 2023-10-16T14:41:00.736 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629357, 2023-10-16T14:41:00.736 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg" 2023-10-16T14:41:00.736 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.736 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 719513445, 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630321, 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg" 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.737 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.738 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.738 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 724650248, 2023-10-16T14:41:00.738 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630174, 2023-10-16T14:41:00.738 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg" 2023-10-16T14:41:00.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.739 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.739 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.739 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 741642874, 2023-10-16T14:41:00.739 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635490, 2023-10-16T14:41:00.739 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg" 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 742397383, 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631420, 2023-10-16T14:41:00.740 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg" 2023-10-16T14:41:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 751093847, 2023-10-16T14:41:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629354, 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg" 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 752193643, 2023-10-16T14:41:00.742 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635536, 2023-10-16T14:41:00.743 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg" 2023-10-16T14:41:00.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.743 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.743 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.743 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 753619016, 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629301, 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg" 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.744 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 763431410, 2023-10-16T14:41:00.745 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635445, 2023-10-16T14:41:00.745 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg" 2023-10-16T14:41:00.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.745 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.745 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 793828138, 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635392, 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg" 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.746 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.747 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 802759863, 2023-10-16T14:41:00.747 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635349, 2023-10-16T14:41:00.747 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml" 2023-10-16T14:41:00.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.747 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 818386344, 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635610, 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg" 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.748 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.749 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.749 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 822276294, 2023-10-16T14:41:00.749 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635211, 2023-10-16T14:41:00.749 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml" 2023-10-16T14:41:00.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 830767948, 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635509, 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg" 2023-10-16T14:41:00.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.751 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.751 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.751 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 839142147, 2023-10-16T14:41:00.751 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629248, 2023-10-16T14:41:00.751 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg" 2023-10-16T14:41:00.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.752 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.752 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 845036951, 2023-10-16T14:41:00.752 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629306, 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg" 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 891090259, 2023-10-16T14:41:00.753 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629724, 2023-10-16T14:41:00.754 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml" 2023-10-16T14:41:00.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.754 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.754 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 896413593, 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629315, 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg" 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.755 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 897745584, 2023-10-16T14:41:00.756 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635484, 2023-10-16T14:41:00.756 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg" 2023-10-16T14:41:00.756 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.756 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.756 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 902214943, 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629376, 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg" 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.757 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.758 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 916737967, 2023-10-16T14:41:00.758 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635276, 2023-10-16T14:41:00.758 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml" 2023-10-16T14:41:00.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 937207320, 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635454, 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg" 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.759 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.760 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.760 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 958645876, 2023-10-16T14:41:00.760 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635251, 2023-10-16T14:41:00.760 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml" 2023-10-16T14:41:00.760 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 963626449, 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629352, 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg" 2023-10-16T14:41:00.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.762 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.762 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.762 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 973774420, 2023-10-16T14:41:00.762 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630013, 2023-10-16T14:41:00.762 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml" 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 975896361, 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635361, 2023-10-16T14:41:00.763 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg" 2023-10-16T14:41:00.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.764 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.764 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.764 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 986216494, 2023-10-16T14:41:00.764 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629988, 2023-10-16T14:41:00.765 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml" 2023-10-16T14:41:00.765 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.765 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.765 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.765 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 994190333, 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629676, 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml" 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.766 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1013808059, 2023-10-16T14:41:00.767 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629340, 2023-10-16T14:41:00.767 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg" 2023-10-16T14:41:00.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.767 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1021146996, 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635566, 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg" 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.768 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1030593308, 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635316, 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml" 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.769 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.770 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1033281165, 2023-10-16T14:41:00.770 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635560, 2023-10-16T14:41:00.770 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg" 2023-10-16T14:41:00.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.770 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.771 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.771 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1033595993, 2023-10-16T14:41:00.773 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629195, 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml" 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1049544777, 2023-10-16T14:41:00.774 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629846, 2023-10-16T14:41:00.775 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml" 2023-10-16T14:41:00.775 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.775 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.775 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.775 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1052431883, 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635325, 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml" 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.776 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1053388024, 2023-10-16T14:41:00.777 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635298, 2023-10-16T14:41:00.777 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml" 2023-10-16T14:41:00.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.777 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.778 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1066548262, 2023-10-16T14:41:00.778 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629831, 2023-10-16T14:41:00.778 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml" 2023-10-16T14:41:00.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.778 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1070415811, 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629211, 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml" 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.779 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.780 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.780 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1072550094, 2023-10-16T14:41:00.780 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635247, 2023-10-16T14:41:00.780 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml" 2023-10-16T14:41:00.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1079212577, 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635340, 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml" 2023-10-16T14:41:00.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.782 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.782 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1093558130, 2023-10-16T14:41:00.782 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629330, 2023-10-16T14:41:00.782 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg" 2023-10-16T14:41:00.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.783 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.783 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.783 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1100129957, 2023-10-16T14:41:00.783 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635447, 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg" 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1127703539, 2023-10-16T14:41:00.784 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629328, 2023-10-16T14:41:00.785 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg" 2023-10-16T14:41:00.785 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.785 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.785 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.785 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1142242836, 2023-10-16T14:41:00.786 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635201, 2023-10-16T14:41:00.786 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml" 2023-10-16T14:41:00.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.786 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.786 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1147535770, 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629325, 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg" 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.787 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.788 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1155749113, 2023-10-16T14:41:00.788 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630423, 2023-10-16T14:41:00.788 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg" 2023-10-16T14:41:00.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.788 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1191330196, 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629645, 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml" 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.790 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.790 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1191811930, 2023-10-16T14:41:00.790 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635328, 2023-10-16T14:41:00.790 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml" 2023-10-16T14:41:00.790 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1197421137, 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630255, 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg" 2023-10-16T14:41:00.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.792 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.792 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1215889868, 2023-10-16T14:41:00.792 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635304, 2023-10-16T14:41:00.792 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml" 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1216029935, 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635319, 2023-10-16T14:41:00.793 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml" 2023-10-16T14:41:00.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.794 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.794 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.794 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1217773918, 2023-10-16T14:41:00.794 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631448, 2023-10-16T14:41:00.795 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml" 2023-10-16T14:41:00.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.795 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.795 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.795 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1217983230, 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629175, 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml" 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1223770740, 2023-10-16T14:41:00.797 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630183, 2023-10-16T14:41:00.797 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg" 2023-10-16T14:41:00.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.797 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.797 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1285928666, 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629336, 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg" 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.798 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.799 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1301014493, 2023-10-16T14:41:00.799 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629339, 2023-10-16T14:41:00.799 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg" 2023-10-16T14:41:00.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.799 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.800 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.800 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1311510592, 2023-10-16T14:41:00.800 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629360, 2023-10-16T14:41:00.800 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg" 2023-10-16T14:41:00.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1324731097, 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630347, 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg" 2023-10-16T14:41:00.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1345359408, 2023-10-16T14:41:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635503, 2023-10-16T14:41:00.802 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg" 2023-10-16T14:41:00.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.803 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.803 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.803 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1358995717, 2023-10-16T14:41:00.803 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635235, 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml" 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1364293234, 2023-10-16T14:41:00.804 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635267, 2023-10-16T14:41:00.805 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml" 2023-10-16T14:41:00.805 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.805 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.805 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.805 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1366620370, 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629372, 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg" 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.806 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1370714045, 2023-10-16T14:41:00.807 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635466, 2023-10-16T14:41:00.807 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg" 2023-10-16T14:41:00.807 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.807 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.807 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1372951927, 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630210, 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg" 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.808 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.809 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1383943694, 2023-10-16T14:41:00.809 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635231, 2023-10-16T14:41:00.809 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml" 2023-10-16T14:41:00.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.809 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1400761585, 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635481, 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg" 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.810 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.811 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.811 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1407239949, 2023-10-16T14:41:00.811 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635524, 2023-10-16T14:41:00.811 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg" 2023-10-16T14:41:00.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.812 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.812 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1407311415, 2023-10-16T14:41:00.812 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630132, 2023-10-16T14:41:00.812 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml" 2023-10-16T14:41:00.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.813 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.813 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.813 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1415308888, 2023-10-16T14:41:00.813 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635551, 2023-10-16T14:41:00.814 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg" 2023-10-16T14:41:00.814 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.814 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.814 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.814 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1436979294, 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629713, 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml" 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.815 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1443678436, 2023-10-16T14:41:00.816 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635307, 2023-10-16T14:41:00.816 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml" 2023-10-16T14:41:00.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.816 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.816 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1453668845, 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629851, 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml" 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.817 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.818 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1458083545, 2023-10-16T14:41:00.818 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635213, 2023-10-16T14:41:00.818 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml" 2023-10-16T14:41:00.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.818 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.819 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.819 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1473766368, 2023-10-16T14:41:00.819 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635402, 2023-10-16T14:41:00.819 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg" 2023-10-16T14:41:00.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1480041060, 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629214, 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml" 2023-10-16T14:41:00.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.821 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.821 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.821 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1485488476, 2023-10-16T14:41:00.821 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629269, 2023-10-16T14:41:00.821 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg" 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1504828445, 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631426, 2023-10-16T14:41:00.822 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml" 2023-10-16T14:41:00.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.823 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.823 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.823 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1510093970, 2023-10-16T14:41:00.823 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635227, 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml" 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1526118002, 2023-10-16T14:41:00.824 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629974, 2023-10-16T14:41:00.825 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml" 2023-10-16T14:41:00.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.825 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.825 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1533932531, 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629279, 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg" 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.826 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1541047422, 2023-10-16T14:41:00.827 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629641, 2023-10-16T14:41:00.827 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml" 2023-10-16T14:41:00.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.827 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.827 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1568316844, 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629171, 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml" 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.828 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.829 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1571657399, 2023-10-16T14:41:00.829 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635310, 2023-10-16T14:41:00.829 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml" 2023-10-16T14:41:00.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.829 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1577371123, 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635539, 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg" 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.830 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.831 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.831 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1577606096, 2023-10-16T14:41:00.831 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635475, 2023-10-16T14:41:00.831 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg" 2023-10-16T14:41:00.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.832 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.832 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.832 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1580329445, 2023-10-16T14:41:00.832 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635530, 2023-10-16T14:41:00.832 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg" 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1600343293, 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635355, 2023-10-16T14:41:00.833 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg" 2023-10-16T14:41:00.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.834 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.834 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.834 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1615809173, 2023-10-16T14:41:00.834 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629806, 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml" 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1639593969, 2023-10-16T14:41:00.835 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629386, 2023-10-16T14:41:00.836 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg" 2023-10-16T14:41:00.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.836 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.836 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.836 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1654479996, 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629946, 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml" 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.837 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1656789341, 2023-10-16T14:41:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630127, 2023-10-16T14:41:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml" 2023-10-16T14:41:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.839 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1660083827, 2023-10-16T14:41:00.839 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630156, 2023-10-16T14:41:00.839 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg" 2023-10-16T14:41:00.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.839 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1665227828, 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629796, 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml" 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.840 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.841 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.841 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1680303391, 2023-10-16T14:41:00.841 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629811, 2023-10-16T14:41:00.841 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml" 2023-10-16T14:41:00.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.842 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.842 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.842 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1705658478, 2023-10-16T14:41:00.842 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635616, 2023-10-16T14:41:00.842 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg" 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1725948260, 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630118, 2023-10-16T14:41:00.843 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml" 2023-10-16T14:41:00.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.844 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.844 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.844 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1738371210, 2023-10-16T14:41:00.844 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635602, 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg" 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1743311664, 2023-10-16T14:41:00.845 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635581, 2023-10-16T14:41:00.846 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg" 2023-10-16T14:41:00.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.846 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.846 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.846 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1749410692, 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630437, 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg" 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.847 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1763589745, 2023-10-16T14:41:00.848 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635193, 2023-10-16T14:41:00.848 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml" 2023-10-16T14:41:00.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.848 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.848 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1770255683, 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635493, 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg" 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.849 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.850 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1782454674, 2023-10-16T14:41:00.850 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629612, 2023-10-16T14:41:00.850 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml" 2023-10-16T14:41:00.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.850 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1789026672, 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631501, 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg" 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.851 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.852 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.852 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1795329898, 2023-10-16T14:41:00.852 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629876, 2023-10-16T14:41:00.852 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml" 2023-10-16T14:41:00.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.853 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.853 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.853 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1795981212, 2023-10-16T14:41:00.853 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629891, 2023-10-16T14:41:00.853 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml" 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1800050328, 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629295, 2023-10-16T14:41:00.854 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg" 2023-10-16T14:41:00.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.855 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.855 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.855 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1807653151, 2023-10-16T14:41:00.855 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635487, 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg" 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1807945322, 2023-10-16T14:41:00.856 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629363, 2023-10-16T14:41:00.857 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg" 2023-10-16T14:41:00.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.857 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.857 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.857 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1822065368, 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635500, 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg" 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.858 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1824838163, 2023-10-16T14:41:00.859 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629374, 2023-10-16T14:41:00.859 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg" 2023-10-16T14:41:00.859 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.859 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1848544959, 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630287, 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg" 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.860 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.861 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1856188820, 2023-10-16T14:41:00.861 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635334, 2023-10-16T14:41:00.861 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml" 2023-10-16T14:41:00.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1857569714, 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635569, 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg" 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.862 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.863 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.863 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1875746856, 2023-10-16T14:41:00.863 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635410, 2023-10-16T14:41:00.863 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg" 2023-10-16T14:41:00.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1895640951, 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629400, 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg" 2023-10-16T14:41:00.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.865 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.865 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.865 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1896143509, 2023-10-16T14:41:00.865 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629355, 2023-10-16T14:41:00.865 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg" 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1899228795, 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629232, 2023-10-16T14:41:00.866 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml" 2023-10-16T14:41:00.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.867 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.867 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1908593229, 2023-10-16T14:41:00.867 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629774, 2023-10-16T14:41:00.868 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml" 2023-10-16T14:41:00.868 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.868 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.868 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.868 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1909686973, 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629662, 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml" 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.869 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1912144191, 2023-10-16T14:41:00.870 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635459, 2023-10-16T14:41:00.870 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg" 2023-10-16T14:41:00.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.870 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.870 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1917575468, 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629866, 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml" 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.871 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1918751011, 2023-10-16T14:41:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630265, 2023-10-16T14:41:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg" 2023-10-16T14:41:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.873 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.873 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1929537657, 2023-10-16T14:41:00.873 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635389, 2023-10-16T14:41:00.873 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg" 2023-10-16T14:41:00.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1930520988, 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631401, 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml" 2023-10-16T14:41:00.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.875 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.875 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.875 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1942040613, 2023-10-16T14:41:00.875 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629236, 2023-10-16T14:41:00.875 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg" 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1966896137, 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635386, 2023-10-16T14:41:00.876 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg" 2023-10-16T14:41:00.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.877 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.877 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.877 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1970224640, 2023-10-16T14:41:00.877 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635512, 2023-10-16T14:41:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg" 2023-10-16T14:41:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1976562987, 2023-10-16T14:41:00.879 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629368, 2023-10-16T14:41:00.879 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg" 2023-10-16T14:41:00.879 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.879 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.879 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.880 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 1994143875, 2023-10-16T14:41:00.880 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630357, 2023-10-16T14:41:00.880 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg" 2023-10-16T14:41:00.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2009233159, 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629331, 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg" 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.882 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.882 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2020224319, 2023-10-16T14:41:00.882 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629366, 2023-10-16T14:41:00.882 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg" 2023-10-16T14:41:00.882 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2033267070, 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629392, 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg" 2023-10-16T14:41:00.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2038665354, 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629388, 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg" 2023-10-16T14:41:00.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.885 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.885 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.885 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2042945960, 2023-10-16T14:41:00.885 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629277, 2023-10-16T14:41:00.885 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg" 2023-10-16T14:41:00.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.886 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.886 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.886 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2050527967, 2023-10-16T14:41:00.886 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635497, 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg" 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2058716029, 2023-10-16T14:41:00.887 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629789, 2023-10-16T14:41:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml" 2023-10-16T14:41:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.888 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2058958732, 2023-10-16T14:41:00.889 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631381, 2023-10-16T14:41:00.889 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml" 2023-10-16T14:41:00.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.889 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.889 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2107897848, 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629192, 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml" 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.890 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.891 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2112067697, 2023-10-16T14:41:00.891 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635607, 2023-10-16T14:41:00.891 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg" 2023-10-16T14:41:00.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.891 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2128205912, 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629380, 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg" 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.892 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.893 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.893 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2134478819, 2023-10-16T14:41:00.893 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635533, 2023-10-16T14:41:00.893 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg" 2023-10-16T14:41:00.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2138192974, 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629826, 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml" 2023-10-16T14:41:00.894 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.895 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.895 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.895 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2157868776, 2023-10-16T14:41:00.895 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635506, 2023-10-16T14:41:00.895 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg" 2023-10-16T14:41:00.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.896 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.896 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2181747610, 2023-10-16T14:41:00.896 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635225, 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml" 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2187080727, 2023-10-16T14:41:00.897 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630201, 2023-10-16T14:41:00.898 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg" 2023-10-16T14:41:00.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.898 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.898 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.898 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2187245537, 2023-10-16T14:41:00.899 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629260, 2023-10-16T14:41:00.899 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg" 2023-10-16T14:41:00.899 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.899 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.899 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2205948747, 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635542, 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg" 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.900 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.901 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2229029594, 2023-10-16T14:41:00.901 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629390, 2023-10-16T14:41:00.901 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg" 2023-10-16T14:41:00.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.901 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2247170872, 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629770, 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml" 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.902 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.903 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.903 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2250809164, 2023-10-16T14:41:00.903 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635188, 2023-10-16T14:41:00.903 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38" 2023-10-16T14:41:00.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2255003689, 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629298, 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg" 2023-10-16T14:41:00.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2271125712, 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635301, 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml" 2023-10-16T14:41:00.905 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.906 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.906 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.906 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2288445668, 2023-10-16T14:41:00.906 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630165, 2023-10-16T14:41:00.906 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg" 2023-10-16T14:41:00.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.907 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.907 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2329918148, 2023-10-16T14:41:00.907 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629761, 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml" 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2331362903, 2023-10-16T14:41:00.908 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629272, 2023-10-16T14:41:00.909 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg" 2023-10-16T14:41:00.909 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.909 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.909 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.909 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2334009065, 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635593, 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg" 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.910 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2338410856, 2023-10-16T14:41:00.911 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629258, 2023-10-16T14:41:00.911 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg" 2023-10-16T14:41:00.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.911 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.911 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.912 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2349787741, 2023-10-16T14:41:00.912 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629313, 2023-10-16T14:41:00.912 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg" 2023-10-16T14:41:00.912 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.912 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2360046095, 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629348, 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg" 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.914 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.914 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2362050552, 2023-10-16T14:41:00.914 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635221, 2023-10-16T14:41:00.914 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml" 2023-10-16T14:41:00.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2384147838, 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635288, 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml" 2023-10-16T14:41:00.915 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.916 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.916 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.916 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2385616485, 2023-10-16T14:41:00.916 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629318, 2023-10-16T14:41:00.916 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg" 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2386334591, 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629896, 2023-10-16T14:41:00.917 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml" 2023-10-16T14:41:00.918 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.918 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.918 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.918 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2394099559, 2023-10-16T14:41:00.918 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635521, 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg" 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2432546724, 2023-10-16T14:41:00.919 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630368, 2023-10-16T14:41:00.920 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg" 2023-10-16T14:41:00.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.920 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.920 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2434852679, 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629667, 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml" 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.921 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2440798345, 2023-10-16T14:41:00.922 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629394, 2023-10-16T14:41:00.922 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg" 2023-10-16T14:41:00.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.922 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.922 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.923 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2461173686, 2023-10-16T14:41:00.923 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629239, 2023-10-16T14:41:00.923 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg" 2023-10-16T14:41:00.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.923 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.924 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.924 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2520083214, 2023-10-16T14:41:00.924 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635545, 2023-10-16T14:41:00.924 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg" 2023-10-16T14:41:00.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2528726729, 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629856, 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml" 2023-10-16T14:41:00.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.926 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.926 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.926 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2578151529, 2023-10-16T14:41:00.926 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635469, 2023-10-16T14:41:00.926 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg" 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2578364653, 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635207, 2023-10-16T14:41:00.927 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml" 2023-10-16T14:41:00.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.928 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.928 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.928 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2579572624, 2023-10-16T14:41:00.928 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629367, 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg" 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2587070502, 2023-10-16T14:41:00.929 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635264, 2023-10-16T14:41:00.930 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml" 2023-10-16T14:41:00.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.930 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.930 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2589951207, 2023-10-16T14:41:00.931 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635352, 2023-10-16T14:41:00.931 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml" 2023-10-16T14:41:00.931 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.931 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.931 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2603179791, 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635255, 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml" 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.932 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.933 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2609234291, 2023-10-16T14:41:00.933 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635398, 2023-10-16T14:41:00.933 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg" 2023-10-16T14:41:00.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.933 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.934 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.934 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2625378895, 2023-10-16T14:41:00.934 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635293, 2023-10-16T14:41:00.934 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml" 2023-10-16T14:41:00.934 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2637120006, 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629378, 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg" 2023-10-16T14:41:00.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.936 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.936 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.936 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2674446966, 2023-10-16T14:41:00.936 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629841, 2023-10-16T14:41:00.936 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml" 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2675359981, 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630023, 2023-10-16T14:41:00.937 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml" 2023-10-16T14:41:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2686501777, 2023-10-16T14:41:00.938 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629289, 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg" 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2712648336, 2023-10-16T14:41:00.939 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629343, 2023-10-16T14:41:00.940 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg" 2023-10-16T14:41:00.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.940 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.940 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.940 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2726989468, 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635279, 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml" 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.941 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2743764131, 2023-10-16T14:41:00.942 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629634, 2023-10-16T14:41:00.942 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml" 2023-10-16T14:41:00.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.942 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.942 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.943 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2756869667, 2023-10-16T14:41:00.943 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630443, 2023-10-16T14:41:00.943 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg" 2023-10-16T14:41:00.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.943 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.944 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.944 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2762471129, 2023-10-16T14:41:00.944 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635379, 2023-10-16T14:41:00.944 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg" 2023-10-16T14:41:00.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.945 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.945 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2767121118, 2023-10-16T14:41:00.945 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630415, 2023-10-16T14:41:00.945 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg" 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2787530854, 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629303, 2023-10-16T14:41:00.946 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg" 2023-10-16T14:41:00.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.947 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.947 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.947 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2794638946, 2023-10-16T14:41:00.947 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635590, 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg" 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2797083752, 2023-10-16T14:41:00.948 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511631817, 2023-10-16T14:41:00.949 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml" 2023-10-16T14:41:00.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.949 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.949 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.949 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2800214017, 2023-10-16T14:41:00.950 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629310, 2023-10-16T14:41:00.950 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg" 2023-10-16T14:41:00.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.950 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.950 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2830652728, 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629871, 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml" 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.951 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2832436113, 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635258, 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml" 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.952 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.953 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2832647213, 2023-10-16T14:41:00.953 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629219, 2023-10-16T14:41:00.953 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml" 2023-10-16T14:41:00.953 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.953 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2838109480, 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629652, 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml" 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.954 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.955 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.955 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2838453864, 2023-10-16T14:41:00.955 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630336, 2023-10-16T14:41:00.955 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg" 2023-10-16T14:41:00.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2861505465, 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630275, 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg" 2023-10-16T14:41:00.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2861856633, 2023-10-16T14:41:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629286, 2023-10-16T14:41:00.957 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg" 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2871528762, 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635241, 2023-10-16T14:41:00.958 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml" 2023-10-16T14:41:00.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.959 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.959 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2886438534, 2023-10-16T14:41:00.959 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635223, 2023-10-16T14:41:00.960 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml" 2023-10-16T14:41:00.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.960 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.960 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.960 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2889749768, 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630218, 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg" 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.961 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2903599470, 2023-10-16T14:41:00.962 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635527, 2023-10-16T14:41:00.962 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg" 2023-10-16T14:41:00.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.962 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.962 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2907514390, 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629147, 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6" 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.963 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.964 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2908296435, 2023-10-16T14:41:00.964 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635420, 2023-10-16T14:41:00.964 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg" 2023-10-16T14:41:00.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.964 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.965 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.965 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2909441302, 2023-10-16T14:41:00.965 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629956, 2023-10-16T14:41:00.965 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml" 2023-10-16T14:41:00.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2920534912, 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629718, 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml" 2023-10-16T14:41:00.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.967 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.967 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.967 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2931347543, 2023-10-16T14:41:00.967 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629292, 2023-10-16T14:41:00.967 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg" 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2941276759, 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635619, 2023-10-16T14:41:00.968 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg" 2023-10-16T14:41:00.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.969 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.969 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.969 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2963797160, 2023-10-16T14:41:00.969 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629755, 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml" 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2969123708, 2023-10-16T14:41:00.970 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629816, 2023-10-16T14:41:00.971 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml" 2023-10-16T14:41:00.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.971 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.971 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.971 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2976934667, 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635572, 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg" 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.972 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2978014550, 2023-10-16T14:41:00.973 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629294, 2023-10-16T14:41:00.973 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg" 2023-10-16T14:41:00.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.973 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 2979254297, 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635195, 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml" 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.974 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.975 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3006009392, 2023-10-16T14:41:00.975 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635313, 2023-10-16T14:41:00.975 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml" 2023-10-16T14:41:00.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.975 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.976 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.976 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3012573077, 2023-10-16T14:41:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629881, 2023-10-16T14:41:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml" 2023-10-16T14:41:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.978 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.979 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3020867338, 2023-10-16T14:41:00.979 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630144, 2023-10-16T14:41:00.979 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg" 2023-10-16T14:41:00.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.979 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3027307385, 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635622, 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg" 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.980 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.981 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.981 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3041322684, 2023-10-16T14:41:00.981 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629264, 2023-10-16T14:41:00.981 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg" 2023-10-16T14:41:00.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3050248287, 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635395, 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg" 2023-10-16T14:41:00.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.983 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.983 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3051020878, 2023-10-16T14:41:00.983 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629978, 2023-10-16T14:41:00.983 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml" 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3064651448, 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629275, 2023-10-16T14:41:00.984 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg" 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3070372257, 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635625, 2023-10-16T14:41:00.985 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg" 2023-10-16T14:41:00.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.986 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.986 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.986 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3073710563, 2023-10-16T14:41:00.986 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630383, 2023-10-16T14:41:00.987 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg" 2023-10-16T14:41:00.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.987 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.987 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.987 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3083378071, 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629268, 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg" 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.988 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3121481209, 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629285, 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg" 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.989 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3122139516, 2023-10-16T14:41:00.990 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629250, 2023-10-16T14:41:00.990 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg" 2023-10-16T14:41:00.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.990 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.990 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3126791205, 2023-10-16T14:41:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635423, 2023-10-16T14:41:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg" 2023-10-16T14:41:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3131223952, 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635237, 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml" 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.992 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3132826901, 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635596, 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg" 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.993 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.994 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.994 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3147892768, 2023-10-16T14:41:00.994 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629692, 2023-10-16T14:41:00.994 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml" 2023-10-16T14:41:00.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.995 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.995 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3170695920, 2023-10-16T14:41:00.995 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635578, 2023-10-16T14:41:00.995 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg" 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3211663905, 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630113, 2023-10-16T14:41:00.996 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml" 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3220915958, 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635478, 2023-10-16T14:41:00.997 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg" 2023-10-16T14:41:00.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.998 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.998 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.998 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3241695974, 2023-10-16T14:41:00.998 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511634443, 2023-10-16T14:41:00.999 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33" 2023-10-16T14:41:00.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:00.999 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:00.999 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:00.999 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3243956402, 2023-10-16T14:41:01.000 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635215, 2023-10-16T14:41:01.000 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml" 2023-10-16T14:41:01.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.000 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.000 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3250174646, 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635429, 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg" 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.001 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.002 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3253415138, 2023-10-16T14:41:01.002 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629297, 2023-10-16T14:41:01.002 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg" 2023-10-16T14:41:01.002 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.002 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.003 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.003 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3254065577, 2023-10-16T14:41:01.003 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629178, 2023-10-16T14:41:01.003 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml" 2023-10-16T14:41:01.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.004 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.004 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.004 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3262425812, 2023-10-16T14:41:01.004 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629165, 2023-10-16T14:41:01.004 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml" 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3274923265, 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629228, 2023-10-16T14:41:01.005 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml" 2023-10-16T14:41:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3284023489, 2023-10-16T14:41:01.006 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635285, 2023-10-16T14:41:01.007 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml" 2023-10-16T14:41:01.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.007 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.007 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.007 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3284688608, 2023-10-16T14:41:01.008 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629265, 2023-10-16T14:41:01.008 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg" 2023-10-16T14:41:01.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.008 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.008 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.009 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3292376884, 2023-10-16T14:41:01.009 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629821, 2023-10-16T14:41:01.009 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml" 2023-10-16T14:41:01.009 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.009 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.010 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.010 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3300321080, 2023-10-16T14:41:01.010 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630374, 2023-10-16T14:41:01.010 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg" 2023-10-16T14:41:01.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.011 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.011 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3302200822, 2023-10-16T14:41:01.011 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629334, 2023-10-16T14:41:01.011 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg" 2023-10-16T14:41:01.012 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.012 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.012 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.012 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3327197188, 2023-10-16T14:41:01.012 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635413, 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg" 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3351798483, 2023-10-16T14:41:01.013 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629208, 2023-10-16T14:41:01.014 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml" 2023-10-16T14:41:01.014 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.014 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.014 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.014 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3361505416, 2023-10-16T14:41:01.015 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629188, 2023-10-16T14:41:01.015 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml" 2023-10-16T14:41:01.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.015 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.015 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3361703167, 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629322, 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg" 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.016 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3363447371, 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629307, 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg" 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.017 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.018 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3368939251, 2023-10-16T14:41:01.018 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635439, 2023-10-16T14:41:01.018 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg" 2023-10-16T14:41:01.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3369967903, 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635245, 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml" 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.019 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.020 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.020 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3371305140, 2023-10-16T14:41:01.020 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629312, 2023-10-16T14:41:01.020 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg" 2023-10-16T14:41:01.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3401537864, 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635449, 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg" 2023-10-16T14:41:01.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.022 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.022 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.022 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3402672624, 2023-10-16T14:41:01.022 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635405, 2023-10-16T14:41:01.022 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg" 2023-10-16T14:41:01.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.023 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.023 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.023 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3403952856, 2023-10-16T14:41:01.023 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635273, 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml" 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3409412098, 2023-10-16T14:41:01.024 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635464, 2023-10-16T14:41:01.025 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg" 2023-10-16T14:41:01.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.025 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.025 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.025 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3437802058, 2023-10-16T14:41:01.026 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629738, 2023-10-16T14:41:01.026 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml" 2023-10-16T14:41:01.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.026 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3440882651, 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629262, 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg" 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.027 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.028 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3449953156, 2023-10-16T14:41:01.028 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629623, 2023-10-16T14:41:01.028 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml" 2023-10-16T14:41:01.028 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.028 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3457373784, 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635575, 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg" 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.029 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.030 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.030 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3483388552, 2023-10-16T14:41:01.030 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629256, 2023-10-16T14:41:01.030 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg" 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3514655329, 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635191, 2023-10-16T14:41:01.031 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml" 2023-10-16T14:41:01.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.032 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.032 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.032 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3541288348, 2023-10-16T14:41:01.032 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629696, 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml" 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3544548114, 2023-10-16T14:41:01.033 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630227, 2023-10-16T14:41:01.034 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg" 2023-10-16T14:41:01.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.034 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.034 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.034 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3590811265, 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630439, 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg" 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.035 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3624355857, 2023-10-16T14:41:01.036 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629245, 2023-10-16T14:41:01.036 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg" 2023-10-16T14:41:01.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.036 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.036 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.037 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3651877151, 2023-10-16T14:41:01.037 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629202, 2023-10-16T14:41:01.037 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml" 2023-10-16T14:41:01.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.037 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.038 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.038 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3658534595, 2023-10-16T14:41:01.038 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635426, 2023-10-16T14:41:01.038 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg" 2023-10-16T14:41:01.038 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.039 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.039 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.039 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3710212193, 2023-10-16T14:41:01.039 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629900, 2023-10-16T14:41:01.039 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml" 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3711519043, 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635613, 2023-10-16T14:41:01.040 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg" 2023-10-16T14:41:01.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.041 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.041 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3718492865, 2023-10-16T14:41:01.041 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629361, 2023-10-16T14:41:01.042 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg" 2023-10-16T14:41:01.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.042 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.042 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.042 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3724749380, 2023-10-16T14:41:01.043 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629324, 2023-10-16T14:41:01.043 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg" 2023-10-16T14:41:01.043 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.043 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.043 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3728381103, 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629917, 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml" 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.044 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3774663865, 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629223, 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml" 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.045 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.046 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3798503245, 2023-10-16T14:41:01.046 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629319, 2023-10-16T14:41:01.046 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg" 2023-10-16T14:41:01.046 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.046 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.047 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.047 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3826927153, 2023-10-16T14:41:01.047 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629702, 2023-10-16T14:41:01.047 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml" 2023-10-16T14:41:01.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3843683581, 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630008, 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml" 2023-10-16T14:41:01.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.049 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.049 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.049 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3880435779, 2023-10-16T14:41:01.049 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635383, 2023-10-16T14:41:01.049 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg" 2023-10-16T14:41:01.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.050 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.050 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.050 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3895780942, 2023-10-16T14:41:01.050 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629225, 2023-10-16T14:41:01.051 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml" 2023-10-16T14:41:01.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.051 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.051 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.051 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3904596907, 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629801, 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml" 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.052 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3937777684, 2023-10-16T14:41:01.053 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635436, 2023-10-16T14:41:01.053 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg" 2023-10-16T14:41:01.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.053 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.053 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.054 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3962290975, 2023-10-16T14:41:01.054 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629398, 2023-10-16T14:41:01.054 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg" 2023-10-16T14:41:01.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.054 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3964508448, 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630407, 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg" 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.055 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.056 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.056 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3973031470, 2023-10-16T14:41:01.056 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629309, 2023-10-16T14:41:01.056 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg" 2023-10-16T14:41:01.056 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3977872469, 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629291, 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg" 2023-10-16T14:41:01.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.058 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.058 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.058 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3992690838, 2023-10-16T14:41:01.058 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630395, 2023-10-16T14:41:01.058 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg" 2023-10-16T14:41:01.059 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.059 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.059 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3995456310, 2023-10-16T14:41:01.059 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630015, 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg" 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 3995912174, 2023-10-16T14:41:01.060 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629655, 2023-10-16T14:41:01.061 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml" 2023-10-16T14:41:01.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.061 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.061 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4008463584, 2023-10-16T14:41:01.062 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635199, 2023-10-16T14:41:01.062 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml" 2023-10-16T14:41:01.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.062 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.062 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4010125979, 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629706, 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml" 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.063 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4011233943, 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629321, 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg" 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.064 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.065 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4012227243, 2023-10-16T14:41:01.065 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629905, 2023-10-16T14:41:01.065 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml" 2023-10-16T14:41:01.065 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.066 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.066 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4018376256, 2023-10-16T14:41:01.066 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629199, 2023-10-16T14:41:01.066 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml" 2023-10-16T14:41:01.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.067 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.067 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.067 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4022011590, 2023-10-16T14:41:01.067 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629922, 2023-10-16T14:41:01.067 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml" 2023-10-16T14:41:01.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.068 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.068 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.068 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4027675642, 2023-10-16T14:41:01.068 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629969, 2023-10-16T14:41:01.069 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml" 2023-10-16T14:41:01.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.069 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.069 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.069 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4028943389, 2023-10-16T14:41:01.070 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635270, 2023-10-16T14:41:01.070 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml" 2023-10-16T14:41:01.070 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.070 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.070 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4047854724, 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635584, 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg" 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.071 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.072 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4049419061, 2023-10-16T14:41:01.072 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635432, 2023-10-16T14:41:01.072 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg" 2023-10-16T14:41:01.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.073 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.073 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4080576981, 2023-10-16T14:41:01.073 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629401, 2023-10-16T14:41:01.073 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg" 2023-10-16T14:41:01.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.074 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.074 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.074 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4102681176, 2023-10-16T14:41:01.074 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629346, 2023-10-16T14:41:01.074 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg" 2023-10-16T14:41:01.075 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.075 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.075 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.075 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4142885377, 2023-10-16T14:41:01.075 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629316, 2023-10-16T14:41:01.076 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg" 2023-10-16T14:41:01.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.076 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.076 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.076 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4146478816, 2023-10-16T14:41:01.077 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630122, 2023-10-16T14:41:01.077 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg" 2023-10-16T14:41:01.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.077 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.078 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4164446158, 2023-10-16T14:41:01.078 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511630432, 2023-10-16T14:41:01.078 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg" 2023-10-16T14:41:01.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.078 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4225285190, 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635600, 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg" 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.079 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.080 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.080 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4227091838, 2023-10-16T14:41:01.080 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629861, 2023-10-16T14:41:01.080 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml" 2023-10-16T14:41:01.080 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.081 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.081 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.081 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4252797253, 2023-10-16T14:41:01.081 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511629288, 2023-10-16T14:41:01.081 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg" 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4265716777, 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635462, 2023-10-16T14:41:01.082 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg" 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: "damage_type": "backtrace", 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 4293548109, 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: "ino": 1099511635239, 2023-10-16T14:41:01.083 INFO:teuthology.orchestra.run.smithi178.stdout: "path": "/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml" 2023-10-16T14:41:01.084 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:41:01.084 INFO:teuthology.orchestra.run.smithi178.stdout:] 2023-10-16T14:41:01.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 msgr2=0x7efd040a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 0x7efd040a3a60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efd08005fd0 tx=0x7efd0800fa90 comp rx=0 tx=0).stop 2023-10-16T14:41:01.085 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 msgr2=0x7efd18074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efd1400ea50 tx=0x7efd1400ed60 comp rx=0 tx=0).stop 2023-10-16T14:41:01.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 shutdown_connections 2023-10-16T14:41:01.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7efd040a15a0 0x7efd040a3a60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd181028a0 0x7efd180737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.086 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7efd18100d10 0x7efd18074c60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 --2- 172.21.15.178:0/2440220271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd18101f00 0x7efd180732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.649+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 >> 172.21.15.178:0/2440220271 conn(0x7efd180fc280 msgr2=0x7efd180fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.650+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 shutdown_connections 2023-10-16T14:41:01.087 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:00.650+0000 7efd1f19d700 1 -- 172.21.15.178:0/2440220271 wait complete. 2023-10-16T14:41:01.089 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-10-16T14:41:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-10-16T14:41:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-16T14:41:01.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-16T14:41:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-16T14:41:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1415734595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:41:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:41:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:41:01.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:00 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:41:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-10-16T14:41:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-16T14:41:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-16T14:41:01.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-16T14:41:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1415734595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:41:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:41:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:41:01.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:00 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:41:01.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 -- 172.21.15.178:0/2350191392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102070 msgr2=0x7fad501024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 --2- 172.21.15.178:0/2350191392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102070 0x7fad501024f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad40009990 tx=0x7fad40009ca0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 -- 172.21.15.178:0/2350191392 shutdown_connections 2023-10-16T14:41:01.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 --2- 172.21.15.178:0/2350191392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102a30 0x7fad50109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 --2- 172.21.15.178:0/2350191392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad501012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 --2- 172.21.15.178:0/2350191392 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102070 0x7fad501024f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 -- 172.21.15.178:0/2350191392 >> 172.21.15.178:0/2350191392 conn(0x7fad500fc430 msgr2=0x7fad500fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 -- 172.21.15.178:0/2350191392 shutdown_connections 2023-10-16T14:41:01.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.401+0000 7fad57dff700 1 -- 172.21.15.178:0/2350191392 wait complete. 2023-10-16T14:41:01.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.402+0000 7fad57dff700 1 Processor -- start 2023-10-16T14:41:01.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 -- start start 2023-10-16T14:41:01.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad50196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 0x7fad50197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019e740 con 0x7fad50100e90 2023-10-16T14:41:01.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019e880 con 0x7fad50102a30 2023-10-16T14:41:01.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad57dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5019ea10 con 0x7fad50102070 2023-10-16T14:41:01.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad50196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad5539a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 0x7fad50197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad5639c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad5539a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 0x7fad50197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:53142/0 (socket says 172.21.15.178:53142) 2023-10-16T14:41:01.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad5639c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53214/0 (socket says 172.21.15.178:53214) 2023-10-16T14:41:01.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.403+0000 7fad5539a700 1 -- 172.21.15.178:0/2039534694 learned_addr learned my addr 172.21.15.178:0/2039534694 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:01.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 -- 172.21.15.178:0/2039534694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 msgr2=0x7fad50197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 0x7fad50197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 -- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 msgr2=0x7fad50196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad50196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 -- 172.21.15.178:0/2039534694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400095f0 con 0x7fad50102a30 2023-10-16T14:41:01.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad55b9b700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad50196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad5639c700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad48007f10 tx=0x7fad4800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad46ffd700 1 -- 172.21.15.178:0/2039534694 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48004db0 con 0x7fad50102a30 2023-10-16T14:41:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad46ffd700 1 -- 172.21.15.178:0/2039534694 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4800f040 con 0x7fad50102a30 2023-10-16T14:41:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5019ecf0 con 0x7fad50102a30 2023-10-16T14:41:01.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.404+0000 7fad46ffd700 1 -- 172.21.15.178:0/2039534694 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48005990 con 0x7fad50102a30 2023-10-16T14:41:01.414 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.405+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5019f1b0 con 0x7fad50102a30 2023-10-16T14:41:01.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.409+0000 7fad46ffd700 1 -- 172.21.15.178:0/2039534694 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fad4800db20 con 0x7fad50102a30 2023-10-16T14:41:01.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 Processor -- start 2023-10-16T14:41:01.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 -- start start 2023-10-16T14:41:01.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 0x5647f6222650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6223eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6192c70 con 0x5647f6222230 2023-10-16T14:41:01.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6177c60 con 0x5647f6222b90 2023-10-16T14:41:01.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.410+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6175580 con 0x5647f6223a30 2023-10-16T14:41:01.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad5539a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 0x5647f6222650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53218/0 (socket says 172.21.15.178:53218) 2023-10-16T14:41:01.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 -- 172.21.15.178:0/2343379219 learned_addr learned my addr 172.21.15.178:0/2343379219 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:01.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad46ffd700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 0x7fad3c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad5539a700 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6223eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.422 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 -- 172.21.15.178:0/2343379219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 msgr2=0x5647f6223eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6223eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 -- 172.21.15.178:0/2343379219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 msgr2=0x5647f6222650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 0x5647f6222650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 -- 172.21.15.178:0/2343379219 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f612e3f0 con 0x5647f6222b90 2023-10-16T14:41:01.423 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad5639c700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 0x7fad3c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad46ffd700 1 -- 172.21.15.178:0/2039534694 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fad480cc8b0 con 0x7fad50102a30 2023-10-16T14:41:01.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad5539a700 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 0x5647f6222650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:41:01.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad55b9b700 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0115c0 tx=0x7fad4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a91c4c265d41d48 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.411+0000 7fad3bfff700 1 -- 172.21.15.178:0/2343379219 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c004020 con 0x5647f6222b90 2023-10-16T14:41:01.424 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad3bfff700 1 -- 172.21.15.178:0/2343379219 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4c004830 con 0x5647f6222b90 2023-10-16T14:41:01.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad5639c700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 0x7fad3c0a3940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad48005250 tx=0x7fad48004f10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad3bfff700 1 -- 172.21.15.178:0/2343379219 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0054a0 con 0x5647f6222b90 2023-10-16T14:41:01.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 -- 172.21.15.178:0/2343379219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 msgr2=0x5647f6223450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0115c0 tx=0x7fad4c0118d0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.425 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 -- 172.21.15.178:0/2343379219 shutdown_connections 2023-10-16T14:41:01.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6223eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222230 0x5647f6222650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/2343379219 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6223450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.426 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 -- 172.21.15.178:0/2343379219 >> 172.21.15.178:0/2343379219 conn(0x5647f614bdd0 msgr2=0x5647f6218f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 -- 172.21.15.178:0/2343379219 shutdown_connections 2023-10-16T14:41:01.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.412+0000 7fad6b69cb80 1 -- 172.21.15.178:0/2343379219 wait complete. 2023-10-16T14:41:01.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.413+0000 7fad6b69cb80 1 -- start start 2023-10-16T14:41:01.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.413+0000 7fad6b69cb80 1 Processor -- start 2023-10-16T14:41:01.427 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f622c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f6233610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6170e70 con 0x5647f6223a30 2023-10-16T14:41:01.428 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f610d710 con 0x5647f6222b90 2023-10-16T14:41:01.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f610d3b0 con 0x5647f6222230 2023-10-16T14:41:01.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad6b69cb80 10 client.0 _ll_register_callbacks cb 0x5647f5f6a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:01.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53232/0 (socket says 172.21.15.178:53232) 2023-10-16T14:41:01.429 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 -- 172.21.15.178:0/615103074 learned_addr learned my addr 172.21.15.178:0/615103074 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:01.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad5539a700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f622c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad5539a700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f6233610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 -- 172.21.15.178:0/615103074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 msgr2=0x5647f622c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.430 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f622c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 -- 172.21.15.178:0/615103074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 msgr2=0x5647f6233610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f6233610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.414+0000 7fad55b9b700 1 -- 172.21.15.178:0/615103074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4c009d50 con 0x5647f6222b90 2023-10-16T14:41:01.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad55b9b700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad4c000c00 tx=0x7fad4c005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=171dc9c5ecdcfe99 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.431 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad397fa700 10 client.0 ms_handle_connect on v2:172.21.15.186:3300/0 2023-10-16T14:41:01.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c005720 con 0x5647f6222b90 2023-10-16T14:41:01.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647f6175ef0 con 0x5647f6222b90 2023-10-16T14:41:01.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mon.2 v2:172.21.15.186:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fad4c005c60 con 0x5647f6222b90 2023-10-16T14:41:01.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4c023420 con 0x5647f6222b90 2023-10-16T14:41:01.432 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad6b69cb80 10 client.26324 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:01.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5647f5f9bde0 con 0x5647f6222b90 2023-10-16T14:41:01.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.415+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mon.2 v2:172.21.15.186:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647f6175ef0 con 0x5647f6222b90 2023-10-16T14:41:01.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mon.2 v2:172.21.15.186:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fad4c019760 con 0x5647f6222b90 2023-10-16T14:41:01.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 10 client.26324 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:01.433 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 10 client.26324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:41:01.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 20 client.26324 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:01.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f6237f00 0x5647f623a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 4 client.26324 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T14:41:01.434 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647f61edc10 con 0x5647f6237f00 2023-10-16T14:41:01.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad5539a700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f6237f00 0x5647f623a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad5539a700 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f6237f00 0x5647f623a320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e183895781267c server_cookie=30a7e13af8c90143 in_seq=0 out_seq=0 2023-10-16T14:41:01.435 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.416+0000 7fad397fa700 10 client.26324 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:41:01.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi187 to 196.2M 2023-10-16T14:41:01.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: Unable to set osd_memory_target on smithi187 to 205752422: error parsing value: Value '205752422' is below minimum 939524096 2023-10-16T14:41:01.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi186 to 196.2M 2023-10-16T14:41:01.436 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: Unable to set osd_memory_target on smithi186 to 205769625: error parsing value: Value '205769625' is below minimum 939524096 2023-10-16T14:41:01.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1415734595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:41:01.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.437 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-10-16T14:41:01.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:41:01.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:41:01.438 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:00 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:41:01.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.417+0000 7fad397fa700 1 -- 172.21.15.178:0/615103074 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5647f61edc10 con 0x5647f6237f00 2023-10-16T14:41:01.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.417+0000 7fad397fa700 10 client.26324 handle_command_reply: tid=0 2023-10-16T14:41:01.443 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.417+0000 7fad6b69cb80 1 client.26324 shutdown 2023-10-16T14:41:01.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 msgr2=0x5647f622c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fad4c000c00 tx=0x7fad4c005480 comp rx=0 tx=0).stop 2023-10-16T14:41:01.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 shutdown_connections 2023-10-16T14:41:01.444 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f6237f00 0x5647f623a320 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f6233610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f622c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/615103074 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f622c5d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad397fa700 0 client.26324 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:41:01.445 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 >> 172.21.15.178:0/615103074 conn(0x5647f614bdd0 msgr2=0x5647f6218b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 shutdown_connections 2023-10-16T14:41:01.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 1 -- 172.21.15.178:0/615103074 wait complete. 2023-10-16T14:41:01.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.418+0000 7fad6b69cb80 20 client.26324 trim_cache size 0 max 16384 2023-10-16T14:41:01.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.431+0000 7fad6b69cb80 1 Processor -- start 2023-10-16T14:41:01.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 -- start start 2023-10-16T14:41:01.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f62269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f62335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f5ff1920 con 0x5647f6223a30 2023-10-16T14:41:01.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f622ab20 con 0x5647f6222b90 2023-10-16T14:41:01.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f622b540 con 0x5647f6222230 2023-10-16T14:41:01.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53244/0 (socket says 172.21.15.178:53244) 2023-10-16T14:41:01.448 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 -- 172.21.15.178:0/1315285355 learned_addr learned my addr 172.21.15.178:0/1315285355 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:01.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad5539a700 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f62335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad5539a700 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f62269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 -- 172.21.15.178:0/1315285355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 msgr2=0x5647f62269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f62269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 -- 172.21.15.178:0/1315285355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 msgr2=0x5647f62335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f62335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.432+0000 7fad55b9b700 1 -- 172.21.15.178:0/1315285355 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f5f84c60 con 0x5647f6222b90 2023-10-16T14:41:01.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.433+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005a80 tx=0x7fad4c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cf6d18b4cccd5e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.433+0000 7fad44bff700 1 -- 172.21.15.178:0/1315285355 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0055a0 con 0x5647f6222b90 2023-10-16T14:41:01.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.433+0000 7fad44bff700 1 -- 172.21.15.178:0/1315285355 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4c005700 con 0x5647f6222b90 2023-10-16T14:41:01.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.433+0000 7fad44bff700 1 -- 172.21.15.178:0/1315285355 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c021c20 con 0x5647f6222b90 2023-10-16T14:41:01.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1315285355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 msgr2=0x5647f6226ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad4c005a80 tx=0x7fad4c0118d0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1315285355 shutdown_connections 2023-10-16T14:41:01.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6223a30 0x5647f62335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6222230 0x5647f62269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1315285355 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222b90 0x5647f6226ee0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1315285355 >> 172.21.15.178:0/1315285355 conn(0x5647f614bdd0 msgr2=0x5647f6229e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1315285355 shutdown_connections 2023-10-16T14:41:01.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1315285355 wait complete. 2023-10-16T14:41:01.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 -- start start 2023-10-16T14:41:01.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.434+0000 7fad6b69cb80 1 Processor -- start 2023-10-16T14:41:01.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 0x5647f6231940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6232950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6232e90 con 0x5647f6222b90 2023-10-16T14:41:01.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f6233000 con 0x5647f6222230 2023-10-16T14:41:01.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f62331c0 con 0x5647f6223a30 2023-10-16T14:41:01.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad6b69cb80 10 client.0 _ll_register_callbacks cb 0x7fad4c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T14:41:01.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad5539a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad5539a700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:53250/0 (socket says 172.21.15.178:53250) 2023-10-16T14:41:01.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad5539a700 1 -- 172.21.15.178:0/1332115183 learned_addr learned my addr 172.21.15.178:0/1332115183 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:41:01.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 0x5647f6231940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad5539a700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6232950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad55b9b700 1 -- 172.21.15.178:0/1332115183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 msgr2=0x5647f6232950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.435+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6232950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad55b9b700 1 -- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 msgr2=0x5647f6231400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T14:41:01.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad55b9b700 1 -- 172.21.15.178:0/1332115183 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647f5f9bde0 con 0x5647f6222b90 2023-10-16T14:41:01.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad5539a700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:41:01.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad55b9b700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 0x5647f6231940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5647f623a830 tx=0x7fad4c005110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7387e06bee8fe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:41:01.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.436+0000 7fad397fa700 10 client.0 ms_handle_connect on v2:172.21.15.178:3300/0 2023-10-16T14:41:01.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.438+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c019610 con 0x5647f6222b90 2023-10-16T14:41:01.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.438+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mon.0 v2:172.21.15.178:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fad4c000e40 con 0x5647f6222b90 2023-10-16T14:41:01.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.438+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647f6169e50 con 0x5647f6222b90 2023-10-16T14:41:01.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.438+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad4c021540 con 0x5647f6222b90 2023-10-16T14:41:01.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 10 client.16251 fetch_fsmap learned FSMap version 36 2023-10-16T14:41:01.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5647f5fc2250 con 0x5647f6222b90 2023-10-16T14:41:01.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mon.0 v2:172.21.15.178:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647f6169e50 con 0x5647f6222b90 2023-10-16T14:41:01.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mon.0 v2:172.21.15.178:3300/0 5 ==== fsmap(e 36) v1 ==== 7914+0+0 (secure 0 0 0) 0x7fad4c03c070 con 0x5647f6222b90 2023-10-16T14:41:01.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 10 client.16251 fetch_fsmap finished waiting for FSMap version 36 2023-10-16T14:41:01.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-10-16T14:41:01.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 20 client.16251 populate_metadata read hostname 'smithi178' 2023-10-16T14:41:01.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f628c7e0 0x5647f628ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:41:01.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 4 client.16251 mds_command: new command op to 24593 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2023-10-16T14:41:01.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.439+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 --> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5647f6169b70 con 0x5647f628c7e0 2023-10-16T14:41:01.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.440+0000 7fad5539a700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f628c7e0 0x5647f628ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:41:01.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.441+0000 7fad5539a700 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f628c7e0 0x5647f628ec00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f2680cd9527af6 server_cookie=b049f67bce8575dd in_seq=0 out_seq=0 2023-10-16T14:41:01.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.441+0000 7fad397fa700 10 client.16251 ms_handle_connect on v2:172.21.15.186:6838/2475230127 2023-10-16T14:41:01.462 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.441+0000 7fad397fa700 1 -- 172.21.15.178:0/1332115183 <== mds.0 v2:172.21.15.186:6838/2475230127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5647f6169b70 con 0x5647f628c7e0 2023-10-16T14:41:01.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.441+0000 7fad397fa700 10 client.16251 handle_command_reply: tid=0 2023-10-16T14:41:01.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.441+0000 7fad6b69cb80 1 client.16251 shutdown 2023-10-16T14:41:01.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 msgr2=0x5647f6231940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 0x5647f6231940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5647f623a830 tx=0x7fad4c005110 comp rx=0 tx=0).stop 2023-10-16T14:41:01.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 shutdown_connections 2023-10-16T14:41:01.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:6838/2475230127,v1:172.21.15.186:6839/2475230127] conn(0x5647f628c7e0 0x5647f628ec00 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5647f6223a30 0x5647f6232950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x5647f6222230 0x5647f6231400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.464 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 --2- 172.21.15.178:0/1332115183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x5647f6222b90 0x5647f6231940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 >> 172.21.15.178:0/1332115183 conn(0x5647f614bdd0 msgr2=0x5647f6229e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad397fa700 0 client.16251 ms_handle_reset on v2:172.21.15.186:6838/2475230127 2023-10-16T14:41:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 shutdown_connections 2023-10-16T14:41:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.443+0000 7fad6b69cb80 1 -- 172.21.15.178:0/1332115183 wait complete. 2023-10-16T14:41:01.465 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.448+0000 7fad6b69cb80 20 client.16251 trim_cache size 0 max 16384 2023-10-16T14:41:01.466 INFO:teuthology.orchestra.run.smithi178.stdout:[] 2023-10-16T14:41:01.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 msgr2=0x7fad3c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 0x7fad3c0a3940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad48005250 tx=0x7fad48004f10 comp rx=0 tx=0).stop 2023-10-16T14:41:01.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 msgr2=0x7fad5019e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:41:01.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fad48007f10 tx=0x7fad4800d3b0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 shutdown_connections 2023-10-16T14:41:01.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fad3c0a1480 0x7fad3c0a3940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fad50102a30 0x7fad5019e170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.468 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad50100e90 0x7fad50196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 --2- 172.21.15.178:0/2039534694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad50102070 0x7fad50197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:41:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.449+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 >> 172.21.15.178:0/2039534694 conn(0x7fad500fc430 msgr2=0x7fad500fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:41:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.450+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 shutdown_connections 2023-10-16T14:41:01.469 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:41:01.450+0000 7fad57dff700 1 -- 172.21.15.178:0/2039534694 wait complete. 2023-10-16T14:41:01.487 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 2319878, 'ino': 1099511629966, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-14.xml'}, {'damage_type': 'backtrace', 'id': 7835518, 'ino': 1099511629345, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 12837385, 'ino': 1099511629672, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-70.xml'}, {'damage_type': 'backtrace', 'id': 24322276, 'ino': 1099511629241, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-95.jpg'}, {'damage_type': 'backtrace', 'id': 26658060, 'ino': 1099511629998, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-7.xml'}, {'damage_type': 'backtrace', 'id': 67181017, 'ino': 1099511629304, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 70917158, 'ino': 1099511629992, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-8.xml'}, {'damage_type': 'backtrace', 'id': 81342961, 'ino': 1099511630309, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 94428883, 'ino': 1099511632676, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-81.jpg'}, {'damage_type': 'backtrace', 'id': 106156555, 'ino': 1099511629150, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-25.xml'}, {'damage_type': 'backtrace', 'id': 126737440, 'ino': 1099511630236, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 143231450, 'ino': 1099511635443, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-64.jpg'}, {'damage_type': 'backtrace', 'id': 144891685, 'ino': 1099511635322, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-11.xml'}, {'damage_type': 'backtrace', 'id': 154245988, 'ino': 1099511630192, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 170618326, 'ino': 1099511635331, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-8.xml'}, {'damage_type': 'backtrace', 'id': 193400914, 'ino': 1099511635229, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-45.xml'}, {'damage_type': 'backtrace', 'id': 205955252, 'ino': 1099511629158, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-23.xml'}, {'damage_type': 'backtrace', 'id': 208984527, 'ino': 1099511635472, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 223304131, 'ino': 1099511629744, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-57.xml'}, {'damage_type': 'backtrace', 'id': 243091576, 'ino': 1099511630245, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 253623843, 'ino': 1099511629609, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10'}, {'damage_type': 'backtrace', 'id': 268762640, 'ino': 1099511629364, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 284056542, 'ino': 1099511635337, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-6.xml'}, {'damage_type': 'backtrace', 'id': 284547879, 'ino': 1099511629181, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-16.xml'}, {'damage_type': 'backtrace', 'id': 288645309, 'ino': 1099511635457, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-58.jpg'}, {'damage_type': 'backtrace', 'id': 291924694, 'ino': 1099511635282, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-23.xml'}, {'damage_type': 'backtrace', 'id': 296312784, 'ino': 1099511629836, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-40.xml'}, {'damage_type': 'backtrace', 'id': 301713849, 'ino': 1099511635375, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-84.jpg'}, {'damage_type': 'backtrace', 'id': 310851966, 'ino': 1099511629349, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 320244501, 'ino': 1099511635197, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-61.xml'}, {'damage_type': 'backtrace', 'id': 334318390, 'ino': 1099511629273, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-81.jpg'}, {'damage_type': 'backtrace', 'id': 346425679, 'ino': 1099511635451, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-60.jpg'}, {'damage_type': 'backtrace', 'id': 349497018, 'ino': 1099511629254, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-91.jpg'}, {'damage_type': 'backtrace', 'id': 351050429, 'ino': 1099511629168, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-20.xml'}, {'damage_type': 'backtrace', 'id': 352636356, 'ino': 1099511635605, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 363016705, 'ino': 1099511635548, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 363171402, 'ino': 1099511635563, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 363691499, 'ino': 1099511635518, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 372422872, 'ino': 1099511635205, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-57.xml'}, {'damage_type': 'backtrace', 'id': 375625430, 'ino': 1099511635557, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 384543122, 'ino': 1099511629327, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 402109119, 'ino': 1099511629283, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-75.jpg'}, {'damage_type': 'backtrace', 'id': 436411263, 'ino': 1099511635189, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-65.xml'}, {'damage_type': 'backtrace', 'id': 437040640, 'ino': 1099511630298, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 474633632, 'ino': 1099511629396, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 476136294, 'ino': 1099511629282, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-76.jpg'}, {'damage_type': 'backtrace', 'id': 478921388, 'ino': 1099511629687, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-67.xml'}, {'damage_type': 'backtrace', 'id': 479002747, 'ino': 1099511629384, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 480336587, 'ino': 1099511629155, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-24.xml'}, {'damage_type': 'backtrace', 'id': 485562124, 'ino': 1099511631649, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-10.xml'}, {'damage_type': 'backtrace', 'id': 489044723, 'ino': 1099511629280, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-77.jpg'}, {'damage_type': 'backtrace', 'id': 490146238, 'ino': 1099511629337, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-39.jpg'}, {'damage_type': 'backtrace', 'id': 503436537, 'ino': 1099511635233, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-43.xml'}, {'damage_type': 'backtrace', 'id': 507358548, 'ino': 1099511629682, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-68.xml'}, {'damage_type': 'backtrace', 'id': 508942911, 'ino': 1099511629342, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 519797832, 'ino': 1099511635203, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-58.xml'}, {'damage_type': 'backtrace', 'id': 541904875, 'ino': 1099511629950, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-17.xml'}, {'damage_type': 'backtrace', 'id': 572675621, 'ino': 1099511629300, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-64.jpg'}, {'damage_type': 'backtrace', 'id': 589455213, 'ino': 1099511629927, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-22.xml'}, {'damage_type': 'backtrace', 'id': 594668103, 'ino': 1099511635554, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 654339824, 'ino': 1099511632683, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-78.xml'}, {'damage_type': 'backtrace', 'id': 687501352, 'ino': 1099511629382, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 688171571, 'ino': 1099511629618, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-80.xml'}, {'damage_type': 'backtrace', 'id': 700224530, 'ino': 1099511629357, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 719513445, 'ino': 1099511630321, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 724650248, 'ino': 1099511630174, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 741642874, 'ino': 1099511635490, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-46.jpg'}, {'damage_type': 'backtrace', 'id': 742397383, 'ino': 1099511631420, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 751093847, 'ino': 1099511629354, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 752193643, 'ino': 1099511635536, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 753619016, 'ino': 1099511629301, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-63.jpg'}, {'damage_type': 'backtrace', 'id': 763431410, 'ino': 1099511635445, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-63.jpg'}, {'damage_type': 'backtrace', 'id': 793828138, 'ino': 1099511635392, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-79.jpg'}, {'damage_type': 'backtrace', 'id': 802759863, 'ino': 1099511635349, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-2.xml'}, {'damage_type': 'backtrace', 'id': 818386344, 'ino': 1099511635610, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 822276294, 'ino': 1099511635211, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-54.xml'}, {'damage_type': 'backtrace', 'id': 830767948, 'ino': 1099511635509, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 839142147, 'ino': 1099511629248, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-93.jpg'}, {'damage_type': 'backtrace', 'id': 845036951, 'ino': 1099511629306, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-60.jpg'}, {'damage_type': 'backtrace', 'id': 891090259, 'ino': 1099511629724, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-60.xml'}, {'damage_type': 'backtrace', 'id': 896413593, 'ino': 1099511629315, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-54.jpg'}, {'damage_type': 'backtrace', 'id': 897745584, 'ino': 1099511635484, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 902214943, 'ino': 1099511629376, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 916737967, 'ino': 1099511635276, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-25.xml'}, {'damage_type': 'backtrace', 'id': 937207320, 'ino': 1099511635454, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-59.jpg'}, {'damage_type': 'backtrace', 'id': 958645876, 'ino': 1099511635251, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-34.xml'}, {'damage_type': 'backtrace', 'id': 963626449, 'ino': 1099511629352, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 973774420, 'ino': 1099511630013, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-4.xml'}, {'damage_type': 'backtrace', 'id': 975896361, 'ino': 1099511635361, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-85.jpg'}, {'damage_type': 'backtrace', 'id': 986216494, 'ino': 1099511629988, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-9.xml'}, {'damage_type': 'backtrace', 'id': 994190333, 'ino': 1099511629676, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-69.xml'}, {'damage_type': 'backtrace', 'id': 1013808059, 'ino': 1099511629340, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-37.jpg'}, {'damage_type': 'backtrace', 'id': 1021146996, 'ino': 1099511635566, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 1030593308, 'ino': 1099511635316, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-13.xml'}, {'damage_type': 'backtrace', 'id': 1033281165, 'ino': 1099511635560, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 1033595993, 'ino': 1099511629195, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1049544777, 'ino': 1099511629846, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-38.xml'}, {'damage_type': 'backtrace', 'id': 1052431883, 'ino': 1099511635325, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-10.xml'}, {'damage_type': 'backtrace', 'id': 1053388024, 'ino': 1099511635298, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-19.xml'}, {'damage_type': 'backtrace', 'id': 1066548262, 'ino': 1099511629831, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-41.xml'}, {'damage_type': 'backtrace', 'id': 1070415811, 'ino': 1099511629211, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-7.xml'}, {'damage_type': 'backtrace', 'id': 1072550094, 'ino': 1099511635247, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-36.xml'}, {'damage_type': 'backtrace', 'id': 1079212577, 'ino': 1099511635340, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-5.xml'}, {'damage_type': 'backtrace', 'id': 1093558130, 'ino': 1099511629330, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-44.jpg'}, {'damage_type': 'backtrace', 'id': 1100129957, 'ino': 1099511635447, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-62.jpg'}, {'damage_type': 'backtrace', 'id': 1127703539, 'ino': 1099511629328, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-45.jpg'}, {'damage_type': 'backtrace', 'id': 1142242836, 'ino': 1099511635201, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-59.xml'}, {'damage_type': 'backtrace', 'id': 1147535770, 'ino': 1099511629325, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-47.jpg'}, {'damage_type': 'backtrace', 'id': 1155749113, 'ino': 1099511630423, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 1191330196, 'ino': 1099511629645, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-75.xml'}, {'damage_type': 'backtrace', 'id': 1191811930, 'ino': 1099511635328, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-9.xml'}, {'damage_type': 'backtrace', 'id': 1197421137, 'ino': 1099511630255, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-21.jpg'}, {'damage_type': 'backtrace', 'id': 1215889868, 'ino': 1099511635304, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-17.xml'}, {'damage_type': 'backtrace', 'id': 1216029935, 'ino': 1099511635319, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1217773918, 'ino': 1099511631448, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-22.xml'}, {'damage_type': 'backtrace', 'id': 1217983230, 'ino': 1099511629175, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-18.xml'}, {'damage_type': 'backtrace', 'id': 1223770740, 'ino': 1099511630183, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 1285928666, 'ino': 1099511629336, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-40.jpg'}, {'damage_type': 'backtrace', 'id': 1301014493, 'ino': 1099511629339, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-38.jpg'}, {'damage_type': 'backtrace', 'id': 1311510592, 'ino': 1099511629360, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 1324731097, 'ino': 1099511630347, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 1345359408, 'ino': 1099511635503, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-42.jpg'}, {'damage_type': 'backtrace', 'id': 1358995717, 'ino': 1099511635235, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-42.xml'}, {'damage_type': 'backtrace', 'id': 1364293234, 'ino': 1099511635267, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-28.xml'}, {'damage_type': 'backtrace', 'id': 1366620370, 'ino': 1099511629372, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 1370714045, 'ino': 1099511635466, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-54.jpg'}, {'damage_type': 'backtrace', 'id': 1372951927, 'ino': 1099511630210, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 1383943694, 'ino': 1099511635231, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-44.xml'}, {'damage_type': 'backtrace', 'id': 1400761585, 'ino': 1099511635481, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-49.jpg'}, {'damage_type': 'backtrace', 'id': 1407239949, 'ino': 1099511635524, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 1407311415, 'ino': 1099511630132, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-25.xml'}, {'damage_type': 'backtrace', 'id': 1415308888, 'ino': 1099511635551, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-26.jpg'}, {'damage_type': 'backtrace', 'id': 1436979294, 'ino': 1099511629713, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-62.xml'}, {'damage_type': 'backtrace', 'id': 1443678436, 'ino': 1099511635307, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-16.xml'}, {'damage_type': 'backtrace', 'id': 1453668845, 'ino': 1099511629851, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-37.xml'}, {'damage_type': 'backtrace', 'id': 1458083545, 'ino': 1099511635213, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-53.xml'}, {'damage_type': 'backtrace', 'id': 1473766368, 'ino': 1099511635402, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-76.jpg'}, {'damage_type': 'backtrace', 'id': 1480041060, 'ino': 1099511629214, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-6.xml'}, {'damage_type': 'backtrace', 'id': 1485488476, 'ino': 1099511629269, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-83.jpg'}, {'damage_type': 'backtrace', 'id': 1504828445, 'ino': 1099511631426, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-20.xml'}, {'damage_type': 'backtrace', 'id': 1510093970, 'ino': 1099511635227, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-46.xml'}, {'damage_type': 'backtrace', 'id': 1526118002, 'ino': 1099511629974, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-12.xml'}, {'damage_type': 'backtrace', 'id': 1533932531, 'ino': 1099511629279, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-78.jpg'}, {'damage_type': 'backtrace', 'id': 1541047422, 'ino': 1099511629641, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-76.xml'}, {'damage_type': 'backtrace', 'id': 1568316844, 'ino': 1099511629171, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-19.xml'}, {'damage_type': 'backtrace', 'id': 1571657399, 'ino': 1099511635310, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-15.xml'}, {'damage_type': 'backtrace', 'id': 1577371123, 'ino': 1099511635539, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-30.jpg'}, {'damage_type': 'backtrace', 'id': 1577606096, 'ino': 1099511635475, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 1580329445, 'ino': 1099511635530, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 1600343293, 'ino': 1099511635355, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-86.jpg'}, {'damage_type': 'backtrace', 'id': 1615809173, 'ino': 1099511629806, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-46.xml'}, {'damage_type': 'backtrace', 'id': 1639593969, 'ino': 1099511629386, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 1654479996, 'ino': 1099511629946, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-18.xml'}, {'damage_type': 'backtrace', 'id': 1656789341, 'ino': 1099511630127, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-1.xml'}, {'damage_type': 'backtrace', 'id': 1660083827, 'ino': 1099511630156, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 1665227828, 'ino': 1099511629796, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-48.xml'}, {'damage_type': 'backtrace', 'id': 1680303391, 'ino': 1099511629811, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-45.xml'}, {'damage_type': 'backtrace', 'id': 1705658478, 'ino': 1099511635616, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 1725948260, 'ino': 1099511630118, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-2.xml'}, {'damage_type': 'backtrace', 'id': 1738371210, 'ino': 1099511635602, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 1743311664, 'ino': 1099511635581, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-16.jpg'}, {'damage_type': 'backtrace', 'id': 1749410692, 'ino': 1099511630437, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 1763589745, 'ino': 1099511635193, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-63.xml'}, {'damage_type': 'backtrace', 'id': 1770255683, 'ino': 1099511635493, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-45.jpg'}, {'damage_type': 'backtrace', 'id': 1782454674, 'ino': 1099511629612, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-81.xml'}, {'damage_type': 'backtrace', 'id': 1789026672, 'ino': 1099511631501, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-88.jpg'}, {'damage_type': 'backtrace', 'id': 1795329898, 'ino': 1099511629876, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-32.xml'}, {'damage_type': 'backtrace', 'id': 1795981212, 'ino': 1099511629891, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-29.xml'}, {'damage_type': 'backtrace', 'id': 1800050328, 'ino': 1099511629295, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 1807653151, 'ino': 1099511635487, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-47.jpg'}, {'damage_type': 'backtrace', 'id': 1807945322, 'ino': 1099511629363, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-22.jpg'}, {'damage_type': 'backtrace', 'id': 1822065368, 'ino': 1099511635500, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-43.jpg'}, {'damage_type': 'backtrace', 'id': 1824838163, 'ino': 1099511629374, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 1848544959, 'ino': 1099511630287, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 1856188820, 'ino': 1099511635334, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-7.xml'}, {'damage_type': 'backtrace', 'id': 1857569714, 'ino': 1099511635569, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 1875746856, 'ino': 1099511635410, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-74.jpg'}, {'damage_type': 'backtrace', 'id': 1895640951, 'ino': 1099511629400, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 1896143509, 'ino': 1099511629355, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 1899228795, 'ino': 1099511629232, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-1.xml'}, {'damage_type': 'backtrace', 'id': 1908593229, 'ino': 1099511629774, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-51.xml'}, {'damage_type': 'backtrace', 'id': 1909686973, 'ino': 1099511629662, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-72.xml'}, {'damage_type': 'backtrace', 'id': 1912144191, 'ino': 1099511635459, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-57.jpg'}, {'damage_type': 'backtrace', 'id': 1917575468, 'ino': 1099511629866, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-34.xml'}, {'damage_type': 'backtrace', 'id': 1918751011, 'ino': 1099511630265, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 1929537657, 'ino': 1099511635389, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-80.jpg'}, {'damage_type': 'backtrace', 'id': 1930520988, 'ino': 1099511631401, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-15.xml'}, {'damage_type': 'backtrace', 'id': 1942040613, 'ino': 1099511629236, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-97.jpg'}, {'damage_type': 'backtrace', 'id': 1966896137, 'ino': 1099511635386, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-81.jpg'}, {'damage_type': 'backtrace', 'id': 1970224640, 'ino': 1099511635512, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-39.jpg'}, {'damage_type': 'backtrace', 'id': 1976562987, 'ino': 1099511629368, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 1994143875, 'ino': 1099511630357, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 2009233159, 'ino': 1099511629331, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-43.jpg'}, {'damage_type': 'backtrace', 'id': 2020224319, 'ino': 1099511629366, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-20.jpg'}, {'damage_type': 'backtrace', 'id': 2033267070, 'ino': 1099511629392, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 2038665354, 'ino': 1099511629388, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 2042945960, 'ino': 1099511629277, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-79.jpg'}, {'damage_type': 'backtrace', 'id': 2050527967, 'ino': 1099511635497, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-44.jpg'}, {'damage_type': 'backtrace', 'id': 2058716029, 'ino': 1099511629789, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-49.xml'}, {'damage_type': 'backtrace', 'id': 2058958732, 'ino': 1099511631381, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-59.xml'}, {'damage_type': 'backtrace', 'id': 2107897848, 'ino': 1099511629192, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-13.xml'}, {'damage_type': 'backtrace', 'id': 2112067697, 'ino': 1099511635607, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 2128205912, 'ino': 1099511629380, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 2134478819, 'ino': 1099511635533, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 2138192974, 'ino': 1099511629826, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-42.xml'}, {'damage_type': 'backtrace', 'id': 2157868776, 'ino': 1099511635506, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-41.jpg'}, {'damage_type': 'backtrace', 'id': 2181747610, 'ino': 1099511635225, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-47.xml'}, {'damage_type': 'backtrace', 'id': 2187080727, 'ino': 1099511630201, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-27.jpg'}, {'damage_type': 'backtrace', 'id': 2187245537, 'ino': 1099511629260, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-88.jpg'}, {'damage_type': 'backtrace', 'id': 2205948747, 'ino': 1099511635542, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-29.jpg'}, {'damage_type': 'backtrace', 'id': 2229029594, 'ino': 1099511629390, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 2247170872, 'ino': 1099511629770, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-52.xml'}, {'damage_type': 'backtrace', 'id': 2250809164, 'ino': 1099511635188, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38'}, {'damage_type': 'backtrace', 'id': 2255003689, 'ino': 1099511629298, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-65.jpg'}, {'damage_type': 'backtrace', 'id': 2271125712, 'ino': 1099511635301, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-18.xml'}, {'damage_type': 'backtrace', 'id': 2288445668, 'ino': 1099511630165, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-31.jpg'}, {'damage_type': 'backtrace', 'id': 2329918148, 'ino': 1099511629761, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-54.xml'}, {'damage_type': 'backtrace', 'id': 2331362903, 'ino': 1099511629272, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-82.jpg'}, {'damage_type': 'backtrace', 'id': 2334009065, 'ino': 1099511635593, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-12.jpg'}, {'damage_type': 'backtrace', 'id': 2338410856, 'ino': 1099511629258, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-89.jpg'}, {'damage_type': 'backtrace', 'id': 2349787741, 'ino': 1099511629313, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-55.jpg'}, {'damage_type': 'backtrace', 'id': 2360046095, 'ino': 1099511629348, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-32.jpg'}, {'damage_type': 'backtrace', 'id': 2362050552, 'ino': 1099511635221, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-49.xml'}, {'damage_type': 'backtrace', 'id': 2384147838, 'ino': 1099511635288, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-21.xml'}, {'damage_type': 'backtrace', 'id': 2385616485, 'ino': 1099511629318, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-52.jpg'}, {'damage_type': 'backtrace', 'id': 2386334591, 'ino': 1099511629896, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-28.xml'}, {'damage_type': 'backtrace', 'id': 2394099559, 'ino': 1099511635521, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 2432546724, 'ino': 1099511630368, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 2434852679, 'ino': 1099511629667, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-71.xml'}, {'damage_type': 'backtrace', 'id': 2440798345, 'ino': 1099511629394, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 2461173686, 'ino': 1099511629239, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-96.jpg'}, {'damage_type': 'backtrace', 'id': 2520083214, 'ino': 1099511635545, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-28.jpg'}, {'damage_type': 'backtrace', 'id': 2528726729, 'ino': 1099511629856, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-36.xml'}, {'damage_type': 'backtrace', 'id': 2578151529, 'ino': 1099511635469, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 2578364653, 'ino': 1099511635207, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-56.xml'}, {'damage_type': 'backtrace', 'id': 2579572624, 'ino': 1099511629367, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 2587070502, 'ino': 1099511635264, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-29.xml'}, {'damage_type': 'backtrace', 'id': 2589951207, 'ino': 1099511635352, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-1.xml'}, {'damage_type': 'backtrace', 'id': 2603179791, 'ino': 1099511635255, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-32.xml'}, {'damage_type': 'backtrace', 'id': 2609234291, 'ino': 1099511635398, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-77.jpg'}, {'damage_type': 'backtrace', 'id': 2625378895, 'ino': 1099511635293, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-20.xml'}, {'damage_type': 'backtrace', 'id': 2637120006, 'ino': 1099511629378, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 2674446966, 'ino': 1099511629841, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-39.xml'}, {'damage_type': 'backtrace', 'id': 2675359981, 'ino': 1099511630023, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-3.xml'}, {'damage_type': 'backtrace', 'id': 2686501777, 'ino': 1099511629289, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-71.jpg'}, {'damage_type': 'backtrace', 'id': 2712648336, 'ino': 1099511629343, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-35.jpg'}, {'damage_type': 'backtrace', 'id': 2726989468, 'ino': 1099511635279, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-24.xml'}, {'damage_type': 'backtrace', 'id': 2743764131, 'ino': 1099511629634, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-77.xml'}, {'damage_type': 'backtrace', 'id': 2756869667, 'ino': 1099511630443, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 2762471129, 'ino': 1099511635379, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-83.jpg'}, {'damage_type': 'backtrace', 'id': 2767121118, 'ino': 1099511630415, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-6.jpg'}, {'damage_type': 'backtrace', 'id': 2787530854, 'ino': 1099511629303, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-62.jpg'}, {'damage_type': 'backtrace', 'id': 2794638946, 'ino': 1099511635590, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-13.jpg'}, {'damage_type': 'backtrace', 'id': 2797083752, 'ino': 1099511631817, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-98.xml'}, {'damage_type': 'backtrace', 'id': 2800214017, 'ino': 1099511629310, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-57.jpg'}, {'damage_type': 'backtrace', 'id': 2830652728, 'ino': 1099511629871, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-33.xml'}, {'damage_type': 'backtrace', 'id': 2832436113, 'ino': 1099511635258, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-31.xml'}, {'damage_type': 'backtrace', 'id': 2832647213, 'ino': 1099511629219, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-5.xml'}, {'damage_type': 'backtrace', 'id': 2838109480, 'ino': 1099511629652, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-74.xml'}, {'damage_type': 'backtrace', 'id': 2838453864, 'ino': 1099511630336, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-14.jpg'}, {'damage_type': 'backtrace', 'id': 2861505465, 'ino': 1099511630275, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 2861856633, 'ino': 1099511629286, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-73.jpg'}, {'damage_type': 'backtrace', 'id': 2871528762, 'ino': 1099511635241, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-39.xml'}, {'damage_type': 'backtrace', 'id': 2886438534, 'ino': 1099511635223, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-48.xml'}, {'damage_type': 'backtrace', 'id': 2889749768, 'ino': 1099511630218, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-25.jpg'}, {'damage_type': 'backtrace', 'id': 2903599470, 'ino': 1099511635527, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 2907514390, 'ino': 1099511629147, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6'}, {'damage_type': 'backtrace', 'id': 2908296435, 'ino': 1099511635420, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-71.jpg'}, {'damage_type': 'backtrace', 'id': 2909441302, 'ino': 1099511629956, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-16.xml'}, {'damage_type': 'backtrace', 'id': 2920534912, 'ino': 1099511629718, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-61.xml'}, {'damage_type': 'backtrace', 'id': 2931347543, 'ino': 1099511629292, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-69.jpg'}, {'damage_type': 'backtrace', 'id': 2941276759, 'ino': 1099511635619, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 2963797160, 'ino': 1099511629755, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-55.xml'}, {'damage_type': 'backtrace', 'id': 2969123708, 'ino': 1099511629816, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-44.xml'}, {'damage_type': 'backtrace', 'id': 2976934667, 'ino': 1099511635572, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-19.jpg'}, {'damage_type': 'backtrace', 'id': 2978014550, 'ino': 1099511629294, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-68.jpg'}, {'damage_type': 'backtrace', 'id': 2979254297, 'ino': 1099511635195, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-62.xml'}, {'damage_type': 'backtrace', 'id': 3006009392, 'ino': 1099511635313, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-14.xml'}, {'damage_type': 'backtrace', 'id': 3012573077, 'ino': 1099511629881, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-31.xml'}, {'damage_type': 'backtrace', 'id': 3020867338, 'ino': 1099511630144, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-34.jpg'}, {'damage_type': 'backtrace', 'id': 3027307385, 'ino': 1099511635622, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 3041322684, 'ino': 1099511629264, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-86.jpg'}, {'damage_type': 'backtrace', 'id': 3050248287, 'ino': 1099511635395, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-78.jpg'}, {'damage_type': 'backtrace', 'id': 3051020878, 'ino': 1099511629978, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-11.xml'}, {'damage_type': 'backtrace', 'id': 3064651448, 'ino': 1099511629275, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-80.jpg'}, {'damage_type': 'backtrace', 'id': 3070372257, 'ino': 1099511635625, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 3073710563, 'ino': 1099511630383, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-9.jpg'}, {'damage_type': 'backtrace', 'id': 3083378071, 'ino': 1099511629268, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-84.jpg'}, {'damage_type': 'backtrace', 'id': 3121481209, 'ino': 1099511629285, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-74.jpg'}, {'damage_type': 'backtrace', 'id': 3122139516, 'ino': 1099511629250, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-92.jpg'}, {'damage_type': 'backtrace', 'id': 3126791205, 'ino': 1099511635423, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-70.jpg'}, {'damage_type': 'backtrace', 'id': 3131223952, 'ino': 1099511635237, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-41.xml'}, {'damage_type': 'backtrace', 'id': 3132826901, 'ino': 1099511635596, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-11.jpg'}, {'damage_type': 'backtrace', 'id': 3147892768, 'ino': 1099511629692, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-66.xml'}, {'damage_type': 'backtrace', 'id': 3170695920, 'ino': 1099511635578, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-17.jpg'}, {'damage_type': 'backtrace', 'id': 3211663905, 'ino': 1099511630113, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-21.xml'}, {'damage_type': 'backtrace', 'id': 3220915958, 'ino': 1099511635478, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-50.jpg'}, {'damage_type': 'backtrace', 'id': 3241695974, 'ino': 1099511634443, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-33'}, {'damage_type': 'backtrace', 'id': 3243956402, 'ino': 1099511635215, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-52.xml'}, {'damage_type': 'backtrace', 'id': 3250174646, 'ino': 1099511635429, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-68.jpg'}, {'damage_type': 'backtrace', 'id': 3253415138, 'ino': 1099511629297, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-66.jpg'}, {'damage_type': 'backtrace', 'id': 3254065577, 'ino': 1099511629178, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-17.xml'}, {'damage_type': 'backtrace', 'id': 3262425812, 'ino': 1099511629165, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-21.xml'}, {'damage_type': 'backtrace', 'id': 3274923265, 'ino': 1099511629228, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-2.xml'}, {'damage_type': 'backtrace', 'id': 3284023489, 'ino': 1099511635285, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-22.xml'}, {'damage_type': 'backtrace', 'id': 3284688608, 'ino': 1099511629265, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-85.jpg'}, {'damage_type': 'backtrace', 'id': 3292376884, 'ino': 1099511629821, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-43.xml'}, {'damage_type': 'backtrace', 'id': 3300321080, 'ino': 1099511630374, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 3302200822, 'ino': 1099511629334, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-41.jpg'}, {'damage_type': 'backtrace', 'id': 3327197188, 'ino': 1099511635413, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-73.jpg'}, {'damage_type': 'backtrace', 'id': 3351798483, 'ino': 1099511629208, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-8.xml'}, {'damage_type': 'backtrace', 'id': 3361505416, 'ino': 1099511629188, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-14.xml'}, {'damage_type': 'backtrace', 'id': 3361703167, 'ino': 1099511629322, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-49.jpg'}, {'damage_type': 'backtrace', 'id': 3363447371, 'ino': 1099511629307, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-59.jpg'}, {'damage_type': 'backtrace', 'id': 3368939251, 'ino': 1099511635439, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-65.jpg'}, {'damage_type': 'backtrace', 'id': 3369967903, 'ino': 1099511635245, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-37.xml'}, {'damage_type': 'backtrace', 'id': 3371305140, 'ino': 1099511629312, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-56.jpg'}, {'damage_type': 'backtrace', 'id': 3401537864, 'ino': 1099511635449, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-61.jpg'}, {'damage_type': 'backtrace', 'id': 3402672624, 'ino': 1099511635405, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-75.jpg'}, {'damage_type': 'backtrace', 'id': 3403952856, 'ino': 1099511635273, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-26.xml'}, {'damage_type': 'backtrace', 'id': 3409412098, 'ino': 1099511635464, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-55.jpg'}, {'damage_type': 'backtrace', 'id': 3437802058, 'ino': 1099511629738, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-58.xml'}, {'damage_type': 'backtrace', 'id': 3440882651, 'ino': 1099511629262, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-87.jpg'}, {'damage_type': 'backtrace', 'id': 3449953156, 'ino': 1099511629623, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-79.xml'}, {'damage_type': 'backtrace', 'id': 3457373784, 'ino': 1099511635575, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-18.jpg'}, {'damage_type': 'backtrace', 'id': 3483388552, 'ino': 1099511629256, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-90.jpg'}, {'damage_type': 'backtrace', 'id': 3514655329, 'ino': 1099511635191, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-64.xml'}, {'damage_type': 'backtrace', 'id': 3541288348, 'ino': 1099511629696, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-65.xml'}, {'damage_type': 'backtrace', 'id': 3544548114, 'ino': 1099511630227, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-24.jpg'}, {'damage_type': 'backtrace', 'id': 3590811265, 'ino': 1099511630439, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-2.jpg'}, {'damage_type': 'backtrace', 'id': 3624355857, 'ino': 1099511629245, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-94.jpg'}, {'damage_type': 'backtrace', 'id': 3651877151, 'ino': 1099511629202, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-10.xml'}, {'damage_type': 'backtrace', 'id': 3658534595, 'ino': 1099511635426, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-69.jpg'}, {'damage_type': 'backtrace', 'id': 3710212193, 'ino': 1099511629900, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-27.xml'}, {'damage_type': 'backtrace', 'id': 3711519043, 'ino': 1099511635613, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-5.jpg'}, {'damage_type': 'backtrace', 'id': 3718492865, 'ino': 1099511629361, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-23.jpg'}, {'damage_type': 'backtrace', 'id': 3724749380, 'ino': 1099511629324, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-48.jpg'}, {'damage_type': 'backtrace', 'id': 3728381103, 'ino': 1099511629917, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-24.xml'}, {'damage_type': 'backtrace', 'id': 3774663865, 'ino': 1099511629223, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-4.xml'}, {'damage_type': 'backtrace', 'id': 3798503245, 'ino': 1099511629319, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-51.jpg'}, {'damage_type': 'backtrace', 'id': 3826927153, 'ino': 1099511629702, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-64.xml'}, {'damage_type': 'backtrace', 'id': 3843683581, 'ino': 1099511630008, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-5.xml'}, {'damage_type': 'backtrace', 'id': 3880435779, 'ino': 1099511635383, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-82.jpg'}, {'damage_type': 'backtrace', 'id': 3895780942, 'ino': 1099511629225, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-3.xml'}, {'damage_type': 'backtrace', 'id': 3904596907, 'ino': 1099511629801, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-47.xml'}, {'damage_type': 'backtrace', 'id': 3937777684, 'ino': 1099511635436, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-66.jpg'}, {'damage_type': 'backtrace', 'id': 3962290975, 'ino': 1099511629398, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-3.jpg'}, {'damage_type': 'backtrace', 'id': 3964508448, 'ino': 1099511630407, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-7.jpg'}, {'damage_type': 'backtrace', 'id': 3973031470, 'ino': 1099511629309, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-58.jpg'}, {'damage_type': 'backtrace', 'id': 3977872469, 'ino': 1099511629291, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-70.jpg'}, {'damage_type': 'backtrace', 'id': 3992690838, 'ino': 1099511630395, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-8.jpg'}, {'damage_type': 'backtrace', 'id': 3995456310, 'ino': 1099511630015, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-97.jpg'}, {'damage_type': 'backtrace', 'id': 3995912174, 'ino': 1099511629655, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-73.xml'}, {'damage_type': 'backtrace', 'id': 4008463584, 'ino': 1099511635199, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-60.xml'}, {'damage_type': 'backtrace', 'id': 4010125979, 'ino': 1099511629706, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-63.xml'}, {'damage_type': 'backtrace', 'id': 4011233943, 'ino': 1099511629321, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-50.jpg'}, {'damage_type': 'backtrace', 'id': 4012227243, 'ino': 1099511629905, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-26.xml'}, {'damage_type': 'backtrace', 'id': 4018376256, 'ino': 1099511629199, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/article-11.xml'}, {'damage_type': 'backtrace', 'id': 4022011590, 'ino': 1099511629922, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-23.xml'}, {'damage_type': 'backtrace', 'id': 4027675642, 'ino': 1099511629969, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-13.xml'}, {'damage_type': 'backtrace', 'id': 4028943389, 'ino': 1099511635270, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-27.xml'}, {'damage_type': 'backtrace', 'id': 4047854724, 'ino': 1099511635584, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-15.jpg'}, {'damage_type': 'backtrace', 'id': 4049419061, 'ino': 1099511635432, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-67.jpg'}, {'damage_type': 'backtrace', 'id': 4080576981, 'ino': 1099511629401, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-1.jpg'}, {'damage_type': 'backtrace', 'id': 4102681176, 'ino': 1099511629346, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-33.jpg'}, {'damage_type': 'backtrace', 'id': 4142885377, 'ino': 1099511629316, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-53.jpg'}, {'damage_type': 'backtrace', 'id': 4146478816, 'ino': 1099511630122, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-36.jpg'}, {'damage_type': 'backtrace', 'id': 4164446158, 'ino': 1099511630432, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/picture-4.jpg'}, {'damage_type': 'backtrace', 'id': 4225285190, 'ino': 1099511635600, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-10.jpg'}, {'damage_type': 'backtrace', 'id': 4227091838, 'ino': 1099511629861, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-10/article-35.xml'}, {'damage_type': 'backtrace', 'id': 4252797253, 'ino': 1099511629288, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-6/picture-72.jpg'}, {'damage_type': 'backtrace', 'id': 4265716777, 'ino': 1099511635462, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/picture-56.jpg'}, {'damage_type': 'backtrace', 'id': 4293548109, 'ino': 1099511635239, 'path': '/volumes/_nogroup/sv_1/0041f271-b242-4329-b233-848e4005d408/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-38/article-40.xml'}] 2023-10-16T14:41:01.488 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 79, in _scrub self._check_damage() File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 90, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2023-10-16T14:41:02.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:01 smithi178 ceph-mon[98495]: Adjusting osd_memory_target on smithi178 to 221.9M 2023-10-16T14:41:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:01 smithi178 ceph-mon[98495]: Unable to set osd_memory_target on smithi178 to 232737792: error parsing value: Value '232737792' is below minimum 939524096 2023-10-16T14:41:02.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:01 smithi178 ceph-mon[98495]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.1 MiB/s wr, 316 op/s 2023-10-16T14:41:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:01 smithi186 ceph-mon[134892]: Adjusting osd_memory_target on smithi178 to 221.9M 2023-10-16T14:41:02.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:01 smithi186 ceph-mon[134892]: Unable to set osd_memory_target on smithi178 to 232737792: error parsing value: Value '232737792' is below minimum 939524096 2023-10-16T14:41:02.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:01 smithi186 ceph-mon[134892]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.1 MiB/s wr, 316 op/s 2023-10-16T14:41:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:01 smithi187 ceph-mon[135297]: Adjusting osd_memory_target on smithi178 to 221.9M 2023-10-16T14:41:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:01 smithi187 ceph-mon[135297]: Unable to set osd_memory_target on smithi178 to 232737792: error parsing value: Value '232737792' is below minimum 939524096 2023-10-16T14:41:02.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:01 smithi187 ceph-mon[135297]: pgmap v558: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.1 MiB/s wr, 316 op/s 2023-10-16T14:41:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:04 smithi178 ceph-mon[98495]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.2 MiB/s wr, 360 op/s 2023-10-16T14:41:04.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:04 smithi186 ceph-mon[134892]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.2 MiB/s wr, 360 op/s 2023-10-16T14:41:04.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:04 smithi187 ceph-mon[135297]: pgmap v559: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 3.2 MiB/s wr, 360 op/s 2023-10-16T14:41:06.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:06 smithi178 ceph-mon[98495]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2023-10-16T14:41:06.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:06 smithi186 ceph-mon[134892]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2023-10-16T14:41:06.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:06 smithi187 ceph-mon[135297]: pgmap v560: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 425 op/s 2023-10-16T14:41:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:08 smithi178 ceph-mon[98495]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-10-16T14:41:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:08 smithi186 ceph-mon[134892]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-10-16T14:41:08.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:08 smithi187 ceph-mon[135297]: pgmap v561: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 538 op/s 2023-10-16T14:41:10.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:10 smithi178 ceph-mon[98495]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2023-10-16T14:41:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:10 smithi186 ceph-mon[134892]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2023-10-16T14:41:10.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:10 smithi187 ceph-mon[135297]: pgmap v562: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.2 MiB/s wr, 425 op/s 2023-10-16T14:41:11.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:11 smithi178 ceph-mon[98495]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 491 op/s 2023-10-16T14:41:11.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:11 smithi186 ceph-mon[134892]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 491 op/s 2023-10-16T14:41:11.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:11 smithi187 ceph-mon[135297]: pgmap v563: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 491 op/s 2023-10-16T14:41:14.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:13 smithi178 ceph-mon[98495]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2023-10-16T14:41:14.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:13 smithi186 ceph-mon[134892]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2023-10-16T14:41:14.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:13 smithi187 ceph-mon[135297]: pgmap v564: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2023-10-16T14:41:16.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:15 smithi178 ceph-mon[98495]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-10-16T14:41:16.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:15 smithi186 ceph-mon[134892]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-10-16T14:41:16.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:15 smithi187 ceph-mon[135297]: pgmap v565: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-10-16T14:41:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:17 smithi178 ceph-mon[98495]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2023-10-16T14:41:18.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:17 smithi186 ceph-mon[134892]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2023-10-16T14:41:18.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:17 smithi187 ceph-mon[135297]: pgmap v566: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 446 op/s 2023-10-16T14:41:20.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:19 smithi178 ceph-mon[98495]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2023-10-16T14:41:20.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:19 smithi186 ceph-mon[134892]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2023-10-16T14:41:20.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:19 smithi187 ceph-mon[135297]: pgmap v567: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 311 op/s 2023-10-16T14:41:21.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:21 smithi178 ceph-mon[98495]: pgmap v568: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2023-10-16T14:41:21.929 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:21 smithi187 ceph-mon[135297]: pgmap v568: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2023-10-16T14:41:21.929 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:21 smithi186 ceph-mon[134892]: pgmap v568: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 390 op/s 2023-10-16T14:41:24.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:23 smithi178 ceph-mon[98495]: pgmap v569: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2023-10-16T14:41:24.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:23 smithi186 ceph-mon[134892]: pgmap v569: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2023-10-16T14:41:24.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:23 smithi187 ceph-mon[135297]: pgmap v569: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 341 op/s 2023-10-16T14:41:26.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:25 smithi178 ceph-mon[98495]: pgmap v570: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:25 smithi186 ceph-mon[134892]: pgmap v570: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:26.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:25 smithi187 ceph-mon[135297]: pgmap v570: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:28.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:27 smithi178 ceph-mon[98495]: pgmap v571: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 392 op/s 2023-10-16T14:41:28.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:27 smithi186 ceph-mon[134892]: pgmap v571: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 392 op/s 2023-10-16T14:41:28.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:27 smithi187 ceph-mon[135297]: pgmap v571: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 392 op/s 2023-10-16T14:41:30.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:29 smithi178 ceph-mon[98495]: pgmap v572: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 295 op/s 2023-10-16T14:41:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:29 smithi186 ceph-mon[134892]: pgmap v572: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 295 op/s 2023-10-16T14:41:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:29 smithi187 ceph-mon[135297]: pgmap v572: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 295 op/s 2023-10-16T14:41:31.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:31 smithi178 ceph-mon[98495]: pgmap v573: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-16T14:41:31.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:31 smithi186 ceph-mon[134892]: pgmap v573: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-16T14:41:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:31 smithi187 ceph-mon[135297]: pgmap v573: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 383 op/s 2023-10-16T14:41:34.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:33 smithi178 ceph-mon[98495]: pgmap v574: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:34.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:33 smithi186 ceph-mon[134892]: pgmap v574: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:34.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:33 smithi187 ceph-mon[135297]: pgmap v574: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 335 op/s 2023-10-16T14:41:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:35 smithi178 ceph-mon[98495]: pgmap v575: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2023-10-16T14:41:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:35 smithi186 ceph-mon[134892]: pgmap v575: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2023-10-16T14:41:36.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:35 smithi187 ceph-mon[135297]: pgmap v575: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 340 op/s 2023-10-16T14:41:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:37 smithi178 ceph-mon[98495]: pgmap v576: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2023-10-16T14:41:38.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:37 smithi186 ceph-mon[134892]: pgmap v576: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2023-10-16T14:41:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:37 smithi187 ceph-mon[135297]: pgmap v576: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 381 op/s 2023-10-16T14:41:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:39 smithi178 ceph-mon[98495]: pgmap v577: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 288 op/s 2023-10-16T14:41:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:39 smithi186 ceph-mon[134892]: pgmap v577: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 288 op/s 2023-10-16T14:41:40.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:39 smithi187 ceph-mon[135297]: pgmap v577: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 288 op/s 2023-10-16T14:41:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:41 smithi178 ceph-mon[98495]: pgmap v578: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2023-10-16T14:41:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:41 smithi186 ceph-mon[134892]: pgmap v578: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2023-10-16T14:41:41.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:41 smithi187 ceph-mon[135297]: pgmap v578: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2023-10-16T14:41:44.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:43 smithi178 ceph-mon[98495]: pgmap v579: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2023-10-16T14:41:44.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:43 smithi186 ceph-mon[134892]: pgmap v579: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2023-10-16T14:41:44.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:43 smithi187 ceph-mon[135297]: pgmap v579: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 250 op/s 2023-10-16T14:41:46.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:45 smithi178 ceph-mon[98495]: pgmap v580: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-16T14:41:46.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:45 smithi186 ceph-mon[134892]: pgmap v580: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-16T14:41:46.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:45 smithi187 ceph-mon[135297]: pgmap v580: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 217 op/s 2023-10-16T14:41:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:47 smithi178 ceph-mon[98495]: pgmap v581: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2023-10-16T14:41:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:47 smithi186 ceph-mon[134892]: pgmap v581: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2023-10-16T14:41:48.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:47 smithi187 ceph-mon[135297]: pgmap v581: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2023-10-16T14:41:50.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:49 smithi178 ceph-mon[98495]: pgmap v582: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 169 op/s 2023-10-16T14:41:50.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:49 smithi186 ceph-mon[134892]: pgmap v582: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 169 op/s 2023-10-16T14:41:50.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:49 smithi187 ceph-mon[135297]: pgmap v582: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 169 op/s 2023-10-16T14:41:51.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:51 smithi178 ceph-mon[98495]: pgmap v583: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2023-10-16T14:41:51.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:51 smithi186 ceph-mon[134892]: pgmap v583: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2023-10-16T14:41:51.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:51 smithi187 ceph-mon[135297]: pgmap v583: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 371 op/s 2023-10-16T14:41:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:53 smithi178 ceph-mon[98495]: pgmap v584: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2023-10-16T14:41:54.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:53 smithi186 ceph-mon[134892]: pgmap v584: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2023-10-16T14:41:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:53 smithi187 ceph-mon[135297]: pgmap v584: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 393 op/s 2023-10-16T14:41:56.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:55 smithi178 ceph-mon[98495]: pgmap v585: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 410 op/s 2023-10-16T14:41:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:55 smithi186 ceph-mon[134892]: pgmap v585: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 410 op/s 2023-10-16T14:41:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:55 smithi187 ceph-mon[135297]: pgmap v585: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 410 op/s 2023-10-16T14:41:58.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:57 smithi178 ceph-mon[98495]: pgmap v586: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 500 op/s 2023-10-16T14:41:58.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:57 smithi186 ceph-mon[134892]: pgmap v586: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 500 op/s 2023-10-16T14:41:58.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:57 smithi187 ceph-mon[135297]: pgmap v586: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 500 op/s 2023-10-16T14:42:00.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:41:59 smithi178 ceph-mon[98495]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 470 op/s 2023-10-16T14:42:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:41:59 smithi186 ceph-mon[134892]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 470 op/s 2023-10-16T14:42:00.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:41:59 smithi187 ceph-mon[135297]: pgmap v587: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 470 op/s 2023-10-16T14:42:01.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:01 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:42:01.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:01 smithi178 ceph-mon[98495]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2023-10-16T14:42:01.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:01 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:42:01.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:01 smithi186 ceph-mon[134892]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2023-10-16T14:42:01.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:01 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:42:01.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:01 smithi187 ceph-mon[135297]: pgmap v588: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 534 op/s 2023-10-16T14:42:02.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:02 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:42:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:42:02.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:42:02.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:02 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:02.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:42:02.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:42:02.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:42:02.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:02 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:02.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:42:02.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:42:02.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:42:03.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:03 smithi178 ceph-mon[98495]: pgmap v589: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 480 op/s 2023-10-16T14:42:03.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:03 smithi186 ceph-mon[134892]: pgmap v589: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 480 op/s 2023-10-16T14:42:03.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:03 smithi187 ceph-mon[135297]: pgmap v589: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 480 op/s 2023-10-16T14:42:06.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:05 smithi178 ceph-mon[98495]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 584 op/s 2023-10-16T14:42:06.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:05 smithi186 ceph-mon[134892]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 584 op/s 2023-10-16T14:42:06.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:05 smithi187 ceph-mon[135297]: pgmap v590: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 584 op/s 2023-10-16T14:42:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:07 smithi178 ceph-mon[98495]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 906 op/s 2023-10-16T14:42:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:07 smithi186 ceph-mon[134892]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 906 op/s 2023-10-16T14:42:08.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:07 smithi187 ceph-mon[135297]: pgmap v591: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 906 op/s 2023-10-16T14:42:10.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:09 smithi178 ceph-mon[98495]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 839 op/s 2023-10-16T14:42:10.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:09 smithi186 ceph-mon[134892]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 839 op/s 2023-10-16T14:42:10.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:09 smithi187 ceph-mon[135297]: pgmap v592: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 839 op/s 2023-10-16T14:42:11.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:11 smithi178 ceph-mon[98495]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 984 op/s 2023-10-16T14:42:11.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:11 smithi186 ceph-mon[134892]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 984 op/s 2023-10-16T14:42:11.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:11 smithi187 ceph-mon[135297]: pgmap v593: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 984 op/s 2023-10-16T14:42:14.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:13 smithi178 ceph-mon[98495]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2023-10-16T14:42:14.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:13 smithi186 ceph-mon[134892]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2023-10-16T14:42:14.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:13 smithi187 ceph-mon[135297]: pgmap v594: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 929 op/s 2023-10-16T14:42:16.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:15 smithi178 ceph-mon[98495]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 808 op/s 2023-10-16T14:42:16.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:15 smithi186 ceph-mon[134892]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 808 op/s 2023-10-16T14:42:16.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:15 smithi187 ceph-mon[135297]: pgmap v595: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 808 op/s 2023-10-16T14:42:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:17 smithi178 ceph-mon[98495]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 751 op/s 2023-10-16T14:42:18.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:17 smithi186 ceph-mon[134892]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 751 op/s 2023-10-16T14:42:18.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:17 smithi187 ceph-mon[135297]: pgmap v596: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 751 op/s 2023-10-16T14:42:20.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:19 smithi178 ceph-mon[98495]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 413 op/s 2023-10-16T14:42:20.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:19 smithi186 ceph-mon[134892]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 413 op/s 2023-10-16T14:42:20.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:19 smithi187 ceph-mon[135297]: pgmap v597: 129 pgs: 129 active+clean; 1.5 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 413 op/s 2023-10-16T14:42:21.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:21 smithi178 ceph-mon[98495]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2023-10-16T14:42:21.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:21 smithi186 ceph-mon[134892]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2023-10-16T14:42:21.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:21 smithi187 ceph-mon[135297]: pgmap v598: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 531 op/s 2023-10-16T14:42:24.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:23 smithi178 ceph-mon[98495]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 442 op/s 2023-10-16T14:42:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:23 smithi186 ceph-mon[134892]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 442 op/s 2023-10-16T14:42:24.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:23 smithi187 ceph-mon[135297]: pgmap v599: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 442 op/s 2023-10-16T14:42:26.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:25 smithi178 ceph-mon[98495]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2023-10-16T14:42:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:25 smithi186 ceph-mon[134892]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2023-10-16T14:42:26.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:25 smithi187 ceph-mon[135297]: pgmap v600: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 467 op/s 2023-10-16T14:42:27.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:26 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:27.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:26 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:27.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:26 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:42:28.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:27 smithi178 ceph-mon[98495]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 547 op/s 2023-10-16T14:42:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:27 smithi186 ceph-mon[134892]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 547 op/s 2023-10-16T14:42:28.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:27 smithi187 ceph-mon[135297]: pgmap v601: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 547 op/s 2023-10-16T14:42:30.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:29 smithi178 ceph-mon[98495]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2023-10-16T14:42:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:29 smithi186 ceph-mon[134892]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2023-10-16T14:42:30.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:29 smithi187 ceph-mon[135297]: pgmap v602: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 445 op/s 2023-10-16T14:42:31.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:31 smithi178 ceph-mon[98495]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2023-10-16T14:42:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:31 smithi186 ceph-mon[134892]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2023-10-16T14:42:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:31 smithi187 ceph-mon[135297]: pgmap v603: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 508 op/s 2023-10-16T14:42:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:33 smithi178 ceph-mon[98495]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-10-16T14:42:34.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:33 smithi186 ceph-mon[134892]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-10-16T14:42:34.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:33 smithi187 ceph-mon[135297]: pgmap v604: 129 pgs: 129 active+clean; 1.5 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 402 op/s 2023-10-16T14:42:36.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:36 smithi178 ceph-mon[98495]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2023-10-16T14:42:36.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:36 smithi186 ceph-mon[134892]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2023-10-16T14:42:36.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:36 smithi187 ceph-mon[135297]: pgmap v605: 129 pgs: 129 active+clean; 1.4 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 497 op/s 2023-10-16T14:42:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:38 smithi178 ceph-mon[98495]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 775 op/s 2023-10-16T14:42:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:38 smithi186 ceph-mon[134892]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 775 op/s 2023-10-16T14:42:38.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:38 smithi187 ceph-mon[135297]: pgmap v606: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 775 op/s 2023-10-16T14:42:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:40 smithi178 ceph-mon[98495]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2023-10-16T14:42:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:40 smithi186 ceph-mon[134892]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2023-10-16T14:42:40.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:40 smithi187 ceph-mon[135297]: pgmap v607: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 700 op/s 2023-10-16T14:42:41.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:41 smithi178 ceph-mon[98495]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 788 op/s 2023-10-16T14:42:41.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:41 smithi186 ceph-mon[134892]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 788 op/s 2023-10-16T14:42:41.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:41 smithi187 ceph-mon[135297]: pgmap v608: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 788 op/s 2023-10-16T14:42:44.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:43 smithi178 ceph-mon[98495]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 756 op/s 2023-10-16T14:42:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:43 smithi186 ceph-mon[134892]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 756 op/s 2023-10-16T14:42:44.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:43 smithi187 ceph-mon[135297]: pgmap v609: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 756 op/s 2023-10-16T14:42:46.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:45 smithi178 ceph-mon[98495]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 795 op/s 2023-10-16T14:42:46.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:45 smithi186 ceph-mon[134892]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 795 op/s 2023-10-16T14:42:46.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:45 smithi187 ceph-mon[135297]: pgmap v610: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 795 op/s 2023-10-16T14:42:48.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:47 smithi178 ceph-mon[98495]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 777 op/s 2023-10-16T14:42:48.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:47 smithi186 ceph-mon[134892]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 777 op/s 2023-10-16T14:42:48.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:47 smithi187 ceph-mon[135297]: pgmap v611: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 777 op/s 2023-10-16T14:42:50.306 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:49 smithi178 ceph-mon[98495]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 500 op/s 2023-10-16T14:42:50.329 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:49 smithi186 ceph-mon[134892]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 500 op/s 2023-10-16T14:42:50.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:49 smithi187 ceph-mon[135297]: pgmap v612: 129 pgs: 129 active+clean; 1.4 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 500 op/s 2023-10-16T14:42:51.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:51 smithi178 ceph-mon[98495]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 634 op/s 2023-10-16T14:42:51.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:51 smithi186 ceph-mon[134892]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 634 op/s 2023-10-16T14:42:51.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:51 smithi187 ceph-mon[135297]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 634 op/s 2023-10-16T14:42:54.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:53 smithi178 ceph-mon[98495]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 575 op/s 2023-10-16T14:42:54.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:53 smithi186 ceph-mon[134892]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 575 op/s 2023-10-16T14:42:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:53 smithi187 ceph-mon[135297]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 575 op/s 2023-10-16T14:42:56.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:55 smithi178 ceph-mon[98495]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 599 op/s 2023-10-16T14:42:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:55 smithi186 ceph-mon[134892]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 599 op/s 2023-10-16T14:42:56.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:55 smithi187 ceph-mon[135297]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 599 op/s 2023-10-16T14:42:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:57 smithi178 ceph-mon[98495]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2023-10-16T14:42:58.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:57 smithi186 ceph-mon[134892]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2023-10-16T14:42:58.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:57 smithi187 ceph-mon[135297]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 607 op/s 2023-10-16T14:43:00.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:42:59 smithi178 ceph-mon[98495]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 476 op/s 2023-10-16T14:43:00.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:42:59 smithi186 ceph-mon[134892]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 476 op/s 2023-10-16T14:43:00.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:42:59 smithi187 ceph-mon[135297]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 971 KiB/s wr, 476 op/s 2023-10-16T14:43:01.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:01 smithi178 ceph-mon[98495]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2023-10-16T14:43:01.827 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:01 smithi186 ceph-mon[134892]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2023-10-16T14:43:01.926 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:01 smithi187 ceph-mon[135297]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 479 op/s 2023-10-16T14:43:03.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:02 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:43:03.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:02 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:43:03.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:02 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:43:04.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:03 smithi178 ceph-mon[98495]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2023-10-16T14:43:04.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:03 smithi186 ceph-mon[134892]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2023-10-16T14:43:04.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:03 smithi187 ceph-mon[135297]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 317 op/s 2023-10-16T14:43:05.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:04 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:43:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:04 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:43:05.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:04 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:43:05.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:04 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:43:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:04 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:43:05.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:04 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:43:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:04 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:43:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:04 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:43:05.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:04 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:43:06.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:05 smithi178 ceph-mon[98495]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-10-16T14:43:06.075 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:05 smithi186 ceph-mon[134892]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-10-16T14:43:06.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:05 smithi187 ceph-mon[135297]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-10-16T14:43:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:07 smithi178 ceph-mon[98495]: pgmap v621: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2023-10-16T14:43:08.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:07 smithi186 ceph-mon[134892]: pgmap v621: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2023-10-16T14:43:08.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:07 smithi187 ceph-mon[135297]: pgmap v621: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 317 op/s 2023-10-16T14:43:10.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:09 smithi178 ceph-mon[98495]: pgmap v622: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2023-10-16T14:43:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:09 smithi186 ceph-mon[134892]: pgmap v622: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2023-10-16T14:43:10.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:09 smithi187 ceph-mon[135297]: pgmap v622: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2023-10-16T14:43:11.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:11 smithi178 ceph-mon[98495]: pgmap v623: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2023-10-16T14:43:11.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:11 smithi186 ceph-mon[134892]: pgmap v623: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2023-10-16T14:43:11.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:11 smithi187 ceph-mon[135297]: pgmap v623: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 320 op/s 2023-10-16T14:43:14.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:13 smithi178 ceph-mon[98495]: pgmap v624: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2023-10-16T14:43:14.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:13 smithi186 ceph-mon[134892]: pgmap v624: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2023-10-16T14:43:14.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:13 smithi187 ceph-mon[135297]: pgmap v624: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2023-10-16T14:43:16.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:15 smithi178 ceph-mon[98495]: pgmap v625: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 308 op/s 2023-10-16T14:43:16.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:15 smithi186 ceph-mon[134892]: pgmap v625: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 308 op/s 2023-10-16T14:43:16.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:15 smithi187 ceph-mon[135297]: pgmap v625: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 308 op/s 2023-10-16T14:43:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:17 smithi178 ceph-mon[98495]: pgmap v626: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2023-10-16T14:43:18.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:17 smithi186 ceph-mon[134892]: pgmap v626: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2023-10-16T14:43:18.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:17 smithi187 ceph-mon[135297]: pgmap v626: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2023-10-16T14:43:20.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:19 smithi178 ceph-mon[98495]: pgmap v627: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 289 op/s 2023-10-16T14:43:20.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:19 smithi186 ceph-mon[134892]: pgmap v627: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 289 op/s 2023-10-16T14:43:20.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:19 smithi187 ceph-mon[135297]: pgmap v627: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 289 op/s 2023-10-16T14:43:21.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:21 smithi178 ceph-mon[98495]: pgmap v628: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 362 op/s 2023-10-16T14:43:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:21 smithi186 ceph-mon[134892]: pgmap v628: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 362 op/s 2023-10-16T14:43:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:21 smithi187 ceph-mon[135297]: pgmap v628: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 362 op/s 2023-10-16T14:43:24.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:23 smithi178 ceph-mon[98495]: pgmap v629: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2023-10-16T14:43:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:23 smithi186 ceph-mon[134892]: pgmap v629: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2023-10-16T14:43:24.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:23 smithi187 ceph-mon[135297]: pgmap v629: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 298 op/s 2023-10-16T14:43:26.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:25 smithi178 ceph-mon[98495]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2023-10-16T14:43:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:25 smithi186 ceph-mon[134892]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2023-10-16T14:43:26.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:25 smithi187 ceph-mon[135297]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 327 op/s 2023-10-16T14:43:28.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:27 smithi178 ceph-mon[98495]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 366 op/s 2023-10-16T14:43:28.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:27 smithi186 ceph-mon[134892]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 366 op/s 2023-10-16T14:43:28.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:27 smithi187 ceph-mon[135297]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 366 op/s 2023-10-16T14:43:30.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:29 smithi178 ceph-mon[98495]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-10-16T14:43:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:29 smithi186 ceph-mon[134892]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-10-16T14:43:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:29 smithi187 ceph-mon[135297]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-10-16T14:43:31.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:31 smithi178 ceph-mon[98495]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 373 op/s 2023-10-16T14:43:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:31 smithi186 ceph-mon[134892]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 373 op/s 2023-10-16T14:43:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:31 smithi187 ceph-mon[135297]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 373 op/s 2023-10-16T14:43:34.305 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:33 smithi178 ceph-mon[98495]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 806 KiB/s wr, 304 op/s 2023-10-16T14:43:34.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:33 smithi186 ceph-mon[134892]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 806 KiB/s wr, 304 op/s 2023-10-16T14:43:34.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:33 smithi187 ceph-mon[135297]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 806 KiB/s wr, 304 op/s 2023-10-16T14:43:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:35 smithi178 ceph-mon[98495]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2023-10-16T14:43:36.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:35 smithi186 ceph-mon[134892]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2023-10-16T14:43:36.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:35 smithi187 ceph-mon[135297]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 355 op/s 2023-10-16T14:43:38.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:38 smithi178 ceph-mon[98495]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 410 op/s 2023-10-16T14:43:38.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:38 smithi186 ceph-mon[134892]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 410 op/s 2023-10-16T14:43:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:38 smithi187 ceph-mon[135297]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 410 op/s 2023-10-16T14:43:40.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:40 smithi178 ceph-mon[98495]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2023-10-16T14:43:40.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:40 smithi186 ceph-mon[134892]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2023-10-16T14:43:40.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:40 smithi187 ceph-mon[135297]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 360 op/s 2023-10-16T14:43:41.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:41 smithi178 ceph-mon[98495]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 440 op/s 2023-10-16T14:43:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:41 smithi186 ceph-mon[134892]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 440 op/s 2023-10-16T14:43:41.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:41 smithi187 ceph-mon[135297]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 440 op/s 2023-10-16T14:43:44.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:43 smithi178 ceph-mon[98495]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2023-10-16T14:43:44.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:43 smithi186 ceph-mon[134892]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2023-10-16T14:43:44.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:43 smithi187 ceph-mon[135297]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 391 op/s 2023-10-16T14:43:46.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:46 smithi178 ceph-mon[98495]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 449 op/s 2023-10-16T14:43:46.328 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:46 smithi186 ceph-mon[134892]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 449 op/s 2023-10-16T14:43:46.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:46 smithi187 ceph-mon[135297]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 449 op/s 2023-10-16T14:43:48.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:48 smithi178 ceph-mon[98495]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 478 op/s 2023-10-16T14:43:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:48 smithi186 ceph-mon[134892]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 478 op/s 2023-10-16T14:43:48.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:48 smithi187 ceph-mon[135297]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 478 op/s 2023-10-16T14:43:50.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:50 smithi178 ceph-mon[98495]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2023-10-16T14:43:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:50 smithi186 ceph-mon[134892]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2023-10-16T14:43:50.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:50 smithi187 ceph-mon[135297]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 396 op/s 2023-10-16T14:43:51.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:51 smithi178 ceph-mon[98495]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2023-10-16T14:43:51.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:51 smithi186 ceph-mon[134892]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2023-10-16T14:43:51.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:51 smithi187 ceph-mon[135297]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 451 op/s 2023-10-16T14:43:54.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:53 smithi178 ceph-mon[98495]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2023-10-16T14:43:54.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:53 smithi186 ceph-mon[134892]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2023-10-16T14:43:54.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:53 smithi187 ceph-mon[135297]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 376 op/s 2023-10-16T14:43:56.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:55 smithi178 ceph-mon[98495]: pgmap v645: 129 pgs: 129 active+clean; 1017 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 408 op/s 2023-10-16T14:43:56.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:55 smithi186 ceph-mon[134892]: pgmap v645: 129 pgs: 129 active+clean; 1017 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 408 op/s 2023-10-16T14:43:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:55 smithi187 ceph-mon[135297]: pgmap v645: 129 pgs: 129 active+clean; 1017 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 408 op/s 2023-10-16T14:43:58.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:43:58 smithi178 ceph-mon[98495]: pgmap v646: 129 pgs: 129 active+clean; 986 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2023-10-16T14:43:58.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:43:58 smithi186 ceph-mon[134892]: pgmap v646: 129 pgs: 129 active+clean; 986 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2023-10-16T14:43:58.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:43:58 smithi187 ceph-mon[135297]: pgmap v646: 129 pgs: 129 active+clean; 986 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 410 op/s 2023-10-16T14:44:00.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:00 smithi178 ceph-mon[98495]: pgmap v647: 129 pgs: 129 active+clean; 981 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 341 op/s 2023-10-16T14:44:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:00 smithi186 ceph-mon[134892]: pgmap v647: 129 pgs: 129 active+clean; 981 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 341 op/s 2023-10-16T14:44:00.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:00 smithi187 ceph-mon[135297]: pgmap v647: 129 pgs: 129 active+clean; 981 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 341 op/s 2023-10-16T14:44:01.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:01 smithi178 ceph-mon[98495]: pgmap v648: 129 pgs: 129 active+clean; 971 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 412 op/s 2023-10-16T14:44:01.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:01 smithi186 ceph-mon[134892]: pgmap v648: 129 pgs: 129 active+clean; 971 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 412 op/s 2023-10-16T14:44:01.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:01 smithi187 ceph-mon[135297]: pgmap v648: 129 pgs: 129 active+clean; 971 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 412 op/s 2023-10-16T14:44:04.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:03 smithi178 ceph-mon[98495]: pgmap v649: 129 pgs: 129 active+clean; 965 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 359 op/s 2023-10-16T14:44:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:03 smithi186 ceph-mon[134892]: pgmap v649: 129 pgs: 129 active+clean; 965 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 359 op/s 2023-10-16T14:44:04.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:03 smithi187 ceph-mon[135297]: pgmap v649: 129 pgs: 129 active+clean; 965 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 359 op/s 2023-10-16T14:44:05.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:04 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:44:05.352 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:04 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:44:05.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:04 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:44:06.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:06 smithi178 ceph-mon[98495]: pgmap v650: 129 pgs: 129 active+clean; 962 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-10-16T14:44:06.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:06 smithi186 ceph-mon[134892]: pgmap v650: 129 pgs: 129 active+clean; 962 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-10-16T14:44:06.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:06 smithi187 ceph-mon[135297]: pgmap v650: 129 pgs: 129 active+clean; 962 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 409 op/s 2023-10-16T14:44:07.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:44:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:44:07.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:07 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:44:07.367 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:07 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:44:07.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:44:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:44:07.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:07 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:44:08.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:08 smithi178 ceph-mon[98495]: pgmap v651: 129 pgs: 129 active+clean; 948 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2023-10-16T14:44:08.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:08 smithi186 ceph-mon[134892]: pgmap v651: 129 pgs: 129 active+clean; 948 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2023-10-16T14:44:08.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:08 smithi187 ceph-mon[135297]: pgmap v651: 129 pgs: 129 active+clean; 948 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 450 op/s 2023-10-16T14:44:10.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:10 smithi178 ceph-mon[98495]: pgmap v652: 129 pgs: 129 active+clean; 949 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 428 op/s 2023-10-16T14:44:10.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:10 smithi187 ceph-mon[135297]: pgmap v652: 129 pgs: 129 active+clean; 949 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 428 op/s 2023-10-16T14:44:10.527 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:10 smithi186 ceph-mon[134892]: pgmap v652: 129 pgs: 129 active+clean; 949 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 428 op/s 2023-10-16T14:44:12.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:12 smithi178 ceph-mon[98495]: pgmap v653: 129 pgs: 129 active+clean; 933 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 630 op/s 2023-10-16T14:44:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:12 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:12 smithi186 ceph-mon[134892]: pgmap v653: 129 pgs: 129 active+clean; 933 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 630 op/s 2023-10-16T14:44:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:12 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:12.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:12 smithi187 ceph-mon[135297]: pgmap v653: 129 pgs: 129 active+clean; 933 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 630 op/s 2023-10-16T14:44:12.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:12 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:14.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:14 smithi178 ceph-mon[98495]: pgmap v654: 129 pgs: 129 active+clean; 926 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2023-10-16T14:44:14.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:14 smithi186 ceph-mon[134892]: pgmap v654: 129 pgs: 129 active+clean; 926 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2023-10-16T14:44:14.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:14 smithi187 ceph-mon[135297]: pgmap v654: 129 pgs: 129 active+clean; 926 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 582 op/s 2023-10-16T14:44:16.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:16 smithi178 ceph-mon[98495]: pgmap v655: 129 pgs: 129 active+clean; 921 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2023-10-16T14:44:16.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:16 smithi186 ceph-mon[134892]: pgmap v655: 129 pgs: 129 active+clean; 921 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2023-10-16T14:44:16.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:16 smithi187 ceph-mon[135297]: pgmap v655: 129 pgs: 129 active+clean; 921 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 660 op/s 2023-10-16T14:44:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:18 smithi178 ceph-mon[98495]: pgmap v656: 129 pgs: 129 active+clean; 910 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 727 op/s 2023-10-16T14:44:18.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:18 smithi186 ceph-mon[134892]: pgmap v656: 129 pgs: 129 active+clean; 910 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 727 op/s 2023-10-16T14:44:18.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:18 smithi187 ceph-mon[135297]: pgmap v656: 129 pgs: 129 active+clean; 910 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 727 op/s 2023-10-16T14:44:20.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:20 smithi186 ceph-mon[134892]: pgmap v657: 129 pgs: 129 active+clean; 905 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 674 op/s 2023-10-16T14:44:20.428 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:20 smithi187 ceph-mon[135297]: pgmap v657: 129 pgs: 129 active+clean; 905 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 674 op/s 2023-10-16T14:44:20.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:20 smithi178 ceph-mon[98495]: pgmap v657: 129 pgs: 129 active+clean; 905 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 674 op/s 2023-10-16T14:44:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:21 smithi186 ceph-mon[134892]: pgmap v658: 129 pgs: 129 active+clean; 898 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 739 op/s 2023-10-16T14:44:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:21 smithi187 ceph-mon[135297]: pgmap v658: 129 pgs: 129 active+clean; 898 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 739 op/s 2023-10-16T14:44:22.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:21 smithi178 ceph-mon[98495]: pgmap v658: 129 pgs: 129 active+clean; 898 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 739 op/s 2023-10-16T14:44:24.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:24 smithi178 ceph-mon[98495]: pgmap v659: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 575 op/s 2023-10-16T14:44:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:24 smithi186 ceph-mon[134892]: pgmap v659: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 575 op/s 2023-10-16T14:44:24.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:24 smithi187 ceph-mon[135297]: pgmap v659: 129 pgs: 129 active+clean; 891 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 575 op/s 2023-10-16T14:44:26.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:26 smithi178 ceph-mon[98495]: pgmap v660: 129 pgs: 129 active+clean; 890 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-10-16T14:44:26.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:26 smithi186 ceph-mon[134892]: pgmap v660: 129 pgs: 129 active+clean; 890 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-10-16T14:44:26.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:26 smithi187 ceph-mon[135297]: pgmap v660: 129 pgs: 129 active+clean; 890 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2023-10-16T14:44:27.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:27 smithi178 ceph-mon[98495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:27.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:27 smithi186 ceph-mon[134892]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:27.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:27 smithi187 ceph-mon[135297]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2023-10-16T14:44:28.304 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:28 smithi178 ceph-mon[98495]: pgmap v661: 129 pgs: 129 active+clean; 859 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 633 op/s 2023-10-16T14:44:28.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:28 smithi186 ceph-mon[134892]: pgmap v661: 129 pgs: 129 active+clean; 859 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 633 op/s 2023-10-16T14:44:28.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:28 smithi187 ceph-mon[135297]: pgmap v661: 129 pgs: 129 active+clean; 859 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 633 op/s 2023-10-16T14:44:30.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:30 smithi186 ceph-mon[134892]: pgmap v662: 129 pgs: 129 active+clean; 854 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2023-10-16T14:44:30.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:30 smithi187 ceph-mon[135297]: pgmap v662: 129 pgs: 129 active+clean; 854 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2023-10-16T14:44:30.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:30 smithi178 ceph-mon[98495]: pgmap v662: 129 pgs: 129 active+clean; 854 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 538 op/s 2023-10-16T14:44:31.804 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:31 smithi178 ceph-mon[98495]: pgmap v663: 129 pgs: 129 active+clean; 843 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 568 op/s 2023-10-16T14:44:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:31 smithi186 ceph-mon[134892]: pgmap v663: 129 pgs: 129 active+clean; 843 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 568 op/s 2023-10-16T14:44:31.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:31 smithi187 ceph-mon[135297]: pgmap v663: 129 pgs: 129 active+clean; 843 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 568 op/s 2023-10-16T14:44:34.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:34 smithi178 ceph-mon[98495]: pgmap v664: 129 pgs: 129 active+clean; 839 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 473 op/s 2023-10-16T14:44:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:34 smithi186 ceph-mon[134892]: pgmap v664: 129 pgs: 129 active+clean; 839 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 473 op/s 2023-10-16T14:44:34.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:34 smithi187 ceph-mon[135297]: pgmap v664: 129 pgs: 129 active+clean; 839 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 473 op/s 2023-10-16T14:44:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:36 smithi178 ceph-mon[98495]: pgmap v665: 129 pgs: 129 active+clean; 835 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-10-16T14:44:36.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:36 smithi186 ceph-mon[134892]: pgmap v665: 129 pgs: 129 active+clean; 835 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-10-16T14:44:36.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:36 smithi187 ceph-mon[135297]: pgmap v665: 129 pgs: 129 active+clean; 835 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 456 op/s 2023-10-16T14:44:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:38 smithi178 ceph-mon[98495]: pgmap v666: 129 pgs: 129 active+clean; 818 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2023-10-16T14:44:38.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:38 smithi186 ceph-mon[134892]: pgmap v666: 129 pgs: 129 active+clean; 818 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2023-10-16T14:44:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:38 smithi187 ceph-mon[135297]: pgmap v666: 129 pgs: 129 active+clean; 818 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2023-10-16T14:44:40.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:40 smithi178 ceph-mon[98495]: pgmap v667: 129 pgs: 129 active+clean; 805 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 353 op/s 2023-10-16T14:44:40.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:40 smithi186 ceph-mon[134892]: pgmap v667: 129 pgs: 129 active+clean; 805 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 353 op/s 2023-10-16T14:44:40.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:40 smithi187 ceph-mon[135297]: pgmap v667: 129 pgs: 129 active+clean; 805 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 353 op/s 2023-10-16T14:44:41.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:41 smithi178 ceph-mon[98495]: pgmap v668: 129 pgs: 129 active+clean; 776 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 387 op/s 2023-10-16T14:44:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:41 smithi186 ceph-mon[134892]: pgmap v668: 129 pgs: 129 active+clean; 776 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 387 op/s 2023-10-16T14:44:41.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:41 smithi187 ceph-mon[135297]: pgmap v668: 129 pgs: 129 active+clean; 776 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 387 op/s 2023-10-16T14:44:44.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:44 smithi178 ceph-mon[98495]: pgmap v669: 129 pgs: 129 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 336 op/s 2023-10-16T14:44:44.327 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:44 smithi186 ceph-mon[134892]: pgmap v669: 129 pgs: 129 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 336 op/s 2023-10-16T14:44:44.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:44 smithi187 ceph-mon[135297]: pgmap v669: 129 pgs: 129 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 336 op/s 2023-10-16T14:44:46.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:46 smithi178 ceph-mon[98495]: pgmap v670: 129 pgs: 129 active+clean; 764 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 364 op/s 2023-10-16T14:44:46.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:46 smithi186 ceph-mon[134892]: pgmap v670: 129 pgs: 129 active+clean; 764 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 364 op/s 2023-10-16T14:44:46.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:46 smithi187 ceph-mon[135297]: pgmap v670: 129 pgs: 129 active+clean; 764 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 364 op/s 2023-10-16T14:44:48.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:48 smithi178 ceph-mon[98495]: pgmap v671: 129 pgs: 129 active+clean; 747 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2023-10-16T14:44:48.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:48 smithi186 ceph-mon[134892]: pgmap v671: 129 pgs: 129 active+clean; 747 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2023-10-16T14:44:48.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:48 smithi187 ceph-mon[135297]: pgmap v671: 129 pgs: 129 active+clean; 747 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2023-10-16T14:44:50.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:50 smithi178 ceph-mon[98495]: pgmap v672: 129 pgs: 129 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2023-10-16T14:44:50.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:50 smithi186 ceph-mon[134892]: pgmap v672: 129 pgs: 129 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2023-10-16T14:44:50.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:50 smithi187 ceph-mon[135297]: pgmap v672: 129 pgs: 129 active+clean; 745 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 349 op/s 2023-10-16T14:44:51.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:51 smithi178 ceph-mon[98495]: pgmap v673: 129 pgs: 129 active+clean; 738 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2023-10-16T14:44:51.825 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:51 smithi186 ceph-mon[134892]: pgmap v673: 129 pgs: 129 active+clean; 738 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2023-10-16T14:44:51.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:51 smithi187 ceph-mon[135297]: pgmap v673: 129 pgs: 129 active+clean; 738 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 360 op/s 2023-10-16T14:44:54.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:54 smithi178 ceph-mon[98495]: pgmap v674: 129 pgs: 129 active+clean; 737 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 826 KiB/s wr, 278 op/s 2023-10-16T14:44:54.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:54 smithi186 ceph-mon[134892]: pgmap v674: 129 pgs: 129 active+clean; 737 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 826 KiB/s wr, 278 op/s 2023-10-16T14:44:54.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:54 smithi187 ceph-mon[135297]: pgmap v674: 129 pgs: 129 active+clean; 737 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 826 KiB/s wr, 278 op/s 2023-10-16T14:44:56.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:56 smithi178 ceph-mon[98495]: pgmap v675: 129 pgs: 129 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 837 KiB/s wr, 258 op/s 2023-10-16T14:44:56.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:56 smithi186 ceph-mon[134892]: pgmap v675: 129 pgs: 129 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 837 KiB/s wr, 258 op/s 2023-10-16T14:44:56.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:56 smithi187 ceph-mon[135297]: pgmap v675: 129 pgs: 129 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 837 KiB/s wr, 258 op/s 2023-10-16T14:44:58.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:44:58 smithi178 ceph-mon[98495]: pgmap v676: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 755 KiB/s wr, 267 op/s 2023-10-16T14:44:58.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:44:58 smithi186 ceph-mon[134892]: pgmap v676: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 755 KiB/s wr, 267 op/s 2023-10-16T14:44:58.427 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:44:58 smithi187 ceph-mon[135297]: pgmap v676: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 755 KiB/s wr, 267 op/s 2023-10-16T14:45:00.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:00 smithi178 ceph-mon[98495]: pgmap v677: 129 pgs: 129 active+clean; 702 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-10-16T14:45:00.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:00 smithi186 ceph-mon[134892]: pgmap v677: 129 pgs: 129 active+clean; 702 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-10-16T14:45:00.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:00 smithi187 ceph-mon[135297]: pgmap v677: 129 pgs: 129 active+clean; 702 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 873 KiB/s rd, 754 KiB/s wr, 225 op/s 2023-10-16T14:45:01.163 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2023-10-16T14:45:01.164 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-16T14:45:01.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:01 smithi178 ceph-mon[98495]: pgmap v678: 129 pgs: 129 active+clean; 686 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 284 op/s 2023-10-16T14:45:01.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:01 smithi186 ceph-mon[134892]: pgmap v678: 129 pgs: 129 active+clean; 686 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 284 op/s 2023-10-16T14:45:01.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:01 smithi187 ceph-mon[135297]: pgmap v678: 129 pgs: 129 active+clean; 686 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 284 op/s 2023-10-16T14:45:02.477 DEBUG:teuthology.parallel:result is None 2023-10-16T14:45:02.478 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-10-16T14:45:02.560 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-10-16T14:45:02.560 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-10-16T14:45:02.571 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-10-16T14:45:02.571 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_33be745ec4531df08cd1bb70b4c9c1f89be58078/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2023-10-16T14:45:02.572 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-10-16T14:45:02.626 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:45:02.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/805608187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 msgr2=0x7fa6c8109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:02.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/805608187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c8109360 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa6c000b0a0 tx=0x7fa6c000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.827 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/805608187 shutdown_connections 2023-10-16T14:45:02.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/805608187 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c8109360 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/805608187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/805608187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 0x7fa6c8102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.828 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/805608187 >> 172.21.15.178:0/805608187 conn(0x7fa6c80fc420 msgr2=0x7fa6c80fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:02.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/805608187 shutdown_connections 2023-10-16T14:45:02.829 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.824+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/805608187 wait complete. 2023-10-16T14:45:02.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 Processor -- start 2023-10-16T14:45:02.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 -- start start 2023-10-16T14:45:02.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:02.832 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 0x7fa6c8197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:02.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:02.833 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819e780 con 0x7fa6c8102080 2023-10-16T14:45:02.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819e8f0 con 0x7fa6c8102a40 2023-10-16T14:45:02.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.825+0000 7fa6cd3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c819eab0 con 0x7fa6c8100ea0 2023-10-16T14:45:02.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:02.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:48368/0 (socket says 172.21.15.178:48368) 2023-10-16T14:45:02.834 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 -- 172.21.15.178:0/1982266938 learned_addr learned my addr 172.21.15.178:0/1982266938 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c67fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 0x7fa6c8197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c77fe700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 -- 172.21.15.178:0/1982266938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 msgr2=0x7fa6c819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 -- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 msgr2=0x7fa6c8197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:02.835 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 0x7fa6c8197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:02.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 -- 172.21.15.178:0/1982266938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009520 con 0x7fa6c8100ea0 2023-10-16T14:45:02.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6c6ffd700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e970 tx=0x7fa6b800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:02.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8009790 con 0x7fa6c8100ea0 2023-10-16T14:45:02.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c0009d00 con 0x7fa6c8100ea0 2023-10-16T14:45:02.836 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6b800c7e0 con 0x7fa6c8100ea0 2023-10-16T14:45:02.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.826+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8003c70 con 0x7fa6c8100ea0 2023-10-16T14:45:02.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.827+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c819f160 con 0x7fa6c8100ea0 2023-10-16T14:45:02.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.827+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8107d60 con 0x7fa6c8100ea0 2023-10-16T14:45:02.837 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.830+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fa6b800cc50 con 0x7fa6c8100ea0 2023-10-16T14:45:02.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.831+0000 7fa6affff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 0x7fa6b00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:02.838 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.831+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa6b8014070 con 0x7fa6c8100ea0 2023-10-16T14:45:02.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.831+0000 7fa6c67fc700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 0x7fa6b00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:02.839 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.832+0000 7fa6c67fc700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 0x7fa6b00a39e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8100bc0 tx=0x7fa6bc009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:02.840 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:02.833+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fa6b8096660 con 0x7fa6c8100ea0 2023-10-16T14:45:03.043 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.041+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6c810c9c0 con 0x7fa6c8100ea0 2023-10-16T14:45:03.045 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.043+0000 7fa6affff700 1 -- 172.21.15.178:0/1982266938 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fa6b8003dd0 con 0x7fa6c8100ea0 2023-10-16T14:45:03.046 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:03.046 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:45:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 msgr2=0x7fa6b00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 0x7fa6b00a39e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8100bc0 tx=0x7fa6bc009260 comp rx=0 tx=0).stop 2023-10-16T14:45:03.049 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 msgr2=0x7fa6c8196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800e970 tx=0x7fa6b800ec80 comp rx=0 tx=0).stop 2023-10-16T14:45:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 shutdown_connections 2023-10-16T14:45:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fa6b00a1520 0x7fa6b00a39e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fa6c8102a40 0x7fa6c819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.050 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6c8100ea0 0x7fa6c8196c40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 --2- 172.21.15.178:0/1982266938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6c8102080 0x7fa6c8197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 >> 172.21.15.178:0/1982266938 conn(0x7fa6c80fc420 msgr2=0x7fa6c80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 shutdown_connections 2023-10-16T14:45:03.051 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.046+0000 7fa6cd3ff700 1 -- 172.21.15.178:0/1982266938 wait complete. 2023-10-16T14:45:03.069 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:45:03.069 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:45:03.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 -- 172.21.15.178:0/314775941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa00ffe00 msgr2=0x7f7fa0100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.272 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/314775941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0100220 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7f880099c0 tx=0x7f7f88009cd0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 -- 172.21.15.178:0/314775941 shutdown_connections 2023-10-16T14:45:03.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/314775941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa0101990 0x7f7fa0108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/314775941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0100220 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/314775941 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0100ff0 0x7f7fa0101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.273 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.269+0000 7f7fa51ff700 1 -- 172.21.15.178:0/314775941 >> 172.21.15.178:0/314775941 conn(0x7f7fa00fb370 msgr2=0x7f7fa00fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:03.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.270+0000 7f7fa51ff700 1 -- 172.21.15.178:0/314775941 shutdown_connections 2023-10-16T14:45:03.274 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.270+0000 7f7fa51ff700 1 -- 172.21.15.178:0/314775941 wait complete. 2023-10-16T14:45:03.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.270+0000 7f7fa51ff700 1 Processor -- start 2023-10-16T14:45:03.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 -- start start 2023-10-16T14:45:03.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0101990 0x7f7fa01a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a0a10 con 0x7f7fa0100ff0 2023-10-16T14:45:03.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a0b50 con 0x7f7fa0101990 2023-10-16T14:45:03.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7fa51ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa01a0ce0 con 0x7f7fa00ffe00 2023-10-16T14:45:03.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7f9e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7f9e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60394/0 (socket says 172.21.15.178:60394) 2023-10-16T14:45:03.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7f9e59c700 1 -- 172.21.15.178:0/67956368 learned_addr learned my addr 172.21.15.178:0/67956368 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:03.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7f9ed9d700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.271+0000 7f7f9f59e700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0101990 0x7f7fa01a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.278 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 -- 172.21.15.178:0/67956368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa00ffe00 msgr2=0x7f7fa0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.279 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 -- 172.21.15.178:0/67956368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0101990 msgr2=0x7f7fa01a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0101990 0x7f7fa01a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 -- 172.21.15.178:0/67956368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009520 con 0x7f7fa0100ff0 2023-10-16T14:45:03.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f9e59c700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000eb30 tx=0x7f7f9000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:03.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90009880 con 0x7f7fa0100ff0 2023-10-16T14:45:03.280 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f900044e0 con 0x7f7fa0100ff0 2023-10-16T14:45:03.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f880095f0 con 0x7f7fa0100ff0 2023-10-16T14:45:03.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003d80 con 0x7f7fa0100ff0 2023-10-16T14:45:03.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.272+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa01a1300 con 0x7f7fa0100ff0 2023-10-16T14:45:03.281 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.273+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa010b660 con 0x7f7fa0100ff0 2023-10-16T14:45:03.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.275+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f7f90003f20 con 0x7f7fa0100ff0 2023-10-16T14:45:03.282 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.277+0000 7f7f97fff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 0x7f7f8c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.278+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7f90014070 con 0x7f7fa0100ff0 2023-10-16T14:45:03.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.278+0000 7f7f9ed9d700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 0x7f7f8c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.283 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.278+0000 7f7f9ed9d700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 0x7f7f8c0a39c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009990 tx=0x7f7f88005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:03.284 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.282+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f7f90096840 con 0x7f7fa0100ff0 2023-10-16T14:45:03.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.481+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fa01a1550 con 0x7f7fa0100ff0 2023-10-16T14:45:03.484 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.481+0000 7f7f97fff700 1 -- 172.21.15.178:0/67956368 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f90099e60 con 0x7f7fa0100ff0 2023-10-16T14:45:03.484 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:45:03.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.483+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 msgr2=0x7f7f8c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.486 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.483+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 0x7f7f8c0a39c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009990 tx=0x7f7f88005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.483+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 msgr2=0x7f7fa01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.483+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000eb30 tx=0x7f7f9000ee40 comp rx=0 tx=0).stop 2023-10-16T14:45:03.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 shutdown_connections 2023-10-16T14:45:03.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f7f8c0a1500 0x7f7f8c0a39c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f7fa0101990 0x7f7fa01a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fa00ffe00 0x7f7fa0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 --2- 172.21.15.178:0/67956368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fa0100ff0 0x7f7fa01993d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 >> 172.21.15.178:0/67956368 conn(0x7f7fa00fb370 msgr2=0x7f7fa0069000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:03.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 shutdown_connections 2023-10-16T14:45:03.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.484+0000 7f7fa51ff700 1 -- 172.21.15.178:0/67956368 wait complete. 2023-10-16T14:45:03.505 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:45:03.707 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 -- 172.21.15.178:0/3083536564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102060 msgr2=0x7f27c81024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 --2- 172.21.15.178:0/3083536564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102060 0x7f27c81024e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009990 tx=0x7f27b8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 -- 172.21.15.178:0/3083536564 shutdown_connections 2023-10-16T14:45:03.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 --2- 172.21.15.178:0/3083536564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102a20 0x7f27c8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.708 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 --2- 172.21.15.178:0/3083536564 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 0x7f27c81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 --2- 172.21.15.178:0/3083536564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102060 0x7f27c81024e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 -- 172.21.15.178:0/3083536564 >> 172.21.15.178:0/3083536564 conn(0x7f27c80fc400 msgr2=0x7f27c80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:03.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 -- 172.21.15.178:0/3083536564 shutdown_connections 2023-10-16T14:45:03.709 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.705+0000 7f27cfdff700 1 -- 172.21.15.178:0/3083536564 wait complete. 2023-10-16T14:45:03.711 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.706+0000 7f27cfdff700 1 Processor -- start 2023-10-16T14:45:03.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.706+0000 7f27cfdff700 1 -- start start 2023-10-16T14:45:03.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 0x7f27c8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102060 0x7f27c8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.712 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a09b0 con 0x7f27c8102a20 2023-10-16T14:45:03.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a0af0 con 0x7f27c8100e80 2023-10-16T14:45:03.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cfdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81a0c80 con 0x7f27c8102060 2023-10-16T14:45:03.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27ce39c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27ce39c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60406/0 (socket says 172.21.15.178:60406) 2023-10-16T14:45:03.713 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27ce39c700 1 -- 172.21.15.178:0/4072795342 learned_addr learned my addr 172.21.15.178:0/4072795342 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:03.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cd39a700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102060 0x7f27c8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.707+0000 7f27cdb9b700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 0x7f27c8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 -- 172.21.15.178:0/4072795342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102060 msgr2=0x7f27c8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102060 0x7f27c8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.714 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 -- 172.21.15.178:0/4072795342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 msgr2=0x7f27c8198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 0x7f27c8198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 -- 172.21.15.178:0/4072795342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80095f0 con 0x7f27c8102a20 2023-10-16T14:45:03.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27ce39c700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef30 tx=0x7f27c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:03.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0018070 con 0x7f27c8102a20 2023-10-16T14:45:03.715 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27c000f040 con 0x7f27c8102a20 2023-10-16T14:45:03.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c81a0f00 con 0x7f27c8102a20 2023-10-16T14:45:03.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c000cd00 con 0x7f27c8102a20 2023-10-16T14:45:03.716 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.708+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c81a1380 con 0x7f27c8102a20 2023-10-16T14:45:03.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.709+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c810c740 con 0x7f27c8102a20 2023-10-16T14:45:03.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.711+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f27c0003a00 con 0x7f27c8102a20 2023-10-16T14:45:03.717 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.713+0000 7f27beffd700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 0x7f27b40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:03.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.713+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f27c00ccbc0 con 0x7f27c8102a20 2023-10-16T14:45:03.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.714+0000 7f27cdb9b700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 0x7f27b40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:03.718 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.714+0000 7f27cdb9b700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 0x7f27b40a39c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c8101bd0 tx=0x7f27c4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:03.720 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.718+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f27c0095ee0 con 0x7f27c8102a20 2023-10-16T14:45:03.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.922+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27c81a1570 con 0x7f27c8102a20 2023-10-16T14:45:03.925 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.923+0000 7f27beffd700 1 -- 172.21.15.178:0/4072795342 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27c0099500 con 0x7f27c8102a20 2023-10-16T14:45:03.926 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:45:03.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 msgr2=0x7f27b40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 0x7f27b40a39c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f27c8101bd0 tx=0x7f27c4009260 comp rx=0 tx=0).stop 2023-10-16T14:45:03.928 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 msgr2=0x7f27c81a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:03.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f27c000ef30 tx=0x7f27c000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 shutdown_connections 2023-10-16T14:45:03.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f27b40a1500 0x7f27b40a39c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27c8102a20 0x7f27c81a03e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.929 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f27c8100e80 0x7f27c8198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 --2- 172.21.15.178:0/4072795342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27c8102060 0x7f27c8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:03.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.925+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 >> 172.21.15.178:0/4072795342 conn(0x7f27c80fc400 msgr2=0x7f27c80fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:03.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.926+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 shutdown_connections 2023-10-16T14:45:03.930 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:03.926+0000 7f27cfdff700 1 -- 172.21.15.178:0/4072795342 wait complete. 2023-10-16T14:45:03.947 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:45:04.150 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.147+0000 7f5d855ff700 1 -- 172.21.15.178:0/88504222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 msgr2=0x7f5d80109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.147+0000 7f5d855ff700 1 --2- 172.21.15.178:0/88504222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d80109360 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f90 tx=0x7f5d7800b330 comp rx=0 tx=0).stop 2023-10-16T14:45:04.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 -- 172.21.15.178:0/88504222 shutdown_connections 2023-10-16T14:45:04.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 --2- 172.21.15.178:0/88504222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d80109360 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 --2- 172.21.15.178:0/88504222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80100ea0 0x7f5d801012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 --2- 172.21.15.178:0/88504222 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80102080 0x7f5d80102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 -- 172.21.15.178:0/88504222 >> 172.21.15.178:0/88504222 conn(0x7f5d800fc420 msgr2=0x7f5d800fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:04.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 -- 172.21.15.178:0/88504222 shutdown_connections 2023-10-16T14:45:04.152 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.148+0000 7f5d855ff700 1 -- 172.21.15.178:0/88504222 wait complete. 2023-10-16T14:45:04.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 Processor -- start 2023-10-16T14:45:04.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 -- start start 2023-10-16T14:45:04.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 0x7f5d80198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 0x7f5d80199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a0ab0 con 0x7f5d80102a40 2023-10-16T14:45:04.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a0bf0 con 0x7f5d80100ea0 2023-10-16T14:45:04.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d855ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d801a0d80 con 0x7f5d80102080 2023-10-16T14:45:04.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d7f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d7f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60426/0 (socket says 172.21.15.178:60426) 2023-10-16T14:45:04.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d7f7fe700 1 -- 172.21.15.178:0/814933225 learned_addr learned my addr 172.21.15.178:0/814933225 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:04.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.149+0000 7f5d7effd700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 0x7f5d80198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7e7fc700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 0x7f5d80199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 -- 172.21.15.178:0/814933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 msgr2=0x7f5d80199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 0x7f5d80199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 -- 172.21.15.178:0/814933225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 msgr2=0x7f5d80198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 0x7f5d80198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 -- 172.21.15.178:0/814933225 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d700095f0 con 0x7f5d80102a40 2023-10-16T14:45:04.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7e7fc700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 0x7f5d80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:45:04.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7effd700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 0x7f5d80198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:45:04.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d7f7fe700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f60 tx=0x7f5d78004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:04.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7800e040 con 0x7f5d80102a40 2023-10-16T14:45:04.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d7800bbf0 con 0x7f5d80102a40 2023-10-16T14:45:04.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d78009c40 con 0x7f5d80102a40 2023-10-16T14:45:04.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780052a0 con 0x7f5d80102a40 2023-10-16T14:45:04.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.150+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d801a12f0 con 0x7f5d80102a40 2023-10-16T14:45:04.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.151+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8010c760 con 0x7f5d80102a40 2023-10-16T14:45:04.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.153+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f5d780054c0 con 0x7f5d80102a40 2023-10-16T14:45:04.160 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.154+0000 7f5d67fff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 0x7f5d680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.155+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5d780ce660 con 0x7f5d80102a40 2023-10-16T14:45:04.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.155+0000 7f5d7effd700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 0x7f5d680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.161 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.155+0000 7f5d7effd700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 0x7f5d680a3940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5d80101bf0 tx=0x7f5d70009310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:04.162 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.158+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f5d78097980 con 0x7f5d80102a40 2023-10-16T14:45:04.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:04 smithi178 ceph-mon[98495]: pgmap v679: 129 pgs: 129 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 989 KiB/s wr, 284 op/s 2023-10-16T14:45:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1982266938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/67956368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:04 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4072795342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:04 smithi186 ceph-mon[134892]: pgmap v679: 129 pgs: 129 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 989 KiB/s wr, 284 op/s 2023-10-16T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1982266938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/67956368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:04 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4072795342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.364 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.362+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d80067d40 con 0x7f5d80102a40 2023-10-16T14:45:04.365 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.362+0000 7f5d67fff700 1 -- 172.21.15.178:0/814933225 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d80067d40 con 0x7f5d80102a40 2023-10-16T14:45:04.365 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:45:04.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 msgr2=0x7f5d680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 0x7f5d680a3940 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5d80101bf0 tx=0x7f5d70009310 comp rx=0 tx=0).stop 2023-10-16T14:45:04.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 msgr2=0x7f5d801a04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d78009f60 tx=0x7f5d78004d10 comp rx=0 tx=0).stop 2023-10-16T14:45:04.368 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 shutdown_connections 2023-10-16T14:45:04.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f5d680a1480 0x7f5d680a3940 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d80102a40 0x7f5d801a04e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f5d80100ea0 0x7f5d80198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 --2- 172.21.15.178:0/814933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d80102080 0x7f5d80199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.369 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 >> 172.21.15.178:0/814933225 conn(0x7f5d800fc420 msgr2=0x7f5d800fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:04.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 shutdown_connections 2023-10-16T14:45:04.370 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.365+0000 7f5d855ff700 1 -- 172.21.15.178:0/814933225 wait complete. 2023-10-16T14:45:04.388 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:45:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:04 smithi187 ceph-mon[135297]: pgmap v679: 129 pgs: 129 active+clean; 679 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 989 KiB/s wr, 284 op/s 2023-10-16T14:45:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1982266938' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:04.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/67956368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:04 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4072795342' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:04.590 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 -- 172.21.15.178:0/101990523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28100ea0 msgr2=0x7fda281012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/101990523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28100ea0 0x7fda281012c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fda18009990 tx=0x7fda18009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 -- 172.21.15.178:0/101990523 shutdown_connections 2023-10-16T14:45:04.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/101990523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28102a40 0x7fda28109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.591 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/101990523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28100ea0 0x7fda281012c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/101990523 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 0x7fda28102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 -- 172.21.15.178:0/101990523 >> 172.21.15.178:0/101990523 conn(0x7fda280fc420 msgr2=0x7fda280fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:04.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 -- 172.21.15.178:0/101990523 shutdown_connections 2023-10-16T14:45:04.592 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.588+0000 7fda2d9ff700 1 -- 172.21.15.178:0/101990523 wait complete. 2023-10-16T14:45:04.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.589+0000 7fda2d9ff700 1 Processor -- start 2023-10-16T14:45:04.594 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 -- start start 2023-10-16T14:45:04.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ea0 0x7fda28074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 0x7fda280732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28073f80 con 0x7fda28102a40 2023-10-16T14:45:04.595 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280740f0 con 0x7fda28102080 2023-10-16T14:45:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda2d9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281a0d10 con 0x7fda28100ea0 2023-10-16T14:45:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda277fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda277fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:60456/0 (socket says 172.21.15.178:60456) 2023-10-16T14:45:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda277fe700 1 -- 172.21.15.178:0/1033034778 learned_addr learned my addr 172.21.15.178:0/1033034778 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:04.596 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda26ffd700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ea0 0x7fda28074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.590+0000 7fda267fc700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 0x7fda280732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 -- 172.21.15.178:0/1033034778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ea0 msgr2=0x7fda28074c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ea0 0x7fda28074c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 -- 172.21.15.178:0/1033034778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 msgr2=0x7fda280732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 0x7fda280732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 -- 172.21.15.178:0/1033034778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180095f0 con 0x7fda28102a40 2023-10-16T14:45:04.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda277fe700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ef30 tx=0x7fda2000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:04.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20018070 con 0x7fda28102a40 2023-10-16T14:45:04.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda2000f040 con 0x7fda28102a40 2023-10-16T14:45:04.598 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda281a0f90 con 0x7fda28102a40 2023-10-16T14:45:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2000cd00 con 0x7fda28102a40 2023-10-16T14:45:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.591+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda281a1410 con 0x7fda28102a40 2023-10-16T14:45:04.599 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.592+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2810c760 con 0x7fda28102a40 2023-10-16T14:45:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.594+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fda20003a00 con 0x7fda28102a40 2023-10-16T14:45:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.596+0000 7fda2c9fd700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 0x7fda100a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.596+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fda200cd3e0 con 0x7fda28102a40 2023-10-16T14:45:04.600 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.596+0000 7fda26ffd700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 0x7fda100a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:04.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.597+0000 7fda26ffd700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 0x7fda100a39c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fda1800b2e0 tx=0x7fda18005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:04.601 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.598+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fda20096760 con 0x7fda28102a40 2023-10-16T14:45:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.798+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda28107ee0 con 0x7fda28102a40 2023-10-16T14:45:04.801 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.798+0000 7fda2c9fd700 1 -- 172.21.15.178:0/1033034778 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda20099d80 con 0x7fda28102a40 2023-10-16T14:45:04.802 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:45:04.803 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 msgr2=0x7fda100a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 0x7fda100a39c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fda1800b2e0 tx=0x7fda18005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 msgr2=0x7fda280737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:04.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda2000ef30 tx=0x7fda2000c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.804 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 shutdown_connections 2023-10-16T14:45:04.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fda100a1500 0x7fda100a39c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda28102a40 0x7fda280737e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ea0 0x7fda28074c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 --2- 172.21.15.178:0/1033034778 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fda28102080 0x7fda280732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:04.805 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 >> 172.21.15.178:0/1033034778 conn(0x7fda280fc420 msgr2=0x7fda280fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:04.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 shutdown_connections 2023-10-16T14:45:04.806 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:04.801+0000 7fda2d9ff700 1 -- 172.21.15.178:0/1033034778 wait complete. 2023-10-16T14:45:04.823 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-10-16T14:45:05.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 -- 172.21.15.178:0/2057799224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0073910 msgr2=0x7f29d0073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:05.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 --2- 172.21.15.178:0/2057799224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0073910 0x7f29d0073d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009990 tx=0x7f29c0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.024 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 -- 172.21.15.178:0/2057799224 shutdown_connections 2023-10-16T14:45:05.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 --2- 172.21.15.178:0/2057799224 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d010b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 --2- 172.21.15.178:0/2057799224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0074ef0 0x7f29d0073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 --2- 172.21.15.178:0/2057799224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0073910 0x7f29d0073d90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 -- 172.21.15.178:0/2057799224 >> 172.21.15.178:0/2057799224 conn(0x7f29d00fc430 msgr2=0x7f29d00fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:05.025 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 -- 172.21.15.178:0/2057799224 shutdown_connections 2023-10-16T14:45:05.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.018+0000 7f29d73ff700 1 -- 172.21.15.178:0/2057799224 wait complete. 2023-10-16T14:45:05.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.019+0000 7f29d73ff700 1 Processor -- start 2023-10-16T14:45:05.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.019+0000 7f29d73ff700 1 -- start start 2023-10-16T14:45:05.026 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:05.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 0x7f29d01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:05.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:05.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a08f0 con 0x7f29d0074ef0 2023-10-16T14:45:05.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a0a60 con 0x7f29d0108dc0 2023-10-16T14:45:05.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d73ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01a0c20 con 0x7f29d0073910 2023-10-16T14:45:05.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:05.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d499a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 0x7f29d01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:05.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d599c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:05.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d519b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:48470/0 (socket says 172.21.15.178:48470) 2023-10-16T14:45:05.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d599c700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:56714/0 (socket says 172.21.15.178:56714) 2023-10-16T14:45:05.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d519b700 1 -- 172.21.15.178:0/851248313 learned_addr learned my addr 172.21.15.178:0/851248313 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:05.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d519b700 1 -- 172.21.15.178:0/851248313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 msgr2=0x7f29d01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:05.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.020+0000 7f29d519b700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d519b700 1 -- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 msgr2=0x7f29d01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:05.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d519b700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 0x7f29d01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:05.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d519b700 1 -- 172.21.15.178:0/851248313 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8009c40 con 0x7f29d0073910 2023-10-16T14:45:05.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d499a700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 0x7f29d01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:05.030 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d599c700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d519b700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9a0 tx=0x7f29cc00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc009790 con 0x7f29d0073910 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29cc00cbd0 con 0x7f29d0073910 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00095f0 con 0x7f29d0073910 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc018680 con 0x7f29d0073910 2023-10-16T14:45:05.031 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.021+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d01a1210 con 0x7f29d0073910 2023-10-16T14:45:05.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.022+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d0107470 con 0x7f29d0073910 2023-10-16T14:45:05.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.024+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f29cc00cd40 con 0x7f29d0073910 2023-10-16T14:45:05.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.025+0000 7f29c67fc700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 0x7f29bc0a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:05.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.025+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f29cc014070 con 0x7f29d0073910 2023-10-16T14:45:05.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.026+0000 7f29d499a700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 0x7f29bc0a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:05.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.026+0000 7f29d499a700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 0x7f29bc0a3a60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f29c000b2e0 tx=0x7f29c0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:05.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.030+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f29cc052100 con 0x7f29d0073910 2023-10-16T14:45:05.292 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.290+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f29d010ebc0 con 0x7f29d0073910 2023-10-16T14:45:05.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/814933225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:05.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:05 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1033034778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:05.303 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.300+0000 7f29c67fc700 1 -- 172.21.15.178:0/851248313 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733880 (secure 0 0 0) 0x7f29cc096530 con 0x7f29d0073910 2023-10-16T14:45:05.312 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:05.312 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "epoch": 18, 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "active_gid": 14148, 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "active_name": "x", 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "active_addrs": { 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2023-10-16T14:45:05.313 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2023-10-16T14:45:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6800", 2023-10-16T14:45:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 706630784 2023-10-16T14:45:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.314 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2023-10-16T14:45:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:6801", 2023-10-16T14:45:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 706630784 2023-10-16T14:45:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.315 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "active_addr": "172.21.15.178:6801/706630784", 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "active_change": "2023-10-16T14:25:06.501968+0000", 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "active_mgr_features": 4540138322906710015, 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "available": true, 2023-10-16T14:45:05.316 INFO:teuthology.orchestra.run.smithi178.stdout: "standbys": [ 2023-10-16T14:45:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "gid": 24109, 2023-10-16T14:45:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "z", 2023-10-16T14:45:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr_features": 4540138322906710015, 2023-10-16T14:45:05.317 INFO:teuthology.orchestra.run.smithi178.stdout: "available_modules": [ 2023-10-16T14:45:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "alerts", 2023-10-16T14:45:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.318 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.319 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How frequently to reexamine health status", 2023-10-16T14:45:05.320 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.321 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.322 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.323 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.324 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.324 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.324 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.324 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.324 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.325 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.326 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.326 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.326 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.326 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.326 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.328 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.328 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.328 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.328 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.328 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.329 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.329 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.329 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.329 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.329 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.330 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.330 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.330 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.330 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.330 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.332 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.332 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.332 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.332 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.333 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_destination": { 2023-10-16T14:45:05.334 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_destination", 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.335 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.336 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.336 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email address to send alerts to", 2023-10-16T14:45:05.336 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.336 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.336 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_from_name": { 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_from_name", 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.337 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "Ceph", 2023-10-16T14:45:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.338 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email From: name", 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_host": { 2023-10-16T14:45:05.339 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_host", 2023-10-16T14:45:05.340 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.340 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.340 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.340 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.340 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP server", 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.341 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.342 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.342 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_password": { 2023-10-16T14:45:05.342 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_password", 2023-10-16T14:45:05.342 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.342 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.343 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.343 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.343 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.343 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Password to authenticate with", 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.344 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_port": { 2023-10-16T14:45:05.345 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_port", 2023-10-16T14:45:05.345 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.345 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.345 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.345 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "465", 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP port", 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.346 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.347 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.347 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.347 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_sender": { 2023-10-16T14:45:05.347 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_sender", 2023-10-16T14:45:05.347 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.348 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.349 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP envelope sender", 2023-10-16T14:45:05.349 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.349 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.349 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.350 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_ssl": { 2023-10-16T14:45:05.350 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_ssl", 2023-10-16T14:45:05.350 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.350 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.350 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.351 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.351 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.351 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.351 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.351 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-16T14:45:05.352 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.352 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.352 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.352 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_user": { 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_user", 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.353 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "User to authenticate as", 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.354 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.355 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.355 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "balancer", 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.356 INFO:teuthology.orchestra.run.smithi178.stdout: "active": { 2023-10-16T14:45:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "active", 2023-10-16T14:45:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically balance PGs across cluster", 2023-10-16T14:45:05.358 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_time": { 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_time", 2023-10-16T14:45:05.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0000", 2023-10-16T14:45:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.360 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "beginning time of day to automatically balance", 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_weekday": { 2023-10-16T14:45:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_weekday", 2023-10-16T14:45:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.362 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-16T14:45:05.363 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:05.364 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.364 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.364 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_max_iterations": { 2023-10-16T14:45:05.364 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_max_iterations", 2023-10-16T14:45:05.365 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:05.365 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.365 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.365 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "25", 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "250", 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.366 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.367 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.367 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.367 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_metrics": { 2023-10-16T14:45:05.367 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_metrics", 2023-10-16T14:45:05.367 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.368 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.368 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.368 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "pgs,objects,bytes", 2023-10-16T14:45:05.368 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.368 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.370 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_step": { 2023-10-16T14:45:05.370 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_step", 2023-10-16T14:45:05.370 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:05.370 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.370 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.371 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.5", 2023-10-16T14:45:05.371 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0.001", 2023-10-16T14:45:05.371 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "0.999", 2023-10-16T14:45:05.371 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.371 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "aggressiveness of optimization", 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: "end_time": { 2023-10-16T14:45:05.372 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_time", 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2359", 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.373 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.374 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.374 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ending time of day to automatically balance", 2023-10-16T14:45:05.374 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:05.374 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.374 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: "end_weekday": { 2023-10-16T14:45:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_weekday", 2023-10-16T14:45:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:05.375 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.376 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-16T14:45:05.377 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:05.377 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.377 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.377 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.378 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.379 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.379 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.379 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.379 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.379 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.380 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.381 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.382 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.382 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.382 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.382 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.383 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.384 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.384 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.384 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.384 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.384 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.385 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.385 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.385 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.385 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.385 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.386 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.386 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.386 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.386 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.386 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.387 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.388 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.388 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.388 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.388 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.389 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.389 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.389 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.389 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.389 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.390 INFO:teuthology.orchestra.run.smithi178.stdout: "min_score": { 2023-10-16T14:45:05.391 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_score", 2023-10-16T14:45:05.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:05.391 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.391 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.391 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:05.392 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.392 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.392 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.392 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-16T14:45:05.392 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.393 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.393 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.393 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:05.393 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "upmap", 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.394 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: "crush-compat", 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: "none", 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap" 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.395 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Balancer mode", 2023-10-16T14:45:05.396 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.396 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.396 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.396 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.396 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_ids": { 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_ids", 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.397 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-16T14:45:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.398 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.399 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.399 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:05.399 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:05.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.400 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.401 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_deviation": { 2023-10-16T14:45:05.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_deviation", 2023-10-16T14:45:05.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.402 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.402 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.402 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-16T14:45:05.403 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_optimizations": { 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_optimizations", 2023-10-16T14:45:05.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:05.405 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.405 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.405 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.405 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-16T14:45:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.406 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm", 2023-10-16T14:45:05.407 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.408 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.408 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.408 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_down_multiplier": { 2023-10-16T14:45:05.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_down_multiplier", 2023-10-16T14:45:05.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.409 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.410 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-16T14:45:05.410 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.410 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.410 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_refresh_rate": { 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_refresh_rate", 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.411 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "20", 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.412 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.413 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.413 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.413 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_starting_port": { 2023-10-16T14:45:05.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_starting_port", 2023-10-16T14:45:05.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4721", 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.414 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.415 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-16T14:45:05.415 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.415 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.415 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.416 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_ptrace": { 2023-10-16T14:45:05.416 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_ptrace", 2023-10-16T14:45:05.416 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.416 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.416 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-16T14:45:05.417 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-16T14:45:05.418 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.418 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.418 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.418 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_interval": { 2023-10-16T14:45:05.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_interval", 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.419 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.420 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.420 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-16T14:45:05.420 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.420 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.420 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_memory_target_ratio": { 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_memory_target_ratio", 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.421 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.7", 2023-10-16T14:45:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.422 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm_log_destination": { 2023-10-16T14:45:05.423 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm_log_destination", 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.424 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.425 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.425 INFO:teuthology.orchestra.run.smithi178.stdout: "file", 2023-10-16T14:45:05.425 INFO:teuthology.orchestra.run.smithi178.stdout: "file,syslog", 2023-10-16T14:45:05.425 INFO:teuthology.orchestra.run.smithi178.stdout: "syslog" 2023-10-16T14:45:05.425 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.426 INFO:teuthology.orchestra.run.smithi178.stdout: "cgroups_split": { 2023-10-16T14:45:05.427 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cgroups_split", 2023-10-16T14:45:05.427 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.427 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.427 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.427 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.428 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.429 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.429 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.429 INFO:teuthology.orchestra.run.smithi178.stdout: "config_checks_enabled": { 2023-10-16T14:45:05.429 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_checks_enabled", 2023-10-16T14:45:05.429 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.430 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.430 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.430 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.430 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.430 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.432 INFO:teuthology.orchestra.run.smithi178.stdout: "config_dashboard": { 2023-10-16T14:45:05.432 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_dashboard", 2023-10-16T14:45:05.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.432 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.432 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-16T14:45:05.433 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.434 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.434 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.434 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_alertmanager": { 2023-10-16T14:45:05.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_alertmanager", 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.435 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.436 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.436 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:05.436 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.436 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.436 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_base": { 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_base", 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.437 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.438 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-16T14:45:05.438 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.438 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.438 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.438 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Container image name, without the tag", 2023-10-16T14:45:05.439 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.439 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.439 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.439 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_elasticsearch": { 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_elasticsearch", 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-16T14:45:05.440 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.441 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.441 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.441 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "elasticsearch container image", 2023-10-16T14:45:05.441 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.441 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.442 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.442 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_grafana": { 2023-10-16T14:45:05.442 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_grafana", 2023-10-16T14:45:05.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.443 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.443 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.443 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-16T14:45:05.443 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.445 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.445 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.445 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:05.445 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.446 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.446 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.446 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_haproxy": { 2023-10-16T14:45:05.446 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_haproxy", 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.447 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.448 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.448 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "HAproxy container image", 2023-10-16T14:45:05.448 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.448 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_agent": { 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_agent", 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.449 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.450 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.450 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-16T14:45:05.450 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.450 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.450 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger agent container image", 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.451 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_collector": { 2023-10-16T14:45:05.452 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_collector", 2023-10-16T14:45:05.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.452 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.452 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.452 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger collector container image", 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.453 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.454 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.454 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_query": { 2023-10-16T14:45:05.454 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_query", 2023-10-16T14:45:05.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.455 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.455 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.455 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-16T14:45:05.455 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger query container image", 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.456 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_keepalived": { 2023-10-16T14:45:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_keepalived", 2023-10-16T14:45:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.457 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.458 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Keepalived container image", 2023-10-16T14:45:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.459 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_loki": { 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_loki", 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-16T14:45:05.460 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Loki container image", 2023-10-16T14:45:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.461 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_node_exporter": { 2023-10-16T14:45:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_node_exporter", 2023-10-16T14:45:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.463 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_nvmeof": { 2023-10-16T14:45:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_nvmeof", 2023-10-16T14:45:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.465 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-16T14:45:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.466 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Nvme-of container image", 2023-10-16T14:45:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.467 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_prometheus": { 2023-10-16T14:45:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_prometheus", 2023-10-16T14:45:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.468 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.469 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_promtail": { 2023-10-16T14:45:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_promtail", 2023-10-16T14:45:05.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.471 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Promtail container image", 2023-10-16T14:45:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_snmp_gateway": { 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_snmp_gateway", 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.473 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-16T14:45:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SNMP Gateway container image", 2023-10-16T14:45:05.474 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "container_init": { 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_init", 2023-10-16T14:45:05.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.476 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Run podman/docker with `--init`", 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "daemon_cache_timeout": { 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "daemon_cache_timeout", 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.478 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.479 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-16T14:45:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: "default_cephadm_command_timeout": { 2023-10-16T14:45:05.481 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_cephadm_command_timeout", 2023-10-16T14:45:05.481 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.481 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.481 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.481 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "900", 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.482 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.483 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.483 INFO:teuthology.orchestra.run.smithi178.stdout: "default_registry": { 2023-10-16T14:45:05.483 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_registry", 2023-10-16T14:45:05.483 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io", 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.484 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.485 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-16T14:45:05.485 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.485 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.485 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.485 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.486 INFO:teuthology.orchestra.run.smithi178.stdout: "device_cache_timeout": { 2023-10-16T14:45:05.486 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_cache_timeout", 2023-10-16T14:45:05.486 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.486 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.486 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.487 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1800", 2023-10-16T14:45:05.487 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.487 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.487 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.487 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache device inventory", 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: "device_enhanced_scan": { 2023-10-16T14:45:05.488 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_enhanced_scan", 2023-10-16T14:45:05.489 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.489 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.489 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.489 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.489 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.490 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.491 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.491 INFO:teuthology.orchestra.run.smithi178.stdout: "facts_cache_timeout": { 2023-10-16T14:45:05.491 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "facts_cache_timeout", 2023-10-16T14:45:05.491 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.491 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.492 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache host facts data", 2023-10-16T14:45:05.493 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.493 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.493 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.493 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.493 INFO:teuthology.orchestra.run.smithi178.stdout: "host_check_interval": { 2023-10-16T14:45:05.494 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "host_check_interval", 2023-10-16T14:45:05.494 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.494 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.494 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.494 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to perform a host check", 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.495 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: "inventory_list_all": { 2023-10-16T14:45:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "inventory_list_all", 2023-10-16T14:45:05.496 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.497 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.498 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.499 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.499 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.499 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.499 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.499 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.500 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.501 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.501 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.501 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.501 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.501 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.502 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.503 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.503 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.503 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.503 INFO:teuthology.orchestra.run.smithi178.stdout: "log_refresh_metadata": { 2023-10-16T14:45:05.503 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_refresh_metadata", 2023-10-16T14:45:05.504 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.504 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.504 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.504 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.504 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.505 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.506 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.506 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.506 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.506 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.507 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.507 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.507 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.507 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.508 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.508 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-16T14:45:05.508 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.508 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.508 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.509 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.510 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.510 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.510 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.510 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.510 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.511 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.512 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.512 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.512 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.512 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.513 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.513 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.513 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.513 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.513 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.514 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.514 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.514 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.514 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.514 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.515 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.516 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.517 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.517 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.517 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-16T14:45:05.517 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.517 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.518 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.518 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.518 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-16T14:45:05.518 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-16T14:45:05.518 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.519 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.519 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.519 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "*", 2023-10-16T14:45:05.519 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.520 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.520 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.520 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-16T14:45:05.520 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.521 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.521 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.521 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.522 INFO:teuthology.orchestra.run.smithi178.stdout: "max_count_per_host": { 2023-10-16T14:45:05.522 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_count_per_host", 2023-10-16T14:45:05.522 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.522 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.523 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of daemons per service per host", 2023-10-16T14:45:05.524 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.524 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.524 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.524 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.524 INFO:teuthology.orchestra.run.smithi178.stdout: "max_osd_draining_count": { 2023-10-16T14:45:05.525 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_osd_draining_count", 2023-10-16T14:45:05.525 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.525 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.525 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.525 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.526 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.526 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.526 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.526 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-16T14:45:05.526 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.527 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.527 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.527 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.527 INFO:teuthology.orchestra.run.smithi178.stdout: "migration_current": { 2023-10-16T14:45:05.527 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "migration_current", 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.528 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.529 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.529 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "internal - do not modify", 2023-10-16T14:45:05.529 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.529 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.529 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.530 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.530 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:05.530 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:05.530 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.530 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "root", 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.531 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm-package", 2023-10-16T14:45:05.532 INFO:teuthology.orchestra.run.smithi178.stdout: "root" 2023-10-16T14:45:05.532 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.532 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "mode for remote execution of cephadm", 2023-10-16T14:45:05.532 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.532 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.533 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.533 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.533 INFO:teuthology.orchestra.run.smithi178.stdout: "prometheus_alerts_path": { 2023-10-16T14:45:05.533 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus_alerts_path", 2023-10-16T14:45:05.533 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.534 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.535 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-16T14:45:05.535 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.535 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.535 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.536 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_insecure": { 2023-10-16T14:45:05.536 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_insecure", 2023-10-16T14:45:05.536 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.536 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.536 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-16T14:45:05.537 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.538 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.538 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.538 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_password": { 2023-10-16T14:45:05.538 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_password", 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.539 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-16T14:45:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.540 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.541 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_url": { 2023-10-16T14:45:05.541 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_url", 2023-10-16T14:45:05.541 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.541 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.542 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.542 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.542 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.542 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.542 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.543 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_username": { 2023-10-16T14:45:05.544 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_username", 2023-10-16T14:45:05.544 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.544 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.544 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.544 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.545 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.546 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.546 INFO:teuthology.orchestra.run.smithi178.stdout: "secure_monitoring_stack": { 2023-10-16T14:45:05.546 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "secure_monitoring_stack", 2023-10-16T14:45:05.546 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.547 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.547 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.547 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.547 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.548 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.549 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.549 INFO:teuthology.orchestra.run.smithi178.stdout: "service_discovery_port": { 2023-10-16T14:45:05.549 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "service_discovery_port", 2023-10-16T14:45:05.549 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.549 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8765", 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.550 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "cephadm service discovery port", 2023-10-16T14:45:05.551 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.551 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.551 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.551 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.551 INFO:teuthology.orchestra.run.smithi178.stdout: "ssh_config_file": { 2023-10-16T14:45:05.552 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssh_config_file", 2023-10-16T14:45:05.552 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.552 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.552 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.552 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.553 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.554 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.554 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.554 INFO:teuthology.orchestra.run.smithi178.stdout: "use_agent": { 2023-10-16T14:45:05.554 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_agent", 2023-10-16T14:45:05.554 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.555 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.555 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.555 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.555 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.555 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.557 INFO:teuthology.orchestra.run.smithi178.stdout: "use_repo_digest": { 2023-10-16T14:45:05.557 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_repo_digest", 2023-10-16T14:45:05.557 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.557 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.557 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.558 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.558 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.558 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.558 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.558 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_failed_host_check": { 2023-10-16T14:45:05.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_failed_host_check", 2023-10-16T14:45:05.560 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.560 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.560 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.560 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.560 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.561 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.561 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.561 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if the host check fails", 2023-10-16T14:45:05.561 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.561 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_daemons": { 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_daemons", 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.562 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.563 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.563 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.563 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.563 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.563 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.564 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-16T14:45:05.564 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.564 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.564 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_hosts": { 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_hosts", 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.565 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.566 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.566 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.566 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.566 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-16T14:45:05.566 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.567 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.567 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.567 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crash", 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.568 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.569 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.569 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.569 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.569 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.570 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.571 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.571 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.571 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.571 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.571 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.572 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.572 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.572 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.572 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.573 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.574 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.574 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.574 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.574 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.574 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.575 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.576 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.576 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.576 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.576 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.576 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.577 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.577 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.577 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.577 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.577 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.578 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.579 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.579 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.579 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.579 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.579 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.580 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.580 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.580 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.580 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.580 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.581 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.581 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.581 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.581 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.581 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "retain_interval": { 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retain_interval", 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.582 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "31536000", 2023-10-16T14:45:05.583 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.583 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.583 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.583 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-16T14:45:05.583 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.584 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.584 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.584 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.584 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_recent_interval": { 2023-10-16T14:45:05.584 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_recent_interval", 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1209600", 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.585 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.586 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.586 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-16T14:45:05.586 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.586 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.586 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.587 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.587 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.587 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dashboard", 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-16T14:45:05.588 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.589 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.589 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.589 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.589 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.589 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.590 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.590 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.590 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.590 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.590 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_HOST": { 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.591 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.592 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.592 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.592 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.592 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.592 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.593 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.593 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.593 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.593 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.593 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.594 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.595 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.595 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.595 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.595 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.595 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_ENABLED": { 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_ENABLED", 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.596 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.597 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.598 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.598 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.598 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.598 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.599 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.600 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.600 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.600 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.600 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.600 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.601 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.601 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.601 INFO:teuthology.orchestra.run.smithi178.stdout: "ENABLE_BROWSABLE_API": { 2023-10-16T14:45:05.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.602 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.603 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.603 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.603 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.603 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.603 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.604 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.604 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-16T14:45:05.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-16T14:45:05.604 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.604 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.605 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.606 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.606 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.606 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.606 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.606 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-16T14:45:05.607 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-16T14:45:05.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.607 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.607 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.607 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.608 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.609 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.609 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.609 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-16T14:45:05.609 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-16T14:45:05.609 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.610 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.611 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.611 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.611 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.611 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.612 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.613 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.613 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.613 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.613 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.613 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.614 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.614 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.614 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.614 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-16T14:45:05.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-16T14:45:05.615 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.615 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.615 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.615 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.615 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.616 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.617 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-16T14:45:05.617 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-16T14:45:05.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.617 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.618 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.619 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-16T14:45:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-16T14:45:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.620 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.621 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-16T14:45:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-16T14:45:05.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.623 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_PASSWORD": { 2023-10-16T14:45:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-16T14:45:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.625 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.626 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-16T14:45:05.627 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-16T14:45:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.628 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.629 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_URL": { 2023-10-16T14:45:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_URL", 2023-10-16T14:45:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.630 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.631 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.632 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_USERNAME": { 2023-10-16T14:45:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-16T14:45:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.633 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.634 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-16T14:45:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-16T14:45:05.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.636 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.637 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-16T14:45:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-16T14:45:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.638 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.639 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-16T14:45:05.640 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.641 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.642 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.643 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.644 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_HOST": { 2023-10-16T14:45:05.645 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-16T14:45:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.646 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.647 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-16T14:45:05.649 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-16T14:45:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.650 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.651 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-16T14:45:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-16T14:45:05.652 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.653 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-16T14:45:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-16T14:45:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.655 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.656 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-16T14:45:05.657 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-16T14:45:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.658 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.659 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-16T14:45:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-16T14:45:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.660 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.661 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.662 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-16T14:45:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-16T14:45:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.663 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.664 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-16T14:45:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-16T14:45:05.665 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.666 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-16T14:45:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-16T14:45:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.668 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.669 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_ENABLED": { 2023-10-16T14:45:05.670 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.671 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.672 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.672 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.672 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.672 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.672 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.673 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.674 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-16T14:45:05.674 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.674 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.674 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.674 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.675 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.675 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.675 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.675 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.676 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.677 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.677 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.677 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.677 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.677 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.678 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.679 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.679 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8", 2023-10-16T14:45:05.679 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.679 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.679 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.680 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.680 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.680 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.680 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.681 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "45", 2023-10-16T14:45:05.682 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.682 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.682 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.682 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.682 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ACCESS_KEY": { 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-16T14:45:05.683 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.684 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.684 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.684 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.684 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.684 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.685 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.685 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.685 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.685 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.685 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.686 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.687 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.688 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.688 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.688 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.688 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.688 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SECRET_KEY": { 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.689 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.690 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.690 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.690 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.690 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.690 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SSL_VERIFY": { 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.691 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.692 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.692 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.692 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.692 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.692 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.693 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.693 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.693 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.693 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "UNSAFE_TLS_v1_2": { 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.694 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.695 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.695 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.695 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.695 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.695 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-16T14:45:05.696 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.697 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.697 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.697 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:05.697 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.697 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.698 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.698 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.698 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.698 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.698 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.699 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.700 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.700 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.700 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.700 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.700 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-16T14:45:05.701 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-16T14:45:05.702 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.702 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.702 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.702 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:05.702 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.703 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.703 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.703 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.703 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.703 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: "cross_origin_url": { 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cross_origin_url", 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.704 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.705 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.705 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.705 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.705 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.705 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.706 INFO:teuthology.orchestra.run.smithi178.stdout: "crt_file": { 2023-10-16T14:45:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crt_file", 2023-10-16T14:45:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.707 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.708 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.708 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.708 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.708 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.708 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: "debug": { 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "debug", 2023-10-16T14:45:05.709 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.710 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.710 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.710 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.710 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.710 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.711 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.711 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable/disable debug options", 2023-10-16T14:45:05.711 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.711 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.711 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.712 INFO:teuthology.orchestra.run.smithi178.stdout: "jwt_token_ttl": { 2023-10-16T14:45:05.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "jwt_token_ttl", 2023-10-16T14:45:05.712 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.712 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "28800", 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.713 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.714 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.714 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.714 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.714 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.715 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.716 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.716 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.716 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.716 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.716 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.717 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.718 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.718 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.718 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.718 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.718 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.719 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.720 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.720 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.720 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.720 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.720 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.721 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.721 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.721 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.721 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.721 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.722 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.723 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.723 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.723 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.723 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.724 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.725 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.725 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.725 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.725 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.725 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.726 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.726 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.726 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.726 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.726 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.727 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.728 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.729 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.730 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.730 INFO:teuthology.orchestra.run.smithi178.stdout: "motd": { 2023-10-16T14:45:05.730 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "motd", 2023-10-16T14:45:05.730 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.731 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.731 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.731 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.731 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.731 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "The message of the day", 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.732 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.733 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect_resolve_ip_addr": { 2023-10-16T14:45:05.733 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "redirect_resolve_ip_addr", 2023-10-16T14:45:05.733 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.733 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.733 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:05.735 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:05.736 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.736 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.736 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.736 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:05.736 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.737 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.738 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.738 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:05.738 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:05.738 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.738 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8080", 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.739 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.740 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.740 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.740 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.740 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.740 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.741 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.742 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl_server_port": { 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl_server_port", 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.743 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.744 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.744 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8443", 2023-10-16T14:45:05.744 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.744 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.744 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.745 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.745 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.745 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.745 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.745 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.746 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "redirect", 2023-10-16T14:45:05.747 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.747 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.747 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.747 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.747 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect" 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.748 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.749 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:05.749 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:05.749 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.749 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.749 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.750 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.751 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.751 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.751 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.751 INFO:teuthology.orchestra.run.smithi178.stdout: "url_prefix": { 2023-10-16T14:45:05.751 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url_prefix", 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.753 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.753 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.753 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.753 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.753 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.754 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.754 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.754 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "devicehealth", 2023-10-16T14:45:05.755 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.755 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.755 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.755 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_monitoring": { 2023-10-16T14:45:05.755 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_monitoring", 2023-10-16T14:45:05.756 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.756 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.756 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.756 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.756 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "monitor device health metrics", 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.757 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.758 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.759 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.760 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.761 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.761 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.761 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.761 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.761 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.762 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.762 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.762 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.762 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.764 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.764 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.764 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.764 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.765 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.766 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.766 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.766 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.766 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.766 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.767 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.767 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.767 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.767 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.767 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.769 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.770 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: "mark_out_threshold": { 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mark_out_threshold", 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.771 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.772 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.772 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2419200", 2023-10-16T14:45:05.772 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.772 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.772 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.773 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-16T14:45:05.773 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.773 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.773 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.773 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.774 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_name": { 2023-10-16T14:45:05.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_name", 2023-10-16T14:45:05.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.774 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.774 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "device_health_metrics", 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-16T14:45:05.775 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: "retention_period": { 2023-10-16T14:45:05.776 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retention_period", 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15552000", 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.777 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.778 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.778 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain device health metrics", 2023-10-16T14:45:05.778 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.778 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.778 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_frequency": { 2023-10-16T14:45:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_frequency", 2023-10-16T14:45:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.779 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.780 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-16T14:45:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.781 INFO:teuthology.orchestra.run.smithi178.stdout: "self_heal": { 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "self_heal", 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:05.782 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.783 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.783 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.783 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-16T14:45:05.783 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.783 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.784 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.785 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and check device health", 2023-10-16T14:45:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_threshold": { 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_threshold", 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.787 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7257600", 2023-10-16T14:45:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-16T14:45:05.788 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "diskprediction_local", 2023-10-16T14:45:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.790 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.791 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.792 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.793 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.794 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.795 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.796 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.798 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.799 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.800 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.801 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.802 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "predict_interval": { 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predict_interval", 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.804 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.805 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: "predictor_model": { 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predictor_model", 2023-10-16T14:45:05.806 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.807 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.807 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.807 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "prophetstor", 2023-10-16T14:45:05.807 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.807 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.809 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:05.809 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:05.809 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.809 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.809 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.810 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:05.810 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.810 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.810 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.810 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.812 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.812 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "influx", 2023-10-16T14:45:05.812 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": false, 2023-10-16T14:45:05.812 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "influxdb python module not found", 2023-10-16T14:45:05.812 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "batch_size": { 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "batch_size", 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.813 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5000", 2023-10-16T14:45:05.814 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.814 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.814 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.814 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-16T14:45:05.814 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: "database": { 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "database", 2023-10-16T14:45:05.815 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.816 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.816 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.816 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "ceph", 2023-10-16T14:45:05.816 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.816 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.817 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.818 INFO:teuthology.orchestra.run.smithi178.stdout: "hostname": { 2023-10-16T14:45:05.818 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "hostname", 2023-10-16T14:45:05.818 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.818 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.818 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.819 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.819 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.819 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.819 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.819 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server hostname", 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:05.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "30", 2023-10-16T14:45:05.821 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "5", 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.822 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.823 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.823 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.823 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.823 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.824 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.824 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.824 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.824 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.824 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.825 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.825 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.825 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.825 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.825 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.826 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.826 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.826 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.826 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.826 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.827 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.827 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.827 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.827 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.827 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.828 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.829 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.829 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.829 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.829 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.830 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.830 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.830 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.830 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.830 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.831 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.832 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.832 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.832 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.832 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.832 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.833 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.833 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.833 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.833 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.833 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.834 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.835 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.835 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.835 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.835 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.835 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.836 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.836 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.836 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.836 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.836 INFO:teuthology.orchestra.run.smithi178.stdout: "password": { 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "password", 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.837 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.838 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.838 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.838 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "password of InfluxDB server user", 2023-10-16T14:45:05.838 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.838 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: "port": { 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "port", 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.839 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.840 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.840 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8086", 2023-10-16T14:45:05.840 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.840 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.840 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server port", 2023-10-16T14:45:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.841 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.842 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "false", 2023-10-16T14:45:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:05.843 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: "threads": { 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threads", 2023-10-16T14:45:05.844 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:05.845 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "32", 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.846 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.847 INFO:teuthology.orchestra.run.smithi178.stdout: "username": { 2023-10-16T14:45:05.847 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "username", 2023-10-16T14:45:05.847 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.847 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.847 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.848 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.848 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.848 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.848 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.848 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "username of InfluxDB server user", 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: "verify_ssl": { 2023-10-16T14:45:05.849 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "verify_ssl", 2023-10-16T14:45:05.850 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.850 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.850 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.850 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "true", 2023-10-16T14:45:05.850 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.851 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.852 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.852 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.852 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.852 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "insights", 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.853 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.854 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.854 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.854 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.854 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.854 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.855 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.855 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.855 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.855 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.855 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.856 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.857 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.857 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.857 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.857 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.858 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.858 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.858 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.858 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.858 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.859 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.859 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.859 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.859 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.859 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.860 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.861 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.861 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.861 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.861 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.861 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.862 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.862 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.862 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.862 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.862 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.863 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.864 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.864 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.864 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.864 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.864 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.865 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.866 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.866 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.866 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.866 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.866 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.867 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.867 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "iostat", 2023-10-16T14:45:05.867 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.867 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.867 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.868 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.869 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.869 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.869 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.869 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.869 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.870 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.870 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.870 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.870 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.870 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.871 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.872 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.872 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.872 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.872 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.872 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.874 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.874 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.874 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.874 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.874 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.875 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.875 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.875 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.875 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.875 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.876 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.877 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.877 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.877 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.877 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.877 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.878 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.879 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.879 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.879 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.879 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.879 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.880 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.881 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "k8sevents", 2023-10-16T14:45:05.881 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.881 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.882 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.882 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_event_retention_days": { 2023-10-16T14:45:05.882 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ceph_event_retention_days", 2023-10-16T14:45:05.882 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.882 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7", 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.883 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-16T14:45:05.884 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.884 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.884 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.884 INFO:teuthology.orchestra.run.smithi178.stdout: "config_check_secs": { 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_check_secs", 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:05.885 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "10", 2023-10-16T14:45:05.886 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.886 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.886 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-16T14:45:05.886 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.886 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.887 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.887 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.887 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.888 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.888 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.888 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.888 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.888 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.890 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.890 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.890 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.890 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.890 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.891 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.892 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.892 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.892 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.892 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.892 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.893 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.894 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.895 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.895 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.895 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.895 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.895 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.896 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.897 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.898 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.898 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.898 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.898 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.898 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.899 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.899 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.899 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.899 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.899 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.900 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.901 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "localpool", 2023-10-16T14:45:05.901 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.901 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.901 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.901 INFO:teuthology.orchestra.run.smithi178.stdout: "failure_domain": { 2023-10-16T14:45:05.902 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "failure_domain", 2023-10-16T14:45:05.902 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.902 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.902 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.902 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "host", 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "failure domain for any created local pool", 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-16T14:45:05.903 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.904 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.904 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.904 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.904 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.905 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.906 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.906 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.906 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.906 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.906 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.907 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.908 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.908 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.908 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.908 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.909 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.910 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.910 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.910 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.910 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.911 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.911 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.911 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.911 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.912 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.913 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.913 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.913 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.913 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.913 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.915 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.915 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.915 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.915 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.916 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.916 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.916 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.916 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.916 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: "min_size": { 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_size", 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.917 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.918 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.918 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.918 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.918 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.918 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default min_size for any created local pool", 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.919 INFO:teuthology.orchestra.run.smithi178.stdout: "num_rep": { 2023-10-16T14:45:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "num_rep", 2023-10-16T14:45:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.920 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3", 2023-10-16T14:45:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default replica count for any created local pool", 2023-10-16T14:45:05.921 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_num": { 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_num", 2023-10-16T14:45:05.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:05.923 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.923 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.923 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "128", 2023-10-16T14:45:05.923 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.923 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default pg_num for any created local pool", 2023-10-16T14:45:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.924 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: "prefix": { 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prefix", 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.925 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.926 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name prefix for any created local pool", 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: "subtree": { 2023-10-16T14:45:05.927 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "subtree", 2023-10-16T14:45:05.928 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.928 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.928 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.928 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "rack", 2023-10-16T14:45:05.928 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.929 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.929 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.929 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-16T14:45:05.929 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-16T14:45:05.929 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.930 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.930 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.930 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.930 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mds_autoscaler", 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.931 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.932 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.932 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.932 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.932 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.932 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.933 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.934 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.935 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.935 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.935 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.936 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.937 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.937 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.937 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.937 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.937 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.938 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.939 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.939 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.939 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.939 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.939 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.940 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.942 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.943 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirroring", 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.945 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.946 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.947 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.948 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.949 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.950 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.951 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.952 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.953 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.954 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.955 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.956 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.957 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.958 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "nfs", 2023-10-16T14:45:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.959 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.961 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.962 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.963 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.964 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.965 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.965 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.965 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.965 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.965 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.966 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.966 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.966 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.967 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.967 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.967 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.967 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.968 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.969 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.969 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.969 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.969 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.969 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.970 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.970 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.970 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.970 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.971 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.972 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.972 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.972 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.972 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.972 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:05.973 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: "fail_fs": { 2023-10-16T14:45:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "fail_fs", 2023-10-16T14:45:05.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.975 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.975 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:05.975 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.975 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.975 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.977 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.977 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.977 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.977 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.977 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.978 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.979 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.979 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.979 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.979 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.979 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.980 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:05.981 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:05.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.981 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.981 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.981 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.982 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.982 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.982 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.982 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.982 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:05.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.984 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.984 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.984 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:05.984 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.985 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.985 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.985 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.985 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.986 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.986 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.986 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.986 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.986 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.987 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.987 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.987 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.987 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.988 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:05.988 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:05.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:05.988 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.988 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.989 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:05.989 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.989 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.989 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:05.989 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.990 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.990 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.990 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.990 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator": { 2023-10-16T14:45:05.991 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:05.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.991 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.991 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.992 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.992 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.992 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.992 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.992 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm", 2023-10-16T14:45:05.993 INFO:teuthology.orchestra.run.smithi178.stdout: "rook", 2023-10-16T14:45:05.993 INFO:teuthology.orchestra.run.smithi178.stdout: "test_orchestrator" 2023-10-16T14:45:05.993 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.993 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Orchestrator backend", 2023-10-16T14:45:05.993 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.994 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:05.994 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:05.994 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.994 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:05.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:05.995 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:05.995 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_perf_query", 2023-10-16T14:45:05.995 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:05.995 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:05.996 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:05.997 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:05.997 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:05.997 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:05.997 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:05.997 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:05.998 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:05.998 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:05.998 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:05.998 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:05.998 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:05.999 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:05.999 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:05.999 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:05.999 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:05.999 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.000 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.000 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.000 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.000 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.001 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.001 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.001 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.001 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.002 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.002 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.002 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.002 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.002 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.003 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.003 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.003 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.003 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.004 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.004 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.004 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.004 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.004 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.005 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.005 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.005 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.005 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.005 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.006 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.006 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.006 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.006 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.007 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.007 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.007 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.007 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.007 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.008 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.008 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.008 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.008 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.008 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.009 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.009 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.009 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.009 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.009 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.010 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.010 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.010 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.011 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.011 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_support", 2023-10-16T14:45:06.011 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.011 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.011 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.012 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.012 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.012 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.012 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.012 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.013 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.013 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.013 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.013 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.013 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.014 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.014 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.014 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.014 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.014 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.015 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.015 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.015 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.015 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.016 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.016 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.016 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.016 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.016 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.017 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.017 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.017 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.017 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.017 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.018 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.018 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.018 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.018 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.018 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.019 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.019 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.019 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.019 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.019 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.020 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.020 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.020 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.020 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.020 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.021 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.021 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.021 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.021 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.021 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.022 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.023 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.023 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.023 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.023 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.023 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.024 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.024 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.024 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.024 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.025 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.025 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.025 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.025 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.025 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.026 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.026 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.026 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.026 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.027 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_autoscaler", 2023-10-16T14:45:06.027 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.027 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.027 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.027 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.028 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.028 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.028 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.028 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.028 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.029 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.029 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.029 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.029 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.029 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.030 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.030 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.030 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.030 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.030 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.031 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.031 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.031 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.031 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.031 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.032 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.032 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.032 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.032 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.032 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.033 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.033 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.033 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.033 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.033 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.034 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.034 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.034 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.034 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.035 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.035 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.035 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.035 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.035 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.036 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.037 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.037 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.037 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.037 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.037 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.038 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.038 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.038 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.038 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.039 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.039 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.039 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.040 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.040 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.040 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.040 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.041 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.041 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.041 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.041 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.042 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:06.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:06.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.042 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.042 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.043 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.044 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.044 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.044 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.044 INFO:teuthology.orchestra.run.smithi178.stdout: "threshold": { 2023-10-16T14:45:06.045 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threshold", 2023-10-16T14:45:06.045 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.045 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.045 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.045 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:06.046 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1.0", 2023-10-16T14:45:06.046 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.046 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.046 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "scaling threshold", 2023-10-16T14:45:06.046 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-16T14:45:06.047 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.047 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.048 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.048 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "progress", 2023-10-16T14:45:06.048 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.048 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.048 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.049 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_pg_recovery_event": { 2023-10-16T14:45:06.049 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_pg_recovery_event", 2023-10-16T14:45:06.049 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.049 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.049 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.050 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.050 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.050 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.050 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.051 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-16T14:45:06.051 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.051 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.051 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.051 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.052 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.053 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.053 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.053 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.053 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.053 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.054 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.054 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.054 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.054 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.055 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.055 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.057 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.058 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.058 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.058 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.058 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.058 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.059 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.059 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.059 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.059 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.059 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.061 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.061 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.061 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.062 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.062 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.062 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.062 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.062 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.063 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.063 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.063 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.063 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.063 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.064 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.064 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.064 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.064 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.065 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.065 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.065 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.065 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.065 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.066 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.066 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.066 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.066 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.067 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.067 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.067 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.067 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.067 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.068 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.068 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.068 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.068 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.069 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.069 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.069 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.069 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.070 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.070 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.070 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.070 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.070 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.071 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.071 INFO:teuthology.orchestra.run.smithi178.stdout: "max_completed_events": { 2023-10-16T14:45:06.071 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_completed_events", 2023-10-16T14:45:06.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.072 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.072 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.072 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "50", 2023-10-16T14:45:06.072 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.072 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.073 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.073 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "number of past completed events to remember", 2023-10-16T14:45:06.073 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.073 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.073 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.074 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.074 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:06.074 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:06.074 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.074 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.075 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.075 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:06.075 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.075 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.075 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long the module is going to sleep", 2023-10-16T14:45:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.076 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus", 2023-10-16T14:45:06.077 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.078 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.078 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.078 INFO:teuthology.orchestra.run.smithi178.stdout: "cache": { 2023-10-16T14:45:06.078 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cache", 2023-10-16T14:45:06.078 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.079 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.079 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.079 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.079 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.079 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.080 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.080 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.080 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.080 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.080 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.081 INFO:teuthology.orchestra.run.smithi178.stdout: "exclude_perf_counters": { 2023-10-16T14:45:06.081 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "exclude_perf_counters", 2023-10-16T14:45:06.081 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.081 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.082 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.082 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.082 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.082 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.082 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.083 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-16T14:45:06.083 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-16T14:45:06.083 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.083 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.084 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.084 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.084 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.085 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.085 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.085 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.085 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.086 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.086 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.086 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.086 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.086 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.087 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.087 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.087 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.087 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.087 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.088 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.088 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.088 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.088 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.089 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.089 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.089 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.089 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.089 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.090 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.090 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.090 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.090 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.090 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.091 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.091 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.091 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.091 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.091 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.092 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.092 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.092 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.092 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.092 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.093 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.093 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.093 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.093 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.093 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.094 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.094 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.094 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.094 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.095 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.095 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.095 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.095 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.096 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.096 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.096 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.096 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.097 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.097 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.097 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.097 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.097 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.098 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.098 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools": { 2023-10-16T14:45:06.098 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools", 2023-10-16T14:45:06.098 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.099 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.099 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.099 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.099 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.099 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.100 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.100 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.100 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.100 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.100 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.101 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.101 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-16T14:45:06.101 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-16T14:45:06.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.101 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300", 2023-10-16T14:45:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.102 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.103 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_interval": { 2023-10-16T14:45:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_interval", 2023-10-16T14:45:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.104 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15.0", 2023-10-16T14:45:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.105 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.106 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.107 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.108 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-16T14:45:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.109 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.110 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "9283", 2023-10-16T14:45:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-16T14:45:06.111 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "stale_cache_strategy": { 2023-10-16T14:45:06.112 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stale_cache_strategy", 2023-10-16T14:45:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "log", 2023-10-16T14:45:06.113 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.114 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:06.115 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "default", 2023-10-16T14:45:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.116 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "default", 2023-10-16T14:45:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "error" 2023-10-16T14:45:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.117 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.118 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.119 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.120 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_support", 2023-10-16T14:45:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.122 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.123 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.124 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.125 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.126 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.127 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.128 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.129 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.131 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.132 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.133 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.134 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.135 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.136 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_snap_create": { 2023-10-16T14:45:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_snap_create", 2023-10-16T14:45:06.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.138 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.139 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "mirror_snapshot_schedule": { 2023-10-16T14:45:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirror_snapshot_schedule", 2023-10-16T14:45:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.140 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.141 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "trash_purge_schedule": { 2023-10-16T14:45:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "trash_purge_schedule", 2023-10-16T14:45:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.143 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.144 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.145 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.146 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.146 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.146 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "restful", 2023-10-16T14:45:06.146 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.147 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.147 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.147 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_auth": { 2023-10-16T14:45:06.147 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_auth", 2023-10-16T14:45:06.147 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.148 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.149 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.149 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.149 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.149 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.149 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.150 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:06.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:06.150 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.150 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.151 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.151 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.151 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.151 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.151 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.152 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.152 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.152 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.152 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.152 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.153 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.153 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.153 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.154 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.154 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.154 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.154 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.154 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.155 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.155 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.155 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.155 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.155 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.157 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.157 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.157 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.157 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.157 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.158 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.158 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.158 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.158 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.158 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.159 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.159 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.159 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.159 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.159 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.160 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.161 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.162 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.162 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.162 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.162 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.163 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.163 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.163 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.163 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.164 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.164 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.164 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.164 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.164 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.165 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.165 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.165 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.165 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.165 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.166 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.166 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.166 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.166 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.166 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.167 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.167 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.167 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:06.167 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:06.167 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.168 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.168 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.168 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.168 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.168 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.169 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.169 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.169 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.169 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.169 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.170 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.170 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:06.170 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:06.170 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.170 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.171 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.171 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.171 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.171 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.171 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.172 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.172 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.172 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.172 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.172 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.173 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.173 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.173 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.173 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rgw", 2023-10-16T14:45:06.173 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.174 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.174 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.174 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.174 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.175 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.175 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.175 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.175 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.175 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.176 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.177 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.177 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.177 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.177 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.177 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.178 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.178 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.178 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.178 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.178 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.179 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.180 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.181 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.181 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.181 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.181 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.181 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.182 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.182 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.182 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.182 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.182 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.183 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.183 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.183 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.183 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.183 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.184 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.184 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.184 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.184 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.184 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.186 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.186 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.186 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.186 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.186 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.187 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.187 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.187 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.187 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.187 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.188 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.188 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.188 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.188 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.188 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.189 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.189 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rook", 2023-10-16T14:45:06.189 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.189 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.189 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.190 INFO:teuthology.orchestra.run.smithi178.stdout: "drive_group_interval": { 2023-10-16T14:45:06.190 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "drive_group_interval", 2023-10-16T14:45:06.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.190 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.190 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.191 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300.0", 2023-10-16T14:45:06.191 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.191 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.191 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.191 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-16T14:45:06.192 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.192 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.192 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.192 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.193 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.193 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.193 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.193 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.194 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.194 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.194 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.194 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.194 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.195 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.195 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.195 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.195 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.195 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.196 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.196 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.196 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.196 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.197 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.197 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.197 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.197 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.198 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.198 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.198 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.198 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.198 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.199 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.199 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.199 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.199 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.199 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.201 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.201 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.201 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.201 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.201 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.202 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.202 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.202 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.202 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.203 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.203 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.203 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.203 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.203 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.204 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.204 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.204 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.204 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.204 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.205 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.205 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.205 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.205 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.205 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.206 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.206 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.206 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.206 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.206 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.207 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.207 INFO:teuthology.orchestra.run.smithi178.stdout: "storage_class": { 2023-10-16T14:45:06.207 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "storage_class", 2023-10-16T14:45:06.207 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.207 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.208 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.208 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "local", 2023-10-16T14:45:06.208 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.208 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.208 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.209 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.210 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.210 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.210 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "selftest", 2023-10-16T14:45:06.210 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.211 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.211 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.211 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.211 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.211 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.212 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.212 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.212 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.212 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.214 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.214 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.214 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.214 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.214 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.215 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.215 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.215 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.215 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.215 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.216 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.216 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.216 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.216 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.216 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.217 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.217 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.217 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.217 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.217 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.218 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.218 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.218 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.218 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.219 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.219 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.219 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.219 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.219 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.220 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.220 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.220 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.220 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.220 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.221 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.221 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.221 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.221 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.221 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.222 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.222 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.222 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.222 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.222 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.223 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.223 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.223 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.223 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.223 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.224 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.224 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.224 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.224 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.224 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.225 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.225 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.225 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.225 INFO:teuthology.orchestra.run.smithi178.stdout: "roption1": { 2023-10-16T14:45:06.225 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption1", 2023-10-16T14:45:06.226 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.226 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.226 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.226 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.226 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.227 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.227 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.227 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.227 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.227 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.228 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.228 INFO:teuthology.orchestra.run.smithi178.stdout: "roption2": { 2023-10-16T14:45:06.228 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption2", 2023-10-16T14:45:06.228 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.229 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.229 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.229 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "xyz", 2023-10-16T14:45:06.229 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.229 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.230 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.230 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.230 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.230 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.230 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.231 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption1": { 2023-10-16T14:45:06.231 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption1", 2023-10-16T14:45:06.231 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.231 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.232 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.233 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.233 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.233 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.233 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.234 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption2": { 2023-10-16T14:45:06.234 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption2", 2023-10-16T14:45:06.234 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.234 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.234 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.235 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.235 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.235 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.235 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.235 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.236 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption3": { 2023-10-16T14:45:06.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption3", 2023-10-16T14:45:06.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.237 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.237 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.237 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.238 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.238 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.238 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.238 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.238 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.239 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.239 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.239 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption4": { 2023-10-16T14:45:06.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption4", 2023-10-16T14:45:06.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.240 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.240 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.240 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.240 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.241 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.241 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.241 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.241 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.241 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.242 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.242 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption5": { 2023-10-16T14:45:06.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption5", 2023-10-16T14:45:06.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.243 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.243 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.243 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.243 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.244 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.244 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.244 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.244 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.244 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.245 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.245 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption6": { 2023-10-16T14:45:06.245 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption6", 2023-10-16T14:45:06.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.246 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.246 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.246 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.246 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.246 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.247 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.247 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.247 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.247 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.247 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.248 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption7": { 2023-10-16T14:45:06.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption7", 2023-10-16T14:45:06.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.248 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.249 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.249 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.249 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:06.249 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "42", 2023-10-16T14:45:06.250 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.250 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.250 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.250 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.250 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.251 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.251 INFO:teuthology.orchestra.run.smithi178.stdout: "testkey": { 2023-10-16T14:45:06.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testkey", 2023-10-16T14:45:06.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.251 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.252 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.252 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.252 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.252 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.252 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.253 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.253 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.253 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.253 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.254 INFO:teuthology.orchestra.run.smithi178.stdout: "testlkey": { 2023-10-16T14:45:06.254 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testlkey", 2023-10-16T14:45:06.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.254 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.254 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.255 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.255 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.255 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.255 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.255 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.256 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.256 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.256 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.256 INFO:teuthology.orchestra.run.smithi178.stdout: "testnewline": { 2023-10-16T14:45:06.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testnewline", 2023-10-16T14:45:06.257 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.257 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.257 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.257 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.258 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.258 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.258 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.258 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.258 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.259 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.259 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.259 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.259 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.260 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snap_schedule", 2023-10-16T14:45:06.260 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.260 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.260 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.261 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_m_granularity": { 2023-10-16T14:45:06.261 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_m_granularity", 2023-10-16T14:45:06.261 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.261 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.262 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.262 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.262 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.262 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.262 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.263 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow minute scheduled snapshots", 2023-10-16T14:45:06.263 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.263 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.263 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.263 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.264 INFO:teuthology.orchestra.run.smithi178.stdout: "dump_on_update": { 2023-10-16T14:45:06.264 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dump_on_update", 2023-10-16T14:45:06.264 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.264 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.264 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.265 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.265 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.265 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.265 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.265 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "dump database to debug log on update", 2023-10-16T14:45:06.266 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.266 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.266 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.266 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.266 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.267 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.267 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.267 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.267 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.267 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.268 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.268 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.268 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.268 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.268 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.269 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.269 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.269 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.269 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.270 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.270 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.270 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.270 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.270 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.271 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.271 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.271 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.272 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.272 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.272 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.272 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.272 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.273 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.273 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.273 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.273 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.274 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.274 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.274 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.274 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.274 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.275 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.275 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.275 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.275 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.275 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.276 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.276 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.276 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.276 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.276 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.277 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.277 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.277 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.277 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.277 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.278 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.278 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.278 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.279 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.279 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.279 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.279 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.279 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.280 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.280 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.280 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.280 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.280 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.281 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.281 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.281 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.281 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stats", 2023-10-16T14:45:06.281 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.282 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.282 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.282 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.282 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.282 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.283 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.283 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.283 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.283 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.283 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.284 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.285 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.285 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.285 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.285 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.285 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.286 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.286 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.286 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.287 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.287 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.287 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.287 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.287 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.288 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.288 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.288 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.288 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.288 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.289 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.289 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.289 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.289 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.290 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.290 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.290 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.290 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.291 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.291 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.291 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.291 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.291 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.292 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.292 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.292 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.292 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.292 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.293 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.293 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.293 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.293 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.294 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.294 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.294 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.294 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.295 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.295 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.295 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.295 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.295 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.296 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.296 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.296 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "status", 2023-10-16T14:45:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.297 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.298 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.299 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.299 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.299 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.299 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.299 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.300 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.300 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.300 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.300 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.300 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.301 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.301 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.301 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.301 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.301 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.302 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.302 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.302 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.303 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.303 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.303 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.303 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.304 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.304 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.304 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.304 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.304 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.305 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.305 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.305 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.306 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.306 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.306 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.306 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.306 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.307 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.307 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.307 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.307 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.308 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.308 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.308 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.308 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.308 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.309 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.309 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.309 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.309 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telegraf", 2023-10-16T14:45:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: "address": { 2023-10-16T14:45:06.314 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "address", 2023-10-16T14:45:06.314 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.314 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.314 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.314 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-16T14:45:06.315 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.315 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.315 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.315 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.315 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.316 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.316 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.316 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:06.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:06.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.317 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.317 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15", 2023-10-16T14:45:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.318 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.320 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.320 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.321 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.321 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.321 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.321 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.322 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.322 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.322 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.322 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.323 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.323 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.323 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.323 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.324 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.324 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.324 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.324 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.325 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.326 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.326 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.326 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.326 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.326 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.328 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.328 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.328 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.328 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.328 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.329 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.329 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.329 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.329 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.329 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.330 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.330 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.330 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.330 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.331 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.331 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.331 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.331 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.332 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.332 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.332 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.333 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.333 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.333 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.333 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.333 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.334 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.334 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.334 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.334 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.335 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.335 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.335 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.336 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.336 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.336 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.336 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telemetry", 2023-10-16T14:45:06.337 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.337 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.337 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.337 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_basic": { 2023-10-16T14:45:06.338 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_basic", 2023-10-16T14:45:06.338 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.338 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.338 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.338 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.339 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.339 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.339 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.340 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-16T14:45:06.340 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.340 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.340 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.341 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.341 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_crash": { 2023-10-16T14:45:06.341 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_crash", 2023-10-16T14:45:06.341 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.342 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.342 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.342 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.342 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.343 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.343 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.343 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-16T14:45:06.343 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.344 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.344 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.344 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.345 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_device": { 2023-10-16T14:45:06.345 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_device", 2023-10-16T14:45:06.345 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.345 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.702 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.702 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.703 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.703 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.703 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.703 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-16T14:45:06.703 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.704 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.704 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.704 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.704 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_ident": { 2023-10-16T14:45:06.704 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_ident", 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.705 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.706 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.706 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-16T14:45:06.706 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.706 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.706 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.707 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.707 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_perf": { 2023-10-16T14:45:06.707 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_perf", 2023-10-16T14:45:06.707 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.707 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.708 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-16T14:45:06.709 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.709 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.709 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.709 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.709 INFO:teuthology.orchestra.run.smithi178.stdout: "contact": { 2023-10-16T14:45:06.710 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "contact", 2023-10-16T14:45:06.710 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.710 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.710 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.710 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.711 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.712 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.712 INFO:teuthology.orchestra.run.smithi178.stdout: "description": { 2023-10-16T14:45:06.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "description", 2023-10-16T14:45:06.712 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.713 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.713 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.713 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.713 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.715 INFO:teuthology.orchestra.run.smithi178.stdout: "device_url": { 2023-10-16T14:45:06.715 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_url", 2023-10-16T14:45:06.715 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.715 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.715 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.716 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.717 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.717 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.717 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:06.717 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:06.718 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.718 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.718 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.718 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.718 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.719 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.719 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.719 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.719 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:06.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.721 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.721 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.721 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "24", 2023-10-16T14:45:06.721 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "8", 2023-10-16T14:45:06.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.722 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: "last_opt_revision": { 2023-10-16T14:45:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "last_opt_revision", 2023-10-16T14:45:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.723 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1", 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.724 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.725 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard": { 2023-10-16T14:45:06.726 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard", 2023-10-16T14:45:06.726 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.726 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.726 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.726 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.727 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.728 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.728 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard_description": { 2023-10-16T14:45:06.728 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard_description", 2023-10-16T14:45:06.728 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.729 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.729 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.729 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.729 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.729 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.731 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.731 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.731 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.731 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.731 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.732 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.732 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.732 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.732 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.732 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.733 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.734 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.734 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.734 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.734 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.734 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.735 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.735 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.735 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.735 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.735 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.736 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.737 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.737 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.737 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.737 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.737 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.738 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.738 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.738 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.738 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.738 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.739 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.740 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.740 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.740 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.740 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.740 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.741 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.741 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.741 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.741 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.742 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.743 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.743 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.743 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.743 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.743 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.744 INFO:teuthology.orchestra.run.smithi178.stdout: "organization": { 2023-10-16T14:45:06.744 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "organization", 2023-10-16T14:45:06.744 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.744 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.745 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.746 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.746 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.746 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.746 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.746 INFO:teuthology.orchestra.run.smithi178.stdout: "proxy": { 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "proxy", 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.747 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.748 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.748 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.748 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.748 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.748 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: "url": { 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url", 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.749 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.750 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.750 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-16T14:45:06.750 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.750 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.750 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.751 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.751 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.751 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.751 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.751 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "test_orchestrator", 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.752 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.753 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.753 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.753 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.753 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.754 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.754 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.754 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.754 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.754 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.755 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.756 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.756 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.756 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.756 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.757 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.758 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.758 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.758 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.758 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.758 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.759 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.759 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.759 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.759 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.759 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.760 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.761 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.761 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.761 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.761 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.761 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.762 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.762 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.762 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.762 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.762 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.763 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.763 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.763 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.763 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.763 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.764 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.765 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.766 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.766 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.766 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.766 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "volumes", 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.767 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.768 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.768 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.768 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.769 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.769 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.769 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.769 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.769 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.770 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.771 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.771 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.771 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.771 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.771 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.772 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.772 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.772 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.772 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.772 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.773 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.774 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.774 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.775 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.775 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.775 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.775 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.775 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.776 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.777 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.777 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.777 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.777 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.777 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.778 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.778 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.778 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.778 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.778 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.779 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.780 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.780 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.780 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.780 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.780 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.781 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_clones": { 2023-10-16T14:45:06.781 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_clones", 2023-10-16T14:45:06.781 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.781 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.781 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.782 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4", 2023-10-16T14:45:06.782 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.782 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.782 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.782 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: "periodic_async_work": { 2023-10-16T14:45:06.783 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "periodic_async_work", 2023-10-16T14:45:06.784 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.784 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.784 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.784 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.784 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.785 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.785 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.785 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Periodically check for async work", 2023-10-16T14:45:06.785 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.785 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: "snapshot_clone_delay": { 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snapshot_clone_delay", 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.786 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.787 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.787 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:06.787 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.787 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.787 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.788 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-16T14:45:06.788 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.788 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.788 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.788 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix", 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.789 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.790 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.790 INFO:teuthology.orchestra.run.smithi178.stdout: "discovery_interval": { 2023-10-16T14:45:06.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "discovery_interval", 2023-10-16T14:45:06.790 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:06.790 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.791 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.791 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "100", 2023-10-16T14:45:06.791 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.791 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.791 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.792 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.793 INFO:teuthology.orchestra.run.smithi178.stdout: "identifier": { 2023-10-16T14:45:06.793 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "identifier", 2023-10-16T14:45:06.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.793 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.793 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:06.795 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:06.796 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.796 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.796 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.796 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:06.796 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.797 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.798 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.798 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.798 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.798 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.798 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.799 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.799 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.799 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.799 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.799 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.800 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.800 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.800 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.800 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.800 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.801 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.802 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.802 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.802 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.802 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.802 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.803 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.803 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.803 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.803 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.803 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.804 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.805 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.805 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.805 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.805 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.805 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.807 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.807 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.807 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.807 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.807 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.808 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.808 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.808 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.808 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.808 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.809 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.810 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.810 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.810 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.810 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.810 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_host": { 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_host", 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.812 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.813 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.813 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.813 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.813 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.813 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_port": { 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_port", 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.814 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10051", 2023-10-16T14:45:06.815 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.815 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.815 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.815 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.815 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.816 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.816 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.816 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_sender": { 2023-10-16T14:45:06.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_sender", 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.817 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.818 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.818 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.818 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.818 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.818 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.819 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.819 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:06.819 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: "gid": 24116, 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "y", 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: "mgr_features": 4540138322906710015, 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: "available_modules": [ 2023-10-16T14:45:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.821 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "alerts", 2023-10-16T14:45:06.821 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.821 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.821 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.821 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:06.822 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:06.822 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.822 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.822 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.822 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How frequently to reexamine health status", 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.823 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.824 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.824 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.824 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.824 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.824 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.825 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.826 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.826 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.826 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.826 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.826 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.827 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.827 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.827 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.827 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.827 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.828 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.828 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.828 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.828 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.828 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.829 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.829 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.829 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.829 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.829 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.830 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.830 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.830 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.830 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.830 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.831 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.832 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.832 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.832 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.832 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.832 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.833 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.833 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.833 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.833 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.833 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.834 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.834 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.834 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.834 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.834 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.835 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.836 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.836 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.836 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.836 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.836 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_destination": { 2023-10-16T14:45:06.837 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_destination", 2023-10-16T14:45:06.838 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.838 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.838 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.838 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.838 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.839 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.839 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.839 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email address to send alerts to", 2023-10-16T14:45:06.839 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.839 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_from_name": { 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_from_name", 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.840 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.841 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.841 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "Ceph", 2023-10-16T14:45:06.841 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.841 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.841 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.842 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email From: name", 2023-10-16T14:45:06.842 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.842 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.842 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_host": { 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_host", 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.844 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.844 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.844 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.844 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP server", 2023-10-16T14:45:06.844 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_password": { 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_password", 2023-10-16T14:45:06.845 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.846 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.846 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.846 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.846 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.846 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.847 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.847 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Password to authenticate with", 2023-10-16T14:45:06.847 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.847 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.847 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_port": { 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_port", 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.848 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.849 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "465", 2023-10-16T14:45:06.849 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.849 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.849 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.849 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP port", 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_sender": { 2023-10-16T14:45:06.850 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_sender", 2023-10-16T14:45:06.851 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.851 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.851 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.851 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.851 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.852 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.852 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.852 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP envelope sender", 2023-10-16T14:45:06.852 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.852 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_ssl": { 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_ssl", 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.853 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.854 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.854 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.854 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.854 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.854 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.855 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_user": { 2023-10-16T14:45:06.856 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_user", 2023-10-16T14:45:06.856 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.856 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.856 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.856 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.857 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.857 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.857 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.857 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "User to authenticate as", 2023-10-16T14:45:06.857 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.858 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.858 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.858 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.858 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.859 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.859 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "balancer", 2023-10-16T14:45:06.859 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.859 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.859 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "active": { 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "active", 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.860 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.861 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.861 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.861 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.861 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically balance PGs across cluster", 2023-10-16T14:45:06.861 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.862 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.862 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.862 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.862 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_time": { 2023-10-16T14:45:06.862 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_time", 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0000", 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.863 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.864 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.864 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "beginning time of day to automatically balance", 2023-10-16T14:45:06.864 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:06.864 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.864 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.865 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.865 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_weekday": { 2023-10-16T14:45:06.865 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_weekday", 2023-10-16T14:45:06.865 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:06.865 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.866 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.866 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:06.866 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:06.866 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:06.866 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.867 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_max_iterations": { 2023-10-16T14:45:06.868 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_max_iterations", 2023-10-16T14:45:06.868 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:06.868 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.868 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.868 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "25", 2023-10-16T14:45:06.869 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:06.869 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "250", 2023-10-16T14:45:06.869 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.869 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-16T14:45:06.869 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_metrics": { 2023-10-16T14:45:06.870 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_metrics", 2023-10-16T14:45:06.871 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.871 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.871 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.871 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "pgs,objects,bytes", 2023-10-16T14:45:06.873 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.873 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.874 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.874 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-16T14:45:06.874 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-16T14:45:06.874 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.874 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.875 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.875 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_step": { 2023-10-16T14:45:06.875 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_step", 2023-10-16T14:45:06.875 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.875 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.5", 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0.001", 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "0.999", 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.876 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "aggressiveness of optimization", 2023-10-16T14:45:06.877 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-16T14:45:06.877 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.877 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.877 INFO:teuthology.orchestra.run.smithi178.stdout: "end_time": { 2023-10-16T14:45:06.878 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_time", 2023-10-16T14:45:06.878 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.878 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.878 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.878 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2359", 2023-10-16T14:45:06.879 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.879 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.879 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.879 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ending time of day to automatically balance", 2023-10-16T14:45:06.879 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: "end_weekday": { 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_weekday", 2023-10-16T14:45:06.880 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:06.881 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:06.882 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.882 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-16T14:45:06.882 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:06.882 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.882 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.883 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.883 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:06.883 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:06.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.883 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.884 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.885 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.885 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.885 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.885 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.885 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.886 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.886 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.886 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.886 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.886 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.887 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.888 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.888 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.888 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.888 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.888 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.889 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.889 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.889 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.889 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:06.890 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:06.890 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.890 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.890 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.890 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:06.891 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:06.892 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:06.892 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:06.892 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:06.892 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.892 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.893 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.893 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.893 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.893 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.894 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.895 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.895 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.895 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:06.895 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.895 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.896 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.896 INFO:teuthology.orchestra.run.smithi178.stdout: "min_score": { 2023-10-16T14:45:06.896 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_score", 2023-10-16T14:45:06.896 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.897 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.898 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-16T14:45:06.898 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.898 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.898 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.899 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:06.899 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:06.899 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.899 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.899 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.900 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "upmap", 2023-10-16T14:45:06.900 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.900 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.900 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.900 INFO:teuthology.orchestra.run.smithi178.stdout: "crush-compat", 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: "none", 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap" 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Balancer mode", 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.901 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.902 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.902 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.902 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_ids": { 2023-10-16T14:45:06.902 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_ids", 2023-10-16T14:45:06.902 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.903 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.905 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:06.905 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:06.905 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.905 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.905 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.906 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:06.906 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.906 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.906 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.906 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-16T14:45:06.907 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.907 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.907 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.907 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_deviation": { 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_deviation", 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:06.908 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:06.909 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.909 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.909 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-16T14:45:06.909 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-16T14:45:06.909 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.910 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.910 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.910 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_optimizations": { 2023-10-16T14:45:06.910 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_optimizations", 2023-10-16T14:45:06.910 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.911 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.912 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-16T14:45:06.912 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.912 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.912 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.912 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm", 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:06.913 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:06.914 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:06.914 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_down_multiplier": { 2023-10-16T14:45:06.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_down_multiplier", 2023-10-16T14:45:06.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.914 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.915 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.915 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:06.915 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.915 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.915 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_refresh_rate": { 2023-10-16T14:45:06.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_refresh_rate", 2023-10-16T14:45:06.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.917 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.917 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.917 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "20", 2023-10-16T14:45:06.918 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.918 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.918 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.918 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-16T14:45:06.918 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_starting_port": { 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_starting_port", 2023-10-16T14:45:06.919 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:06.920 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.920 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.920 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4721", 2023-10-16T14:45:06.920 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.920 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.921 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.921 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-16T14:45:06.921 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.921 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.921 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.922 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.922 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_ptrace": { 2023-10-16T14:45:06.922 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_ptrace", 2023-10-16T14:45:06.922 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.922 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.923 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-16T14:45:06.924 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-16T14:45:06.924 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.924 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.924 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.924 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_interval": { 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_interval", 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:06.925 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.926 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.926 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.926 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-16T14:45:06.926 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.926 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.927 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.927 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.927 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_memory_target_ratio": { 2023-10-16T14:45:06.927 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_memory_target_ratio", 2023-10-16T14:45:06.927 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.7", 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.928 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.929 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-16T14:45:06.929 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.929 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.929 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.929 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm_log_destination": { 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm_log_destination", 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.930 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:06.931 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.931 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.931 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:06.931 INFO:teuthology.orchestra.run.smithi178.stdout: "file", 2023-10-16T14:45:06.931 INFO:teuthology.orchestra.run.smithi178.stdout: "file,syslog", 2023-10-16T14:45:06.932 INFO:teuthology.orchestra.run.smithi178.stdout: "syslog" 2023-10-16T14:45:06.932 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:06.932 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-16T14:45:06.932 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.932 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.933 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.933 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.933 INFO:teuthology.orchestra.run.smithi178.stdout: "cgroups_split": { 2023-10-16T14:45:06.933 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cgroups_split", 2023-10-16T14:45:06.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.934 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.935 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-16T14:45:06.935 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.935 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.935 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.936 INFO:teuthology.orchestra.run.smithi178.stdout: "config_checks_enabled": { 2023-10-16T14:45:06.936 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_checks_enabled", 2023-10-16T14:45:06.936 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.936 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.936 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-16T14:45:06.937 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi178.stdout: "config_dashboard": { 2023-10-16T14:45:06.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_dashboard", 2023-10-16T14:45:06.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.939 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.939 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.939 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.939 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.940 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.940 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.940 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-16T14:45:06.940 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.940 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_alertmanager": { 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_alertmanager", 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.941 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.942 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.942 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-16T14:45:06.942 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.942 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.942 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.943 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:06.943 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.943 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.943 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.943 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_base": { 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_base", 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:06.944 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-16T14:45:06.945 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.945 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.945 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.945 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Container image name, without the tag", 2023-10-16T14:45:06.945 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.946 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.946 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.946 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_elasticsearch": { 2023-10-16T14:45:06.946 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_elasticsearch", 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.947 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "elasticsearch container image", 2023-10-16T14:45:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.948 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.949 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.949 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_grafana": { 2023-10-16T14:45:06.949 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_grafana", 2023-10-16T14:45:06.949 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.949 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.950 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.950 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-16T14:45:06.950 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.950 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.950 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.951 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_haproxy": { 2023-10-16T14:45:06.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_haproxy", 2023-10-16T14:45:06.952 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.952 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.952 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.952 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-16T14:45:06.953 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.953 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.953 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.953 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "HAproxy container image", 2023-10-16T14:45:06.953 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_agent": { 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_agent", 2023-10-16T14:45:06.954 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.955 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.955 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.955 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-16T14:45:06.955 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.955 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.956 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.956 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger agent container image", 2023-10-16T14:45:06.956 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.956 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.956 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_collector": { 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_collector", 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.957 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.958 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-16T14:45:06.958 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.958 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.958 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.958 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger collector container image", 2023-10-16T14:45:06.959 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.959 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.959 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.959 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_query": { 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_query", 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-16T14:45:06.960 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.961 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.961 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.961 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger query container image", 2023-10-16T14:45:06.961 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.961 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_keepalived": { 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_keepalived", 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.962 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.963 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.963 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-16T14:45:06.963 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.963 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.963 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.964 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Keepalived container image", 2023-10-16T14:45:06.964 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.964 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.964 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.964 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.965 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_loki": { 2023-10-16T14:45:06.965 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_loki", 2023-10-16T14:45:06.965 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.965 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.965 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Loki container image", 2023-10-16T14:45:06.966 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_node_exporter": { 2023-10-16T14:45:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_node_exporter", 2023-10-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-16T14:45:06.968 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.969 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.970 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_nvmeof": { 2023-10-16T14:45:06.970 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_nvmeof", 2023-10-16T14:45:06.970 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.970 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.971 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.971 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-16T14:45:06.971 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.971 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.971 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Nvme-of container image", 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.972 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_prometheus": { 2023-10-16T14:45:06.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_prometheus", 2023-10-16T14:45:06.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.973 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.973 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.973 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-16T14:45:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:06.974 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_promtail": { 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_promtail", 2023-10-16T14:45:06.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.976 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.976 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.976 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-16T14:45:06.976 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.976 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Promtail container image", 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.978 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_snmp_gateway": { 2023-10-16T14:45:06.978 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_snmp_gateway", 2023-10-16T14:45:06.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.978 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.978 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.979 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-16T14:45:06.979 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.979 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.979 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.979 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SNMP Gateway container image", 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: "container_init": { 2023-10-16T14:45:06.980 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_init", 2023-10-16T14:45:06.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.981 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.981 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.981 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:06.981 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.982 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.982 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.982 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Run podman/docker with `--init`", 2023-10-16T14:45:06.982 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.982 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: "daemon_cache_timeout": { 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "daemon_cache_timeout", 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.983 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.984 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.984 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:06.984 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.984 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.984 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.985 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-16T14:45:06.985 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.985 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.985 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.986 INFO:teuthology.orchestra.run.smithi178.stdout: "default_cephadm_command_timeout": { 2023-10-16T14:45:06.986 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_cephadm_command_timeout", 2023-10-16T14:45:06.986 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.986 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.986 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "900", 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-16T14:45:06.987 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.988 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.988 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.988 INFO:teuthology.orchestra.run.smithi178.stdout: "default_registry": { 2023-10-16T14:45:06.988 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_registry", 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io", 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.989 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.990 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.990 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-16T14:45:06.990 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.990 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.990 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.991 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.991 INFO:teuthology.orchestra.run.smithi178.stdout: "device_cache_timeout": { 2023-10-16T14:45:06.991 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_cache_timeout", 2023-10-16T14:45:06.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.991 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1800", 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.992 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache device inventory", 2023-10-16T14:45:06.993 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.993 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.993 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.993 INFO:teuthology.orchestra.run.smithi178.stdout: "device_enhanced_scan": { 2023-10-16T14:45:06.994 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_enhanced_scan", 2023-10-16T14:45:06.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:06.994 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.994 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.994 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:06.995 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.995 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.995 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.995 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: "facts_cache_timeout": { 2023-10-16T14:45:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "facts_cache_timeout", 2023-10-16T14:45:06.997 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.997 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:06.997 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:06.997 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:06.997 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:06.998 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:06.998 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:06.998 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache host facts data", 2023-10-16T14:45:06.998 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:06.998 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: "host_check_interval": { 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "host_check_interval", 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:06.999 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.000 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.000 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:07.000 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.000 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.000 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.001 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to perform a host check", 2023-10-16T14:45:07.001 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.001 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.001 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.002 INFO:teuthology.orchestra.run.smithi178.stdout: "inventory_list_all": { 2023-10-16T14:45:07.002 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "inventory_list_all", 2023-10-16T14:45:07.002 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.002 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.002 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-16T14:45:07.003 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.004 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.004 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.004 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.004 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.005 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.005 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.005 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.005 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.005 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.006 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.007 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.007 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.007 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.007 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.007 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: "log_refresh_metadata": { 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_refresh_metadata", 2023-10-16T14:45:07.008 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.009 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.009 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.009 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.009 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.009 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.010 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.010 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-16T14:45:07.010 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.010 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.011 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.012 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.012 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.012 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.012 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.012 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-16T14:45:07.013 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.013 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.013 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.013 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.014 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.015 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.015 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.015 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.015 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.015 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.016 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.016 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.016 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.016 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.016 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.017 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.018 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.018 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.018 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.018 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.018 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.019 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.019 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.019 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.019 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.019 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.020 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.021 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.022 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-16T14:45:07.023 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-16T14:45:07.023 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.023 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.023 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.023 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "*", 2023-10-16T14:45:07.024 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.024 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.024 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.024 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-16T14:45:07.024 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: "max_count_per_host": { 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_count_per_host", 2023-10-16T14:45:07.025 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.026 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.026 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.026 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.026 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.026 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of daemons per service per host", 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.027 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.028 INFO:teuthology.orchestra.run.smithi178.stdout: "max_osd_draining_count": { 2023-10-16T14:45:07.028 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_osd_draining_count", 2023-10-16T14:45:07.028 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.028 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.028 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.029 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.029 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.029 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.029 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.029 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-16T14:45:07.030 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.030 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.030 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.030 INFO:teuthology.orchestra.run.smithi178.stdout: "migration_current": { 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "migration_current", 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.031 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "internal - do not modify", 2023-10-16T14:45:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.032 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.033 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.033 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:07.033 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:07.033 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "root", 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.034 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.035 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm-package", 2023-10-16T14:45:07.035 INFO:teuthology.orchestra.run.smithi178.stdout: "root" 2023-10-16T14:45:07.035 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.035 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "mode for remote execution of cephadm", 2023-10-16T14:45:07.035 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.036 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.036 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.036 INFO:teuthology.orchestra.run.smithi178.stdout: "prometheus_alerts_path": { 2023-10-16T14:45:07.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus_alerts_path", 2023-10-16T14:45:07.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.038 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.038 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.038 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-16T14:45:07.038 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.038 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.039 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.039 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-16T14:45:07.039 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.039 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.039 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.040 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_insecure": { 2023-10-16T14:45:07.040 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_insecure", 2023-10-16T14:45:07.040 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.040 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.041 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-16T14:45:07.042 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.042 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.042 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.042 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.042 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_password": { 2023-10-16T14:45:07.043 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_password", 2023-10-16T14:45:07.043 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.043 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.043 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.043 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.044 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.044 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.044 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.044 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-16T14:45:07.044 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_url": { 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_url", 2023-10-16T14:45:07.045 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.046 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.047 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.047 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-16T14:45:07.047 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.047 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.047 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_username": { 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_username", 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.048 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.049 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.049 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.049 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.049 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.049 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: "secure_monitoring_stack": { 2023-10-16T14:45:07.050 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "secure_monitoring_stack", 2023-10-16T14:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.051 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.052 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.052 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.052 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-16T14:45:07.052 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.052 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: "service_discovery_port": { 2023-10-16T14:45:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "service_discovery_port", 2023-10-16T14:45:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8765", 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.054 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.055 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "cephadm service discovery port", 2023-10-16T14:45:07.055 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.055 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.055 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.055 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.056 INFO:teuthology.orchestra.run.smithi178.stdout: "ssh_config_file": { 2023-10-16T14:45:07.056 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssh_config_file", 2023-10-16T14:45:07.056 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.056 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.056 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-16T14:45:07.057 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.058 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.058 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.058 INFO:teuthology.orchestra.run.smithi178.stdout: "use_agent": { 2023-10-16T14:45:07.058 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_agent", 2023-10-16T14:45:07.059 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.059 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.059 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.059 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.059 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.060 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.060 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.060 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-16T14:45:07.060 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.060 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: "use_repo_digest": { 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_repo_digest", 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.061 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.062 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.062 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.062 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.062 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.064 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_failed_host_check": { 2023-10-16T14:45:07.064 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_failed_host_check", 2023-10-16T14:45:07.064 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.064 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.064 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.065 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.065 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.065 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.065 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.065 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if the host check fails", 2023-10-16T14:45:07.066 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.066 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.066 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.066 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_daemons": { 2023-10-16T14:45:07.067 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_daemons", 2023-10-16T14:45:07.067 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.067 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.067 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.067 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.068 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.069 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.069 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.069 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_hosts": { 2023-10-16T14:45:07.069 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_hosts", 2023-10-16T14:45:07.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.070 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.070 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.070 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.070 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.070 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.071 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crash", 2023-10-16T14:45:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.073 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.073 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.073 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.073 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.073 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.074 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.075 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.075 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.075 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.075 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.075 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.076 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.077 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.077 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.077 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.077 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.077 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.078 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.078 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.078 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.078 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.078 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.079 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.080 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.080 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.080 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.080 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.080 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.081 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.082 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.082 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.082 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.082 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.082 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.083 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.083 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.083 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.083 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.083 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.084 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.085 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.086 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.086 INFO:teuthology.orchestra.run.smithi178.stdout: "retain_interval": { 2023-10-16T14:45:07.086 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retain_interval", 2023-10-16T14:45:07.086 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "31536000", 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.087 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.088 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-16T14:45:07.088 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.088 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.088 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.088 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.089 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_recent_interval": { 2023-10-16T14:45:07.089 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_recent_interval", 2023-10-16T14:45:07.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.089 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.089 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1209600", 2023-10-16T14:45:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.090 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-16T14:45:07.091 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.091 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.091 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.091 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.092 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dashboard", 2023-10-16T14:45:07.092 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.092 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.093 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.093 INFO:teuthology.orchestra.run.smithi178.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-16T14:45:07.093 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-16T14:45:07.093 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.093 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.094 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.094 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.094 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.094 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.094 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.095 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.095 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.095 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.096 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.096 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.096 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_HOST": { 2023-10-16T14:45:07.096 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-16T14:45:07.097 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.097 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.097 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.098 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.098 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.098 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.099 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.099 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.099 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.100 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.100 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.100 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.101 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-16T14:45:07.101 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-16T14:45:07.101 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.101 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.102 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.102 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.102 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.102 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.103 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.103 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.103 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.104 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.104 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.104 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_ENABLED": { 2023-10-16T14:45:07.105 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_ENABLED", 2023-10-16T14:45:07.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.105 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.106 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.106 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.107 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.107 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.107 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.107 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.108 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.108 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.108 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.109 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-16T14:45:07.109 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-16T14:45:07.109 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.109 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.110 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.110 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.110 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.111 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.111 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.111 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.111 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.112 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.112 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.112 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.112 INFO:teuthology.orchestra.run.smithi178.stdout: "ENABLE_BROWSABLE_API": { 2023-10-16T14:45:07.113 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-16T14:45:07.113 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.113 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.114 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.114 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.114 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.114 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.115 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.115 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.115 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.116 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.116 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.116 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-16T14:45:07.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-16T14:45:07.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.117 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.117 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.117 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.117 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.118 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.119 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-16T14:45:07.119 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-16T14:45:07.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.119 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.120 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.120 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.120 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.120 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.120 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.121 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-16T14:45:07.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-16T14:45:07.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.122 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.122 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.122 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.123 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.123 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.123 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.123 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.123 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-16T14:45:07.124 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.125 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.125 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.125 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.125 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.125 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.126 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.126 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.126 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.126 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.126 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.127 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.128 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.128 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.128 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.128 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.128 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.129 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.129 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.129 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.129 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.130 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.131 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.131 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.131 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.131 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.131 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.132 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.133 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.133 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.133 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.133 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.133 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.134 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.134 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.134 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.134 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.135 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.136 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.136 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.136 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.136 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.136 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_PASSWORD": { 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-16T14:45:07.137 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.138 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.138 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.138 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:07.138 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.138 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.139 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.139 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.139 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.139 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.139 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.140 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.140 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-16T14:45:07.140 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-16T14:45:07.140 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.140 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.141 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.142 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.142 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.142 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.142 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.142 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_URL": { 2023-10-16T14:45:07.143 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_URL", 2023-10-16T14:45:07.143 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.143 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.143 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.143 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.144 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.145 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.145 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.145 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_USERNAME": { 2023-10-16T14:45:07.145 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-16T14:45:07.145 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.146 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.146 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.146 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:07.146 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.146 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.148 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-16T14:45:07.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-16T14:45:07.148 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.148 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.148 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.149 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.150 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.150 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.150 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-16T14:45:07.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-16T14:45:07.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.151 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.151 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.151 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.151 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.152 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.153 INFO:teuthology.orchestra.run.smithi178.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-16T14:45:07.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-16T14:45:07.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.153 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.154 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.154 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.154 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.154 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.154 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.155 INFO:teuthology.orchestra.run.smithi178.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-16T14:45:07.156 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-16T14:45:07.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.156 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.156 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.156 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.157 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.157 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.157 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.157 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.157 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_HOST": { 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-16T14:45:07.158 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.159 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.159 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.159 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.159 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.159 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.160 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.161 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.162 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.162 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.162 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.162 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.162 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.163 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.163 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.163 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.163 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.164 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.165 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.165 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.165 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.165 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.165 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.166 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.166 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-16T14:45:07.166 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-16T14:45:07.166 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.167 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.168 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.168 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.168 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.168 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.168 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.169 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.170 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.170 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.170 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.170 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.170 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.171 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.171 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.171 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-16T14:45:07.171 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.172 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.173 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.174 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-16T14:45:07.174 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-16T14:45:07.174 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.174 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.175 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.175 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.175 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.175 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.175 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.176 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-16T14:45:07.177 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-16T14:45:07.177 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.177 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.177 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.177 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.178 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.178 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.178 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.178 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.178 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.179 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.179 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.179 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-16T14:45:07.179 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.180 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.181 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.181 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.181 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.181 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.181 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.182 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_ENABLED": { 2023-10-16T14:45:07.182 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-16T14:45:07.182 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.182 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.183 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.184 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-16T14:45:07.185 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.186 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.186 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.186 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.186 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.186 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.187 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.187 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-16T14:45:07.187 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-16T14:45:07.187 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.188 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.188 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.188 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.188 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.188 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.189 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.190 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-16T14:45:07.190 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-16T14:45:07.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.190 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.190 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.191 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8", 2023-10-16T14:45:07.191 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.191 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.191 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.191 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-16T14:45:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-16T14:45:07.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.193 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.193 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.193 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "45", 2023-10-16T14:45:07.193 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.194 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.194 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.194 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.194 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.194 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ACCESS_KEY": { 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.195 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.196 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.196 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.196 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.196 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.196 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.197 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-16T14:45:07.198 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-16T14:45:07.198 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.198 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.198 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.198 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:07.199 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.199 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.199 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.199 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.199 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SECRET_KEY": { 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-16T14:45:07.200 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.201 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.201 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.201 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.201 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.201 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.202 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.202 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.202 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.202 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.202 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SSL_VERIFY": { 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.203 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.204 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.204 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.204 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.204 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.204 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: "UNSAFE_TLS_v1_2": { 2023-10-16T14:45:07.205 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-16T14:45:07.206 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.206 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.206 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.206 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.206 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.207 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.207 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.207 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.207 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.207 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.208 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.208 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-16T14:45:07.208 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-16T14:45:07.208 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.209 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.209 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.209 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:07.209 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.209 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.210 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.211 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-16T14:45:07.211 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-16T14:45:07.211 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.211 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.211 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.212 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.212 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.212 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.212 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.212 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-16T14:45:07.213 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-16T14:45:07.214 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.214 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.214 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.214 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:07.214 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.215 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: "cross_origin_url": { 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cross_origin_url", 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.216 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.217 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.217 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.217 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.217 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.217 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.218 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.218 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.218 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.218 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.218 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "crt_file": { 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crt_file", 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.219 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.220 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.220 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.220 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.220 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.220 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.221 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.221 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.221 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.221 INFO:teuthology.orchestra.run.smithi178.stdout: "debug": { 2023-10-16T14:45:07.221 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "debug", 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.222 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.223 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.223 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable/disable debug options", 2023-10-16T14:45:07.223 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.223 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.223 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.224 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.224 INFO:teuthology.orchestra.run.smithi178.stdout: "jwt_token_ttl": { 2023-10-16T14:45:07.224 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "jwt_token_ttl", 2023-10-16T14:45:07.224 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.224 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "28800", 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.225 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.226 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.226 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.226 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.226 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:07.227 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:07.227 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.227 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.227 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.227 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.228 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.228 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.228 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.228 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.228 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.229 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.229 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.229 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.229 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.229 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.230 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.230 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.230 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.230 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.230 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.231 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.232 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.232 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.232 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.232 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.232 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.233 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.233 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.233 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.233 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.233 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.234 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.234 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.234 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.234 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.234 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.235 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.235 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.235 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.235 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.235 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.236 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.237 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.237 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.237 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.237 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.237 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.238 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.238 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.238 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.238 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.238 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.239 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.240 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.241 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.241 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.241 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.241 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.241 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.243 INFO:teuthology.orchestra.run.smithi178.stdout: "motd": { 2023-10-16T14:45:07.243 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "motd", 2023-10-16T14:45:07.243 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.243 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.243 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "The message of the day", 2023-10-16T14:45:07.244 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect_resolve_ip_addr": { 2023-10-16T14:45:07.245 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "redirect_resolve_ip_addr", 2023-10-16T14:45:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.246 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.247 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.247 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.247 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.247 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.247 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.248 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.249 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.249 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:07.249 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.249 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.249 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.250 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.250 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.250 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.250 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.250 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.251 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:07.251 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:07.251 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.251 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.251 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8080", 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.252 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.253 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.253 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.253 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.253 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:07.253 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:07.254 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.254 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.254 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.254 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.254 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.256 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.256 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.256 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl_server_port": { 2023-10-16T14:45:07.256 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl_server_port", 2023-10-16T14:45:07.256 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8443", 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.257 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.258 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.258 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.258 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.258 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.258 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.259 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:07.259 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:07.259 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.259 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.259 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.260 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "redirect", 2023-10-16T14:45:07.260 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.260 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.260 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.260 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect" 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.261 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.262 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:07.262 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:07.262 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.262 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:07.263 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.264 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.264 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.264 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.264 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.264 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "url_prefix": { 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url_prefix", 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.265 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.266 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.266 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.266 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.266 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.266 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.267 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.267 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.267 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.268 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.268 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "devicehealth", 2023-10-16T14:45:07.268 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.268 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.268 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_monitoring": { 2023-10-16T14:45:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_monitoring", 2023-10-16T14:45:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.269 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.270 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "monitor device health metrics", 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.271 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.272 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.272 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.272 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.272 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.272 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.273 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.274 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.275 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.275 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.275 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.275 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.276 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.277 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.277 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.277 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.277 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.277 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.278 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.279 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.279 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.279 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.279 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.279 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.280 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.280 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.280 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.280 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.280 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.281 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.281 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.281 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.281 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.281 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.282 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.282 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.282 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.282 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.282 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.283 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.284 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.284 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.285 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.286 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.286 INFO:teuthology.orchestra.run.smithi178.stdout: "mark_out_threshold": { 2023-10-16T14:45:07.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mark_out_threshold", 2023-10-16T14:45:07.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2419200", 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.287 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.288 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-16T14:45:07.288 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.288 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.288 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.289 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_name": { 2023-10-16T14:45:07.289 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_name", 2023-10-16T14:45:07.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.289 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.289 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "device_health_metrics", 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-16T14:45:07.290 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.291 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.291 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.291 INFO:teuthology.orchestra.run.smithi178.stdout: "retention_period": { 2023-10-16T14:45:07.291 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retention_period", 2023-10-16T14:45:07.292 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.292 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.292 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.292 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15552000", 2023-10-16T14:45:07.292 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.293 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.293 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.293 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain device health metrics", 2023-10-16T14:45:07.293 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.293 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_frequency": { 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_frequency", 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.294 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.295 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.295 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:07.295 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.295 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.295 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.296 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-16T14:45:07.296 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.296 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.296 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.297 INFO:teuthology.orchestra.run.smithi178.stdout: "self_heal": { 2023-10-16T14:45:07.297 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "self_heal", 2023-10-16T14:45:07.297 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.297 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.297 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-16T14:45:07.298 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.299 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.299 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.299 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:07.299 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:07.300 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.300 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.300 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.300 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:07.300 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.301 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.301 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.301 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and check device health", 2023-10-16T14:45:07.301 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.301 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_threshold": { 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_threshold", 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.302 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.303 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.303 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7257600", 2023-10-16T14:45:07.303 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.303 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.303 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-16T14:45:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.304 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "diskprediction_local", 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.305 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.306 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.306 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.306 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.306 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.306 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.307 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.307 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.307 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.307 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.307 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.308 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.309 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.309 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.309 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.309 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.309 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.310 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.310 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.310 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.310 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.311 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.311 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.311 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.311 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.311 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.312 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.313 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.313 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.313 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.313 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.313 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.314 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.315 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.315 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.315 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.315 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.315 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.316 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.317 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.317 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.318 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.318 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.318 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.318 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.318 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: "predict_interval": { 2023-10-16T14:45:07.319 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predict_interval", 2023-10-16T14:45:07.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.320 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.320 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.320 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:07.320 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.321 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.321 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.321 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.321 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.321 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: "predictor_model": { 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predictor_model", 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.322 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.323 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.323 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "prophetstor", 2023-10-16T14:45:07.323 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.323 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.323 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.324 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.324 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.324 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.324 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.324 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.325 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:07.325 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:07.325 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.325 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.325 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.326 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:07.326 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.326 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.326 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.326 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.328 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.328 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "influx", 2023-10-16T14:45:07.328 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": false, 2023-10-16T14:45:07.328 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "influxdb python module not found", 2023-10-16T14:45:07.328 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.329 INFO:teuthology.orchestra.run.smithi178.stdout: "batch_size": { 2023-10-16T14:45:07.329 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "batch_size", 2023-10-16T14:45:07.329 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.329 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.329 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.330 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5000", 2023-10-16T14:45:07.330 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.330 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.330 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.330 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: "database": { 2023-10-16T14:45:07.331 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "database", 2023-10-16T14:45:07.332 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.332 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.332 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.332 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "ceph", 2023-10-16T14:45:07.332 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.333 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.334 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.334 INFO:teuthology.orchestra.run.smithi178.stdout: "hostname": { 2023-10-16T14:45:07.334 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "hostname", 2023-10-16T14:45:07.334 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.334 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.335 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.335 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.335 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.335 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.335 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server hostname", 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.336 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.337 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "30", 2023-10-16T14:45:07.338 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "5", 2023-10-16T14:45:07.338 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.338 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.338 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-16T14:45:07.338 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.339 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.340 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.341 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.342 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.343 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.344 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.346 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.347 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.348 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.349 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.350 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.351 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.352 INFO:teuthology.orchestra.run.smithi178.stdout: "password": { 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "password", 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.353 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "password of InfluxDB server user", 2023-10-16T14:45:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.354 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "port": { 2023-10-16T14:45:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "port", 2023-10-16T14:45:07.355 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8086", 2023-10-16T14:45:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.356 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server port", 2023-10-16T14:45:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.357 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.358 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "false", 2023-10-16T14:45:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.359 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.360 INFO:teuthology.orchestra.run.smithi178.stdout: "threads": { 2023-10-16T14:45:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threads", 2023-10-16T14:45:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.361 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "32", 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.362 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "username": { 2023-10-16T14:45:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "username", 2023-10-16T14:45:07.363 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.364 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "username of InfluxDB server user", 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.365 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "verify_ssl": { 2023-10-16T14:45:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "verify_ssl", 2023-10-16T14:45:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.366 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "true", 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:07.367 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "insights", 2023-10-16T14:45:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.369 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.370 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.371 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.372 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.373 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.374 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.375 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.376 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.377 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.378 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.379 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.380 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.381 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.382 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.383 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "iostat", 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.384 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.385 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.386 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.387 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.388 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.389 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.390 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.391 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.392 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.393 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.394 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.395 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.396 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.397 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "k8sevents", 2023-10-16T14:45:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.398 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_event_retention_days": { 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ceph_event_retention_days", 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.399 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7", 2023-10-16T14:45:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-16T14:45:07.400 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "config_check_secs": { 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_check_secs", 2023-10-16T14:45:07.401 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "10", 2023-10-16T14:45:07.402 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.403 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.404 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.405 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.406 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.408 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.410 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.411 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.412 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.413 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.414 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.415 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.416 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "localpool", 2023-10-16T14:45:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "failure_domain": { 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "failure_domain", 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.418 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "host", 2023-10-16T14:45:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.419 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "failure domain for any created local pool", 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.420 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.421 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.422 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.423 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.424 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.424 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.424 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.424 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.424 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.425 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.426 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.427 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.428 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.429 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.430 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.431 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.432 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.433 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "min_size": { 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_size", 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.434 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default min_size for any created local pool", 2023-10-16T14:45:07.435 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-16T14:45:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "num_rep": { 2023-10-16T14:45:07.436 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "num_rep", 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3", 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.437 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default replica count for any created local pool", 2023-10-16T14:45:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.438 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_num": { 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_num", 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.439 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "128", 2023-10-16T14:45:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.440 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default pg_num for any created local pool", 2023-10-16T14:45:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.441 INFO:teuthology.orchestra.run.smithi178.stdout: "prefix": { 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prefix", 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.442 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name prefix for any created local pool", 2023-10-16T14:45:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.443 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "subtree": { 2023-10-16T14:45:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "subtree", 2023-10-16T14:45:07.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "rack", 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.445 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-16T14:45:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-16T14:45:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.446 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mds_autoscaler", 2023-10-16T14:45:07.447 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.448 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.449 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.450 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.451 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.452 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.453 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.454 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.454 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.454 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.454 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.454 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.455 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.456 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.456 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.456 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.456 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.456 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.457 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.458 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.458 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.458 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.458 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.458 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.459 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.459 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.459 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.459 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.460 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.460 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.460 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.460 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.460 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.461 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.462 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirroring", 2023-10-16T14:45:07.462 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.462 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.462 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.462 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.463 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.463 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.463 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.463 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.463 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.464 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.465 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.465 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.465 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.465 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.465 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.466 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.466 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.466 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.466 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.467 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.468 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.468 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.468 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.468 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.468 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.469 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.470 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.470 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.470 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.470 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.470 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.471 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.471 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.471 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.471 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.471 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.472 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.472 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.472 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.472 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.472 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.473 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.474 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.474 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.474 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.474 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.474 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.475 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.475 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.475 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.475 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.475 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "nfs", 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.476 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.477 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.477 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.477 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.477 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.478 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.478 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.478 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.478 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.478 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.479 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.480 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.480 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.480 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.480 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.480 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.481 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.481 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.481 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.481 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.481 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.482 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.483 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.483 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.483 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.483 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.483 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.484 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.485 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.485 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.485 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.485 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.485 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.486 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.486 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.486 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.486 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.486 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.487 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.488 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.488 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.488 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.488 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.488 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.489 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.490 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.490 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.490 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.490 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "fail_fs": { 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "fail_fs", 2023-10-16T14:45:07.491 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.492 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.492 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.493 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.494 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.494 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.494 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.494 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-16T14:45:07.494 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.495 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.496 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.496 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.496 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.496 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.496 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.497 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.497 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.497 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.497 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.497 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.498 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.498 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.498 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.498 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.498 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.499 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.500 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.500 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.500 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.500 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.500 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.502 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.502 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.502 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.502 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.502 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.503 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.503 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.503 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.503 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.503 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.504 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.505 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.505 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.505 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.505 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.505 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.506 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.506 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.506 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.506 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.506 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.507 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.508 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.508 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.508 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator": { 2023-10-16T14:45:07.508 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.509 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.510 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.510 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm", 2023-10-16T14:45:07.510 INFO:teuthology.orchestra.run.smithi178.stdout: "rook", 2023-10-16T14:45:07.510 INFO:teuthology.orchestra.run.smithi178.stdout: "test_orchestrator" 2023-10-16T14:45:07.510 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.511 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Orchestrator backend", 2023-10-16T14:45:07.511 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.511 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.511 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_perf_query", 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.512 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.513 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.513 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.513 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.513 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.513 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.514 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.515 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.515 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.515 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.515 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.515 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.516 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.517 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.517 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.517 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.517 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.517 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.519 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.519 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.519 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.519 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.519 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.520 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.521 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.521 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.521 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.521 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.521 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.522 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.522 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.522 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.522 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.522 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.523 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.523 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.523 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.523 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.523 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.524 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.525 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.525 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.525 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.525 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.525 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.526 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.526 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.526 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.526 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.526 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_support", 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.527 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.528 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.528 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.528 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.528 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.528 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.529 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.530 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.530 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.530 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.530 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.530 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.531 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.532 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.532 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.532 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.532 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.532 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.533 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.533 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.533 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.533 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.533 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.534 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.534 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.534 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.534 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.534 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.535 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.535 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.535 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.535 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.535 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.536 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.537 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.537 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.537 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.537 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.537 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.538 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.538 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.538 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.538 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.538 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.539 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.540 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.540 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.540 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.540 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.540 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.541 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.541 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_autoscaler", 2023-10-16T14:45:07.541 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.541 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.542 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.542 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.542 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.542 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.542 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.543 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.545 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.545 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.545 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.545 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.545 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.546 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.547 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.547 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.547 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.547 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.547 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.548 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.548 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.548 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.548 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.548 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.549 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.550 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.550 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.550 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.550 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.550 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.551 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.552 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.552 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.552 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.552 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.552 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.553 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.553 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.553 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.553 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.553 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.554 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.554 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.554 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.554 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.554 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.555 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.556 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.556 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:07.556 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.556 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.556 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.557 INFO:teuthology.orchestra.run.smithi178.stdout: "threshold": { 2023-10-16T14:45:07.558 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threshold", 2023-10-16T14:45:07.558 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:07.558 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.558 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.558 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:07.559 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1.0", 2023-10-16T14:45:07.559 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.559 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.559 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "scaling threshold", 2023-10-16T14:45:07.559 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-16T14:45:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.560 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "progress", 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.561 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_pg_recovery_event": { 2023-10-16T14:45:07.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_pg_recovery_event", 2023-10-16T14:45:07.562 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.562 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.562 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.562 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.563 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.563 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.563 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.563 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-16T14:45:07.563 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:07.564 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.565 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.565 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.565 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.565 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.565 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.566 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.566 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.566 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.566 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.566 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.567 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.568 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.568 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.568 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.568 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.568 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.569 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.569 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.569 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.569 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.569 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.570 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.570 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.570 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.570 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.570 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.571 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.572 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.572 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.572 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.572 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.572 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.573 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.573 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.573 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.573 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.573 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.574 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.575 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.575 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.575 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.575 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.575 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.576 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.577 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.577 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.577 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.577 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.577 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.578 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.578 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.578 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.578 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.578 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.579 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.579 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.579 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.579 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.579 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.580 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.580 INFO:teuthology.orchestra.run.smithi178.stdout: "max_completed_events": { 2023-10-16T14:45:07.580 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_completed_events", 2023-10-16T14:45:07.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "50", 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.581 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.582 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "number of past completed events to remember", 2023-10-16T14:45:07.582 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.582 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.582 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.582 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.583 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:07.584 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.584 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.584 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.584 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long the module is going to sleep", 2023-10-16T14:45:07.584 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.585 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.586 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus", 2023-10-16T14:45:07.586 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.586 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.586 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.586 INFO:teuthology.orchestra.run.smithi178.stdout: "cache": { 2023-10-16T14:45:07.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cache", 2023-10-16T14:45:07.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.587 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.587 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.587 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.588 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.589 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.589 INFO:teuthology.orchestra.run.smithi178.stdout: "exclude_perf_counters": { 2023-10-16T14:45:07.589 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "exclude_perf_counters", 2023-10-16T14:45:07.589 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.590 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.590 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.590 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.590 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.590 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.591 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.591 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-16T14:45:07.591 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-16T14:45:07.591 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.591 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.592 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.592 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.592 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.592 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.592 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.593 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.594 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.594 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.594 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.594 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.594 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.595 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.595 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.595 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.595 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.595 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.596 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.597 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.597 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.597 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.597 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.597 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.598 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.598 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.598 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.598 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.598 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.599 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.600 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.600 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.600 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.600 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.600 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.601 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.602 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.602 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.602 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.602 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.603 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.604 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.604 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.604 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.604 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.604 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.605 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.605 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.605 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.605 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.605 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools": { 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools", 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.606 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.607 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.607 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.607 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.607 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.607 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.608 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.608 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.608 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-16T14:45:07.608 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-16T14:45:07.608 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.609 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.609 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.609 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300", 2023-10-16T14:45:07.609 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.609 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.610 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.611 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_interval": { 2023-10-16T14:45:07.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_interval", 2023-10-16T14:45:07.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:07.611 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.611 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.612 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15.0", 2023-10-16T14:45:07.612 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.612 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.612 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.612 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.613 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.613 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.613 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.613 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:07.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:07.614 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.614 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.614 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.614 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:07.615 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.615 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.615 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.615 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-16T14:45:07.615 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.616 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.616 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.616 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.616 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:07.616 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "9283", 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.617 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.618 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.618 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-16T14:45:07.618 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.618 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.618 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.619 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.619 INFO:teuthology.orchestra.run.smithi178.stdout: "stale_cache_strategy": { 2023-10-16T14:45:07.619 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stale_cache_strategy", 2023-10-16T14:45:07.619 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.619 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "log", 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.620 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.621 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.621 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.621 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.621 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.621 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:07.622 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:07.622 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.622 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.622 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.622 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "default", 2023-10-16T14:45:07.623 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.623 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.623 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.623 INFO:teuthology.orchestra.run.smithi178.stdout: "default", 2023-10-16T14:45:07.623 INFO:teuthology.orchestra.run.smithi178.stdout: "error" 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.624 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.625 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:07.625 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:07.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.625 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.625 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.626 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.627 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.627 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.627 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.627 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.628 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.628 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_support", 2023-10-16T14:45:07.628 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.628 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.629 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.629 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.629 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.629 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.629 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.630 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.631 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.631 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.631 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.631 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.631 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.632 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.632 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.632 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.632 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.632 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.633 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.634 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.634 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.634 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.634 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.634 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.635 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.636 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.637 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.638 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.639 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.640 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.641 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_snap_create": { 2023-10-16T14:45:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_snap_create", 2023-10-16T14:45:07.642 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.643 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.644 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "mirror_snapshot_schedule": { 2023-10-16T14:45:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirror_snapshot_schedule", 2023-10-16T14:45:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.645 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "trash_purge_schedule": { 2023-10-16T14:45:07.647 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "trash_purge_schedule", 2023-10-16T14:45:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.648 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.649 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.650 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "restful", 2023-10-16T14:45:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_auth": { 2023-10-16T14:45:07.651 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_auth", 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.652 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.653 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.654 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.655 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.656 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.657 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.658 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.659 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.660 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.661 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.662 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.663 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.664 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.665 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.666 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.667 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.668 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.669 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.670 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.671 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:07.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.673 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.674 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rgw", 2023-10-16T14:45:07.675 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.676 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.676 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.676 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.676 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.676 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.677 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.678 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.678 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.678 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.678 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.678 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.679 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.679 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.679 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.679 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.679 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.680 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.680 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.680 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.680 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.680 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.681 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.682 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.682 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.682 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.682 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.682 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.683 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.683 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.683 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.683 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.683 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.684 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.685 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.685 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.685 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.685 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.685 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.686 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.687 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.687 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.687 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.687 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.687 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.688 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.688 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.688 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.688 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.688 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.689 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.690 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rook", 2023-10-16T14:45:07.690 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.690 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.690 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.690 INFO:teuthology.orchestra.run.smithi178.stdout: "drive_group_interval": { 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "drive_group_interval", 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300.0", 2023-10-16T14:45:07.691 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.692 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.692 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.692 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-16T14:45:07.692 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.692 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.693 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.693 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.693 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.693 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.693 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.694 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.695 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.695 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.695 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.695 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.695 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.696 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.697 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.697 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.697 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.697 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.697 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.698 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.698 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.698 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.698 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.698 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.699 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.701 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.701 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.702 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.703 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.703 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.703 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.703 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.703 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.704 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.705 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.705 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.705 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.705 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.705 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.706 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.706 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.706 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.706 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.706 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.707 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.709 INFO:teuthology.orchestra.run.smithi178.stdout: "storage_class": { 2023-10-16T14:45:07.709 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "storage_class", 2023-10-16T14:45:07.709 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.709 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.709 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.710 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "local", 2023-10-16T14:45:07.710 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.710 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.710 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.710 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-16T14:45:07.711 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.711 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.711 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.711 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.711 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "selftest", 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.712 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.713 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.713 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.713 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.713 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.713 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.714 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.714 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.714 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.714 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.714 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.715 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.716 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.716 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.716 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.716 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.716 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.717 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.717 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.717 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.717 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.717 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.718 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.718 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.718 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.718 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.718 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.719 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.719 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.719 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.719 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.720 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.721 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.721 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.721 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.721 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.722 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.722 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.722 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.722 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.722 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.723 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.724 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.724 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.724 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.724 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.724 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.725 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.725 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.725 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.725 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.725 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.726 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.726 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.726 INFO:teuthology.orchestra.run.smithi178.stdout: "roption1": { 2023-10-16T14:45:07.726 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption1", 2023-10-16T14:45:07.726 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.727 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.728 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.728 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.728 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.728 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.728 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "roption2": { 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption2", 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.729 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "xyz", 2023-10-16T14:45:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.730 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption1": { 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption1", 2023-10-16T14:45:07.731 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.732 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.732 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.732 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.732 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.732 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.733 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.734 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption2": { 2023-10-16T14:45:07.734 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption2", 2023-10-16T14:45:07.734 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.734 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.734 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.735 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.735 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.735 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.735 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.735 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption3": { 2023-10-16T14:45:07.736 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption3", 2023-10-16T14:45:07.737 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:07.737 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.737 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.737 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.737 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.738 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.738 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.738 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.738 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.738 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption4": { 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption4", 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.739 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.740 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.740 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.740 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.740 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.740 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.741 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.741 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.741 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.741 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption5": { 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption5", 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.742 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.743 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.743 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.743 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.743 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.743 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption6": { 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption6", 2023-10-16T14:45:07.744 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.745 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.745 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.745 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.745 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.745 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.746 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.746 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.746 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.746 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.746 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption7": { 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption7", 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.747 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.748 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.748 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:07.748 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "42", 2023-10-16T14:45:07.748 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.748 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.749 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.749 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.749 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.749 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.749 INFO:teuthology.orchestra.run.smithi178.stdout: "testkey": { 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testkey", 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.750 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.751 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.751 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.751 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.751 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.751 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.752 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.752 INFO:teuthology.orchestra.run.smithi178.stdout: "testlkey": { 2023-10-16T14:45:07.752 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testlkey", 2023-10-16T14:45:07.752 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.755 INFO:teuthology.orchestra.run.smithi178.stdout: "testnewline": { 2023-10-16T14:45:07.755 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testnewline", 2023-10-16T14:45:07.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.755 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.755 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.756 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.757 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.757 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.757 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snap_schedule", 2023-10-16T14:45:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.758 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_m_granularity": { 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_m_granularity", 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.759 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.760 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.760 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.760 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.760 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow minute scheduled snapshots", 2023-10-16T14:45:07.760 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.761 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.761 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.761 INFO:teuthology.orchestra.run.smithi178.stdout: "dump_on_update": { 2023-10-16T14:45:07.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dump_on_update", 2023-10-16T14:45:07.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.762 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.762 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.762 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.762 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "dump database to debug log on update", 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.763 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.764 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.764 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.765 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.766 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.766 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.766 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.766 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.766 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.767 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.767 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.767 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.767 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.767 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.768 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.769 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.769 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.769 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.769 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.769 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.770 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.770 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.770 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.770 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.772 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.772 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.772 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.772 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.772 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.773 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.774 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.774 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.774 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.775 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.775 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.775 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.775 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.775 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.776 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.776 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.776 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.776 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.776 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stats", 2023-10-16T14:45:07.778 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.778 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.778 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.778 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.778 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.779 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.780 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.780 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.780 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.780 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.780 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.781 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.781 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.781 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.781 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.781 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.782 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.783 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.783 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.783 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.783 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.783 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.784 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.785 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.785 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.785 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.785 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.785 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.786 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.787 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.787 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.787 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.787 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.787 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.788 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.789 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.789 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.789 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.789 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.789 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.790 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.790 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.790 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.790 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.790 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.791 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.791 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.791 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.791 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "status", 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.792 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.793 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.793 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.793 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.793 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.794 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.795 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.795 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.795 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.795 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.795 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.796 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.796 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.796 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.796 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.797 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.798 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.798 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.798 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.798 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.798 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.799 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.799 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.799 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.799 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.799 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.800 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.801 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.801 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.801 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.801 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.801 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.802 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.803 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.803 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.803 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.803 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.803 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.804 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.805 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.805 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.805 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.805 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.806 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.806 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telegraf", 2023-10-16T14:45:07.806 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.806 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "address": { 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "address", 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.807 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.808 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-16T14:45:07.808 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.808 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.808 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.808 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.809 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.809 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.809 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.809 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.809 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15", 2023-10-16T14:45:07.810 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.811 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.811 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.811 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.811 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.811 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.812 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.813 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.813 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.813 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.813 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.813 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.814 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.814 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.814 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.814 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.814 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.815 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.816 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.816 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.816 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.816 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.817 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.818 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.818 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.818 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.818 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.818 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.819 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.819 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.819 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.819 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.819 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.820 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.821 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.821 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.821 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.821 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.821 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.822 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.823 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.823 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.823 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.823 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.823 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.824 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.825 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.825 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.825 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.825 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.825 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telemetry", 2023-10-16T14:45:07.826 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.826 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.826 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.826 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_basic": { 2023-10-16T14:45:07.826 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_basic", 2023-10-16T14:45:07.827 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.827 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.827 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.827 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.827 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.828 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_crash": { 2023-10-16T14:45:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_crash", 2023-10-16T14:45:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.829 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.830 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-16T14:45:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.831 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_device": { 2023-10-16T14:45:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_device", 2023-10-16T14:45:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.832 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.833 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_ident": { 2023-10-16T14:45:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_ident", 2023-10-16T14:45:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.835 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-16T14:45:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.836 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_perf": { 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_perf", 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.837 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.838 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-16T14:45:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.839 INFO:teuthology.orchestra.run.smithi178.stdout: "contact": { 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "contact", 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.840 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.841 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "description": { 2023-10-16T14:45:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "description", 2023-10-16T14:45:07.842 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.843 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "device_url": { 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_url", 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.845 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-16T14:45:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.846 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:07.847 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.848 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.849 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.850 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "24", 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "8", 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.851 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.852 INFO:teuthology.orchestra.run.smithi178.stdout: "last_opt_revision": { 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "last_opt_revision", 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1", 2023-10-16T14:45:07.853 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.854 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard": { 2023-10-16T14:45:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard", 2023-10-16T14:45:07.855 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.856 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.857 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard_description": { 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard_description", 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.858 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.859 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.860 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.861 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.862 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.863 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.864 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.865 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.866 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.867 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.868 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.869 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.870 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.871 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.872 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.873 INFO:teuthology.orchestra.run.smithi178.stdout: "organization": { 2023-10-16T14:45:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "organization", 2023-10-16T14:45:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.874 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.875 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: "proxy": { 2023-10-16T14:45:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "proxy", 2023-10-16T14:45:07.876 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.877 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.878 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.878 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.878 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.878 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.878 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "url": { 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url", 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.879 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-16T14:45:07.880 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.880 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.880 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.880 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.880 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.881 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.882 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "test_orchestrator", 2023-10-16T14:45:07.882 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.882 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.882 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.882 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.883 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.883 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.883 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.883 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.883 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.884 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.885 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.886 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.887 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.888 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.889 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.890 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.891 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.892 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.893 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.894 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.895 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "volumes", 2023-10-16T14:45:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.896 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.897 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.898 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.899 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.900 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.901 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.902 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.903 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.904 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.905 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.905 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.905 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.905 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.905 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.906 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.906 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.906 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.906 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.906 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.907 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.909 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.909 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.911 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.911 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.911 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.911 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_clones": { 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_clones", 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.912 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4", 2023-10-16T14:45:07.913 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.913 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.913 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.913 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-16T14:45:07.913 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.914 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.914 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.914 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.914 INFO:teuthology.orchestra.run.smithi178.stdout: "periodic_async_work": { 2023-10-16T14:45:07.914 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "periodic_async_work", 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.915 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.916 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.916 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Periodically check for async work", 2023-10-16T14:45:07.916 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.916 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.916 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.917 INFO:teuthology.orchestra.run.smithi178.stdout: "snapshot_clone_delay": { 2023-10-16T14:45:07.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snapshot_clone_delay", 2023-10-16T14:45:07.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.917 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.918 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-16T14:45:07.919 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.919 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.919 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.919 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix", 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.920 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.921 INFO:teuthology.orchestra.run.smithi178.stdout: "discovery_interval": { 2023-10-16T14:45:07.921 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "discovery_interval", 2023-10-16T14:45:07.921 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:07.921 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.921 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.922 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "100", 2023-10-16T14:45:07.922 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.922 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.922 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.922 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: "identifier": { 2023-10-16T14:45:07.923 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "identifier", 2023-10-16T14:45:07.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.924 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.924 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.924 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.924 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.925 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.926 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:07.926 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:07.926 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.926 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.927 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.928 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.929 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.930 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.931 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.932 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.932 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.932 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.932 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.933 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.934 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.936 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.937 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.938 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.939 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.940 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.941 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_host": { 2023-10-16T14:45:07.942 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_host", 2023-10-16T14:45:07.942 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.942 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.942 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.942 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.943 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.943 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.943 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.943 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.943 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_port": { 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_port", 2023-10-16T14:45:07.944 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.945 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.945 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.945 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10051", 2023-10-16T14:45:07.945 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.945 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.946 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.947 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_sender": { 2023-10-16T14:45:07.947 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_sender", 2023-10-16T14:45:07.947 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.947 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.947 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.948 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.949 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.949 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.949 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.950 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:07.950 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.950 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.950 INFO:teuthology.orchestra.run.smithi178.stdout: "modules": [ 2023-10-16T14:45:07.950 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm", 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: "dashboard", 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: "iostat", 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: "nfs", 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: "restful" 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.951 INFO:teuthology.orchestra.run.smithi178.stdout: "available_modules": [ 2023-10-16T14:45:07.952 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.952 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "alerts", 2023-10-16T14:45:07.952 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.952 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.952 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.953 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:07.954 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.954 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.954 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.954 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How frequently to reexamine health status", 2023-10-16T14:45:07.954 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:07.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.956 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.956 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.956 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.956 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.956 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.957 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.957 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.957 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.957 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.957 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.958 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.959 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.959 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.959 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:07.959 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:07.959 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.960 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.961 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.962 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:07.962 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:07.962 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.962 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.962 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:07.963 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:07.964 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:07.964 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:07.964 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:07.964 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:07.964 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.965 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:07.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:07.966 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.966 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.966 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.966 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:07.967 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.967 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.967 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.967 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:07.967 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_destination": { 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_destination", 2023-10-16T14:45:07.968 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.969 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.969 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.969 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.969 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.969 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.970 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.970 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email address to send alerts to", 2023-10-16T14:45:07.970 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.970 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.970 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_from_name": { 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_from_name", 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.971 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "Ceph", 2023-10-16T14:45:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Email From: name", 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_host": { 2023-10-16T14:45:07.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_host", 2023-10-16T14:45:07.974 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.974 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.974 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.974 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.974 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP server", 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.975 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_password": { 2023-10-16T14:45:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_password", 2023-10-16T14:45:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.976 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.977 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.978 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Password to authenticate with", 2023-10-16T14:45:07.978 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.978 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.978 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_port": { 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_port", 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.979 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "465", 2023-10-16T14:45:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP port", 2023-10-16T14:45:07.980 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_sender": { 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_sender", 2023-10-16T14:45:07.981 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.982 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.982 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.982 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.982 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.982 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.983 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.983 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SMTP envelope sender", 2023-10-16T14:45:07.983 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.983 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.983 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_ssl": { 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_ssl", 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.984 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.985 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.985 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.985 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.985 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.985 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use SSL to connect to SMTP server", 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: "smtp_user": { 2023-10-16T14:45:07.986 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "smtp_user", 2023-10-16T14:45:07.987 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.987 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.987 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.987 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:07.987 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.988 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.988 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.988 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "User to authenticate as", 2023-10-16T14:45:07.988 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.988 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:07.989 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "balancer", 2023-10-16T14:45:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: "active": { 2023-10-16T14:45:07.990 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "active", 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.991 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.992 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.992 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically balance PGs across cluster", 2023-10-16T14:45:07.992 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:07.992 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.992 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_time": { 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_time", 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.993 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.994 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0000", 2023-10-16T14:45:07.994 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:07.994 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:07.994 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.994 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "beginning time of day to automatically balance", 2023-10-16T14:45:07.995 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:07.995 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.995 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.995 INFO:teuthology.orchestra.run.smithi178.stdout: "begin_weekday": { 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "begin_weekday", 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:07.996 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:07.997 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:07.997 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:07.997 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-10-16T14:45:07.997 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:07.997 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_max_iterations": { 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_max_iterations", 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:07.998 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "25", 2023-10-16T14:45:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "250", 2023-10-16T14:45:07.999 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-10-16T14:45:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.000 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_metrics": { 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_metrics", 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.001 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "pgs,objects,bytes", 2023-10-16T14:45:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-10-16T14:45:08.002 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "crush_compat_step": { 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crush_compat_step", 2023-10-16T14:45:08.003 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.5", 2023-10-16T14:45:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0.001", 2023-10-16T14:45:08.004 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "0.999", 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "aggressiveness of optimization", 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.005 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "end_time": { 2023-10-16T14:45:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_time", 2023-10-16T14:45:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.006 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2359", 2023-10-16T14:45:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.007 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ending time of day to automatically balance", 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "end_weekday": { 2023-10-16T14:45:08.008 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "end_weekday", 2023-10-16T14:45:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:08.009 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "0", 2023-10-16T14:45:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "6", 2023-10-16T14:45:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-10-16T14:45:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-10-16T14:45:08.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.011 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.012 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.013 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.014 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.015 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.016 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.017 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.018 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.019 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.020 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.021 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.022 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.023 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "min_score": { 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_score", 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.024 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.025 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-10-16T14:45:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.027 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "upmap", 2023-10-16T14:45:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "crush-compat", 2023-10-16T14:45:08.028 INFO:teuthology.orchestra.run.smithi178.stdout: "none", 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap" 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Balancer mode", 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.029 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_ids": { 2023-10-16T14:45:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_ids", 2023-10-16T14:45:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.030 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.031 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-10-16T14:45:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.032 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:08.033 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-10-16T14:45:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.034 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_deviation": { 2023-10-16T14:45:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_deviation", 2023-10-16T14:45:08.035 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.036 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "deviation below which no optimization is attempted", 2023-10-16T14:45:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-10-16T14:45:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.037 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "upmap_max_optimizations": { 2023-10-16T14:45:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "upmap_max_optimizations", 2023-10-16T14:45:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.038 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-10-16T14:45:08.039 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.040 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm", 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.041 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_down_multiplier": { 2023-10-16T14:45:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_down_multiplier", 2023-10-16T14:45:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.042 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-10-16T14:45:08.043 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_refresh_rate": { 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_refresh_rate", 2023-10-16T14:45:08.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "20", 2023-10-16T14:45:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.045 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-10-16T14:45:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.046 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "agent_starting_port": { 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "agent_starting_port", 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.047 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4721", 2023-10-16T14:45:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.048 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_ptrace": { 2023-10-16T14:45:08.049 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_ptrace", 2023-10-16T14:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.050 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-10-16T14:45:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-10-16T14:45:08.051 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_interval": { 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_interval", 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.052 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.053 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to autotune daemon memory", 2023-10-16T14:45:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.054 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "autotune_memory_target_ratio": { 2023-10-16T14:45:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "autotune_memory_target_ratio", 2023-10-16T14:45:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.055 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0.7", 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-10-16T14:45:08.056 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm_log_destination": { 2023-10-16T14:45:08.057 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cephadm_log_destination", 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.058 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "file", 2023-10-16T14:45:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "file,syslog", 2023-10-16T14:45:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: "syslog" 2023-10-16T14:45:08.059 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-10-16T14:45:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.060 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "cgroups_split": { 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cgroups_split", 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.061 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-10-16T14:45:08.062 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "config_checks_enabled": { 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_checks_enabled", 2023-10-16T14:45:08.063 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.064 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-10-16T14:45:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.065 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "config_dashboard": { 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_dashboard", 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.066 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.067 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_alertmanager": { 2023-10-16T14:45:08.068 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_alertmanager", 2023-10-16T14:45:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-10-16T14:45:08.069 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.070 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_base": { 2023-10-16T14:45:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_base", 2023-10-16T14:45:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.071 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph", 2023-10-16T14:45:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.072 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Container image name, without the tag", 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.073 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_elasticsearch": { 2023-10-16T14:45:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_elasticsearch", 2023-10-16T14:45:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.074 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "elasticsearch container image", 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.075 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_grafana": { 2023-10-16T14:45:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_grafana", 2023-10-16T14:45:08.076 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-10-16T14:45:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.077 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.078 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_haproxy": { 2023-10-16T14:45:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_haproxy", 2023-10-16T14:45:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.079 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "HAproxy container image", 2023-10-16T14:45:08.080 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_agent": { 2023-10-16T14:45:08.081 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_agent", 2023-10-16T14:45:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-10-16T14:45:08.082 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger agent container image", 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.083 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_collector": { 2023-10-16T14:45:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_collector", 2023-10-16T14:45:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.084 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.085 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger collector container image", 2023-10-16T14:45:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.086 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_jaeger_query": { 2023-10-16T14:45:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_jaeger_query", 2023-10-16T14:45:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.087 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Jaeger query container image", 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.088 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_keepalived": { 2023-10-16T14:45:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_keepalived", 2023-10-16T14:45:08.089 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-10-16T14:45:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.090 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Keepalived container image", 2023-10-16T14:45:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.091 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_loki": { 2023-10-16T14:45:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_loki", 2023-10-16T14:45:08.092 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-10-16T14:45:08.093 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Loki container image", 2023-10-16T14:45:08.094 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_node_exporter": { 2023-10-16T14:45:08.095 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_node_exporter", 2023-10-16T14:45:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-10-16T14:45:08.096 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.097 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.098 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_nvmeof": { 2023-10-16T14:45:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_nvmeof", 2023-10-16T14:45:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.099 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.3", 2023-10-16T14:45:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.100 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Nvme-of container image", 2023-10-16T14:45:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.101 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_prometheus": { 2023-10-16T14:45:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_prometheus", 2023-10-16T14:45:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.102 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-10-16T14:45:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.103 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Prometheus container image", 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.104 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_promtail": { 2023-10-16T14:45:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_promtail", 2023-10-16T14:45:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.105 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-10-16T14:45:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Promtail container image", 2023-10-16T14:45:08.106 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "container_image_snmp_gateway": { 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_image_snmp_gateway", 2023-10-16T14:45:08.107 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-10-16T14:45:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.108 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "SNMP Gateway container image", 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.109 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "container_init": { 2023-10-16T14:45:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "container_init", 2023-10-16T14:45:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.110 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Run podman/docker with `--init`", 2023-10-16T14:45:08.111 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.112 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.113 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "daemon_cache_timeout": { 2023-10-16T14:45:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "daemon_cache_timeout", 2023-10-16T14:45:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.114 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-10-16T14:45:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.116 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.116 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.116 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.116 INFO:teuthology.orchestra.run.smithi178.stdout: "default_cephadm_command_timeout": { 2023-10-16T14:45:08.116 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_cephadm_command_timeout", 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "900", 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.117 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-10-16T14:45:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.118 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.119 INFO:teuthology.orchestra.run.smithi178.stdout: "default_registry": { 2023-10-16T14:45:08.119 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "default_registry", 2023-10-16T14:45:08.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.119 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "docker.io", 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.120 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-10-16T14:45:08.121 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.121 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.121 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.121 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.121 INFO:teuthology.orchestra.run.smithi178.stdout: "device_cache_timeout": { 2023-10-16T14:45:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_cache_timeout", 2023-10-16T14:45:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.122 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1800", 2023-10-16T14:45:08.123 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.123 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.123 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.123 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache device inventory", 2023-10-16T14:45:08.123 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: "device_enhanced_scan": { 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_enhanced_scan", 2023-10-16T14:45:08.124 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.125 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.125 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.125 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.125 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.125 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use libstoragemgmt during device scans", 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.126 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.127 INFO:teuthology.orchestra.run.smithi178.stdout: "facts_cache_timeout": { 2023-10-16T14:45:08.127 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "facts_cache_timeout", 2023-10-16T14:45:08.127 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.127 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.127 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "seconds to cache host facts data", 2023-10-16T14:45:08.128 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.129 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.129 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.129 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.129 INFO:teuthology.orchestra.run.smithi178.stdout: "host_check_interval": { 2023-10-16T14:45:08.129 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "host_check_interval", 2023-10-16T14:45:08.130 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.130 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.130 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.130 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:08.130 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to perform a host check", 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.131 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.132 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.132 INFO:teuthology.orchestra.run.smithi178.stdout: "inventory_list_all": { 2023-10-16T14:45:08.132 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "inventory_list_all", 2023-10-16T14:45:08.132 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.132 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.133 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.133 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.133 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.133 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.133 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.134 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-10-16T14:45:08.134 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.134 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.134 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.134 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.135 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.136 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.136 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.136 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.136 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.136 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.137 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.138 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.138 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.138 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.138 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.138 INFO:teuthology.orchestra.run.smithi178.stdout: "log_refresh_metadata": { 2023-10-16T14:45:08.139 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_refresh_metadata", 2023-10-16T14:45:08.139 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.139 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.139 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.139 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.141 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.141 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.141 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.141 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.141 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.142 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.142 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.142 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.142 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.142 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.143 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.143 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-10-16T14:45:08.143 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.143 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.143 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.144 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.145 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.145 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.145 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.145 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.145 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.146 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.146 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.146 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.146 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.146 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.147 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.148 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.148 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.148 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.148 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.148 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.149 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.150 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.150 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.150 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.150 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf": { 2023-10-16T14:45:08.150 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.151 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.152 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.152 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-10-16T14:45:08.152 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.152 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.152 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.153 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.153 INFO:teuthology.orchestra.run.smithi178.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-10-16T14:45:08.153 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-10-16T14:45:08.153 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.153 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "*", 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.154 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-10-16T14:45:08.155 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.155 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.155 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.155 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.155 INFO:teuthology.orchestra.run.smithi178.stdout: "max_count_per_host": { 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_count_per_host", 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.156 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.157 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.157 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.157 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of daemons per service per host", 2023-10-16T14:45:08.157 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.157 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.158 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.158 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.158 INFO:teuthology.orchestra.run.smithi178.stdout: "max_osd_draining_count": { 2023-10-16T14:45:08.158 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_osd_draining_count", 2023-10-16T14:45:08.158 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.159 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.160 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-10-16T14:45:08.160 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.160 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.160 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.160 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "migration_current": { 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "migration_current", 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.161 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.162 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.162 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.162 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.162 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "internal - do not modify", 2023-10-16T14:45:08.162 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: "mode": { 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mode", 2023-10-16T14:45:08.163 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.164 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.164 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.164 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "root", 2023-10-16T14:45:08.164 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.164 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm-package", 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: "root" 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "mode for remote execution of cephadm", 2023-10-16T14:45:08.165 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.166 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.166 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.166 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.166 INFO:teuthology.orchestra.run.smithi178.stdout: "prometheus_alerts_path": { 2023-10-16T14:45:08.166 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus_alerts_path", 2023-10-16T14:45:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-10-16T14:45:08.167 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.168 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.169 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.169 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_insecure": { 2023-10-16T14:45:08.169 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_insecure", 2023-10-16T14:45:08.169 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.169 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.170 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-10-16T14:45:08.171 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.171 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.171 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.171 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.171 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_password": { 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_password", 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.172 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.173 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.173 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.173 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-10-16T14:45:08.173 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.173 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.174 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.174 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.174 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_url": { 2023-10-16T14:45:08.174 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_url", 2023-10-16T14:45:08.174 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.175 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.176 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-10-16T14:45:08.176 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.176 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.176 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.176 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.177 INFO:teuthology.orchestra.run.smithi178.stdout: "registry_username": { 2023-10-16T14:45:08.177 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "registry_username", 2023-10-16T14:45:08.177 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.177 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.177 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-10-16T14:45:08.178 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.179 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.179 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.179 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.179 INFO:teuthology.orchestra.run.smithi178.stdout: "secure_monitoring_stack": { 2023-10-16T14:45:08.179 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "secure_monitoring_stack", 2023-10-16T14:45:08.180 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.180 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.180 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.180 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.180 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.181 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.182 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.182 INFO:teuthology.orchestra.run.smithi178.stdout: "service_discovery_port": { 2023-10-16T14:45:08.182 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "service_discovery_port", 2023-10-16T14:45:08.182 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.182 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.183 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.183 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8765", 2023-10-16T14:45:08.183 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.183 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.183 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "cephadm service discovery port", 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.184 INFO:teuthology.orchestra.run.smithi178.stdout: "ssh_config_file": { 2023-10-16T14:45:08.185 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssh_config_file", 2023-10-16T14:45:08.185 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.185 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.185 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.185 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-10-16T14:45:08.186 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: "use_agent": { 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_agent", 2023-10-16T14:45:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.189 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.189 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-10-16T14:45:08.189 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.189 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.189 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: "use_repo_digest": { 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "use_repo_digest", 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.190 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.191 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.191 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.191 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.191 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.191 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_failed_host_check": { 2023-10-16T14:45:08.192 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_failed_host_check", 2023-10-16T14:45:08.193 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.193 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.193 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.193 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.193 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.194 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.194 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.194 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if the host check fails", 2023-10-16T14:45:08.194 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.194 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_daemons": { 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_daemons", 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.195 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.196 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.196 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.196 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.196 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.196 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.197 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_on_stray_hosts": { 2023-10-16T14:45:08.198 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_on_stray_hosts", 2023-10-16T14:45:08.198 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.198 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.198 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.198 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.199 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.200 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.200 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.200 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.200 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.200 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crash", 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.201 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.202 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.202 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.202 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.202 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.202 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.203 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.203 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.203 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.203 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.203 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.204 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.205 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.205 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.205 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.205 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.205 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.206 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.207 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.207 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.207 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.207 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.207 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.208 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.209 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.209 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.209 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.209 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.209 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.210 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.210 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.210 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.210 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.210 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.211 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.212 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.212 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.212 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.212 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.212 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.213 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.214 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.214 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.214 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.214 INFO:teuthology.orchestra.run.smithi178.stdout: "retain_interval": { 2023-10-16T14:45:08.214 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retain_interval", 2023-10-16T14:45:08.215 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.215 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.215 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.215 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "31536000", 2023-10-16T14:45:08.215 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain crashes before pruning them", 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.216 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.217 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.217 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_recent_interval": { 2023-10-16T14:45:08.217 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_recent_interval", 2023-10-16T14:45:08.217 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.217 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1209600", 2023-10-16T14:45:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.218 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "time interval in which to warn about recent crashes", 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.219 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.220 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.220 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.220 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dashboard", 2023-10-16T14:45:08.220 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.220 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.221 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.221 INFO:teuthology.orchestra.run.smithi178.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-10-16T14:45:08.221 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-10-16T14:45:08.221 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.221 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.222 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.223 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.223 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.223 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.223 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.223 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_HOST": { 2023-10-16T14:45:08.224 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_HOST", 2023-10-16T14:45:08.224 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.224 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.224 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.224 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.225 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.226 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.226 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.226 INFO:teuthology.orchestra.run.smithi178.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-10-16T14:45:08.226 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-10-16T14:45:08.226 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.227 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.227 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.227 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.227 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.227 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.228 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.229 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_ENABLED": { 2023-10-16T14:45:08.229 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_ENABLED", 2023-10-16T14:45:08.229 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.229 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.229 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.230 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.230 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.230 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.230 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.230 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-10-16T14:45:08.231 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-10-16T14:45:08.232 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.232 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.232 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.232 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.232 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.233 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.234 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.234 INFO:teuthology.orchestra.run.smithi178.stdout: "ENABLE_BROWSABLE_API": { 2023-10-16T14:45:08.234 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ENABLE_BROWSABLE_API", 2023-10-16T14:45:08.234 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.234 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.235 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.235 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.235 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.235 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.235 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.236 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-10-16T14:45:08.237 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-10-16T14:45:08.237 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.237 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.237 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.237 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.238 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.238 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.238 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.238 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.238 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-10-16T14:45:08.239 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.240 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.240 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.240 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.240 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.240 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.241 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.242 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-10-16T14:45:08.242 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-10-16T14:45:08.242 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.242 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.242 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.243 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.244 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.244 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.244 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.244 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-10-16T14:45:08.244 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.245 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.246 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.246 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.246 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.246 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.246 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.247 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.247 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_NFS": { 2023-10-16T14:45:08.247 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-10-16T14:45:08.247 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.247 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.248 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.249 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.249 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.249 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.249 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.249 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RBD": { 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.250 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.251 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.251 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.251 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.251 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.251 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: "FEATURE_TOGGLE_RGW": { 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.252 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.253 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.253 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.253 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.253 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.253 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.254 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.255 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.256 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.256 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.256 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.256 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.256 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.257 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.257 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.257 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.257 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_PASSWORD": { 2023-10-16T14:45:08.257 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_PASSWORD", 2023-10-16T14:45:08.258 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.258 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.258 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.258 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:08.258 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.259 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.260 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.260 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-10-16T14:45:08.260 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-10-16T14:45:08.260 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.260 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.261 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.262 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.262 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.262 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.262 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.262 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_URL": { 2023-10-16T14:45:08.263 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_URL", 2023-10-16T14:45:08.263 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.263 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.263 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.263 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.264 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.265 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.265 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.265 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_API_USERNAME": { 2023-10-16T14:45:08.265 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_API_USERNAME", 2023-10-16T14:45:08.265 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.266 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.266 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.266 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:08.266 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.266 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.267 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.268 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-10-16T14:45:08.268 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-10-16T14:45:08.268 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.268 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.268 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.269 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.269 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.269 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.269 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.269 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-10-16T14:45:08.270 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-10-16T14:45:08.271 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.271 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.271 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.271 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.271 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.272 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.273 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.273 INFO:teuthology.orchestra.run.smithi178.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-10-16T14:45:08.273 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-10-16T14:45:08.273 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.273 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.274 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.275 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.275 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.275 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.275 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.275 INFO:teuthology.orchestra.run.smithi178.stdout: "ISSUE_TRACKER_API_KEY": { 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.276 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.277 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.277 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.277 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.277 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.277 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.278 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.278 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.278 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_HOST": { 2023-10-16T14:45:08.278 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_HOST", 2023-10-16T14:45:08.278 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.279 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.280 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.280 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.280 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.280 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.280 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.281 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.282 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.282 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.282 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.282 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.282 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.283 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.283 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.283 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.283 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-10-16T14:45:08.283 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.284 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.285 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.285 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.285 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.285 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.285 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.286 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.287 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.287 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.287 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.287 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.287 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-10-16T14:45:08.288 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-10-16T14:45:08.289 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.289 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.289 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.289 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.289 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.290 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.290 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.290 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.290 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.290 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.291 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.292 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.292 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.292 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.292 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.292 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.293 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-10-16T14:45:08.294 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-10-16T14:45:08.294 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.294 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.294 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.294 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.295 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.295 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.295 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.295 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.295 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-10-16T14:45:08.296 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.297 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.297 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.297 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.297 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.297 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.298 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.298 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.298 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.298 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.298 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.299 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.300 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.300 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.300 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.300 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.300 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.301 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.301 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.301 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.301 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.301 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_ENABLED": { 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_ENABLED", 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.302 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.303 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.303 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.303 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.303 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.303 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.304 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.305 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.305 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-10-16T14:45:08.305 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.305 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.305 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.306 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.306 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.306 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.306 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.306 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.307 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-10-16T14:45:08.307 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-10-16T14:45:08.307 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.307 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.307 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.308 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.309 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.309 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.309 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.309 INFO:teuthology.orchestra.run.smithi178.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-10-16T14:45:08.309 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-10-16T14:45:08.310 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.310 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.310 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.310 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8", 2023-10-16T14:45:08.310 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.311 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.312 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.312 INFO:teuthology.orchestra.run.smithi178.stdout: "REST_REQUESTS_TIMEOUT": { 2023-10-16T14:45:08.312 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-10-16T14:45:08.312 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.312 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "45", 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.313 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.314 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.314 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.314 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.314 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.314 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ACCESS_KEY": { 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ACCESS_KEY", 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.315 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.316 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.316 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.316 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.316 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.316 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.317 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.317 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.317 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-10-16T14:45:08.317 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-10-16T14:45:08.317 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "admin", 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.318 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.319 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.319 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.319 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.319 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.319 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SECRET_KEY": { 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SECRET_KEY", 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.320 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.321 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.321 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.321 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.321 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.321 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: "RGW_API_SSL_VERIFY": { 2023-10-16T14:45:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "RGW_API_SSL_VERIFY", 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.323 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.324 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.325 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.325 INFO:teuthology.orchestra.run.smithi178.stdout: "UNSAFE_TLS_v1_2": { 2023-10-16T14:45:08.325 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "UNSAFE_TLS_v1_2", 2023-10-16T14:45:08.325 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.325 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.326 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.326 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.326 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.326 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.326 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.327 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-10-16T14:45:08.328 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-10-16T14:45:08.328 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.328 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.328 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.328 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.329 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.330 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.330 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.330 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-10-16T14:45:08.330 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-10-16T14:45:08.330 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.331 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.332 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.332 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.332 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.332 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.332 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.333 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.334 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.335 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.335 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.335 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.335 INFO:teuthology.orchestra.run.smithi178.stdout: "cross_origin_url": { 2023-10-16T14:45:08.335 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cross_origin_url", 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.336 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.337 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.337 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.337 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.337 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.337 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: "crt_file": { 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "crt_file", 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.338 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.339 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.339 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.339 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.339 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.339 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.340 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.340 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.340 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.340 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.340 INFO:teuthology.orchestra.run.smithi178.stdout: "debug": { 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "debug", 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.341 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.342 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.342 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.342 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Enable/disable debug options", 2023-10-16T14:45:08.342 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.342 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.343 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.343 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.343 INFO:teuthology.orchestra.run.smithi178.stdout: "jwt_token_ttl": { 2023-10-16T14:45:08.343 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "jwt_token_ttl", 2023-10-16T14:45:08.343 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "28800", 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.344 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.345 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.345 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.345 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.345 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.345 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.346 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.347 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.347 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.347 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.347 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.347 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.348 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.348 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.348 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.348 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.348 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.349 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.350 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.351 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.351 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.351 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.351 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.351 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.352 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.353 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.353 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.353 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.353 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.353 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.355 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.355 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.355 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.355 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.355 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.356 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.356 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.356 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.356 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.356 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.357 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.358 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.358 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.358 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.358 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.358 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.359 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.359 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.359 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.359 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.359 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.360 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.361 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.361 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.361 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.361 INFO:teuthology.orchestra.run.smithi178.stdout: "motd": { 2023-10-16T14:45:08.361 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "motd", 2023-10-16T14:45:08.362 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.362 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.362 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.362 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.362 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.363 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.363 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.363 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "The message of the day", 2023-10-16T14:45:08.363 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.363 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.364 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.364 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.364 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect_resolve_ip_addr": { 2023-10-16T14:45:08.364 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "redirect_resolve_ip_addr", 2023-10-16T14:45:08.364 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.365 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.366 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.366 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.366 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.366 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.366 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.367 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:08.367 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:08.367 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.367 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.367 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.368 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.369 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.369 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.369 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.369 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:08.369 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8080", 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.370 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.371 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.372 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.373 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.373 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.373 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.373 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.373 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.374 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.374 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.374 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.374 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.374 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl_server_port": { 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl_server_port", 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8443", 2023-10-16T14:45:08.375 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.376 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.377 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.377 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:08.377 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:08.377 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.377 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "redirect", 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.378 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.379 INFO:teuthology.orchestra.run.smithi178.stdout: "redirect" 2023-10-16T14:45:08.379 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.379 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.379 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.379 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.380 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.381 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.381 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:08.381 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:08.381 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:08.381 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.382 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.382 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.382 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.382 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.382 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "url_prefix": { 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url_prefix", 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.383 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.384 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.384 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.384 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.384 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.384 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.385 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.386 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "devicehealth", 2023-10-16T14:45:08.386 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.386 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.386 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.386 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_monitoring": { 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_monitoring", 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.387 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.388 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.388 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.388 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "monitor device health metrics", 2023-10-16T14:45:08.388 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.388 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.389 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.390 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.390 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.390 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.390 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.390 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.391 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.391 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.391 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.391 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.391 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.392 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.392 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.392 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.392 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.392 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.393 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.394 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.394 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.394 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.394 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.394 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.395 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.396 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.397 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.397 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.397 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.397 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.397 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.398 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.399 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.399 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.399 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.399 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.399 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.400 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.401 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.401 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.401 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.401 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.401 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: "mark_out_threshold": { 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mark_out_threshold", 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.402 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.403 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.403 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "2419200", 2023-10-16T14:45:08.403 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.403 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.403 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.404 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-10-16T14:45:08.404 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.404 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.404 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: "pool_name": { 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pool_name", 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.405 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.406 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "device_health_metrics", 2023-10-16T14:45:08.406 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.406 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.406 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.406 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name of pool in which to store device health metrics", 2023-10-16T14:45:08.407 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.407 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.407 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.407 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.407 INFO:teuthology.orchestra.run.smithi178.stdout: "retention_period": { 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "retention_period", 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15552000", 2023-10-16T14:45:08.408 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.409 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.409 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.409 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long to retain device health metrics", 2023-10-16T14:45:08.409 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.409 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_frequency": { 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_frequency", 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.410 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.411 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.411 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:08.411 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.411 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.411 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to scrape device health metrics", 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.412 INFO:teuthology.orchestra.run.smithi178.stdout: "self_heal": { 2023-10-16T14:45:08.413 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "self_heal", 2023-10-16T14:45:08.413 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.413 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.413 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.413 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.414 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.414 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.414 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.414 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-10-16T14:45:08.414 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:08.415 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.416 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how frequently to wake up and check device health", 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.417 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.418 INFO:teuthology.orchestra.run.smithi178.stdout: "warn_threshold": { 2023-10-16T14:45:08.418 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "warn_threshold", 2023-10-16T14:45:08.418 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.418 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.418 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7257600", 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-10-16T14:45:08.419 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.420 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.420 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.420 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.420 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "diskprediction_local", 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.421 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.422 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.422 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.422 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.422 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.422 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.423 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.424 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.424 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.424 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.424 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.424 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.425 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.425 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.425 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.425 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.425 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.426 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.427 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.427 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.427 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.427 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.427 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.428 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.429 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.429 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.429 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.429 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.429 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.430 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.431 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.431 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.431 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.431 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.431 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.432 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.433 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.433 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.433 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.433 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.433 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.434 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.434 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.434 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.434 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.434 INFO:teuthology.orchestra.run.smithi178.stdout: "predict_interval": { 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predict_interval", 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "86400", 2023-10-16T14:45:08.435 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.436 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.436 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.436 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.436 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.436 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: "predictor_model": { 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "predictor_model", 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.437 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.438 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.438 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "prophetstor", 2023-10-16T14:45:08.438 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.438 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.438 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.439 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:08.440 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:08.440 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.440 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.440 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.440 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "600", 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.441 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.442 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.442 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.442 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.442 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "influx", 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": false, 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "influxdb python module not found", 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "batch_size": { 2023-10-16T14:45:08.443 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "batch_size", 2023-10-16T14:45:08.444 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.444 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.444 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.444 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5000", 2023-10-16T14:45:08.444 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.445 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.446 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.446 INFO:teuthology.orchestra.run.smithi178.stdout: "database": { 2023-10-16T14:45:08.446 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "database", 2023-10-16T14:45:08.446 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.446 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "ceph", 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.447 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-10-16T14:45:08.448 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.448 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.448 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.448 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.448 INFO:teuthology.orchestra.run.smithi178.stdout: "hostname": { 2023-10-16T14:45:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "hostname", 2023-10-16T14:45:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.449 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server hostname", 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.450 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.451 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.451 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.451 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:08.451 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:08.451 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "30", 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "5", 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.452 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.453 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.454 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.454 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.454 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.454 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.454 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.455 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.456 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.456 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.456 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.456 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.456 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.457 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.458 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.458 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.458 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.458 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.458 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.459 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.459 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.459 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.459 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.459 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.460 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.461 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.461 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.461 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.461 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.461 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.462 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.463 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.463 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.463 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.463 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.463 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.464 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.465 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.465 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.465 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.465 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.465 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.466 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.466 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.466 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.466 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.466 INFO:teuthology.orchestra.run.smithi178.stdout: "password": { 2023-10-16T14:45:08.467 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "password", 2023-10-16T14:45:08.467 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.467 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.467 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "password of InfluxDB server user", 2023-10-16T14:45:08.468 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.469 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.469 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.469 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.469 INFO:teuthology.orchestra.run.smithi178.stdout: "port": { 2023-10-16T14:45:08.469 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "port", 2023-10-16T14:45:08.470 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.470 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.470 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.470 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "8086", 2023-10-16T14:45:08.470 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "InfluxDB server port", 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.471 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.472 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.472 INFO:teuthology.orchestra.run.smithi178.stdout: "ssl": { 2023-10-16T14:45:08.472 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ssl", 2023-10-16T14:45:08.472 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.472 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.473 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.473 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "false", 2023-10-16T14:45:08.473 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.473 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.473 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.474 INFO:teuthology.orchestra.run.smithi178.stdout: "threads": { 2023-10-16T14:45:08.475 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threads", 2023-10-16T14:45:08.475 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.475 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.475 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.475 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "32", 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.476 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.477 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.477 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.477 INFO:teuthology.orchestra.run.smithi178.stdout: "username": { 2023-10-16T14:45:08.477 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "username", 2023-10-16T14:45:08.477 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.478 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.479 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "username of InfluxDB server user", 2023-10-16T14:45:08.479 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.479 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.479 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.479 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "verify_ssl": { 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "verify_ssl", 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.480 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "true", 2023-10-16T14:45:08.481 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.481 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.481 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.481 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-10-16T14:45:08.481 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.482 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.482 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.482 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.482 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "insights", 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.483 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.484 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.484 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.484 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.484 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.484 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.485 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.486 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.487 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.487 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.487 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.487 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.487 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.488 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.489 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.489 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.489 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.489 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.489 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.490 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.491 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.491 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.491 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.491 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.491 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.492 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.492 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.492 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.492 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.492 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.493 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.494 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.494 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.494 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.494 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.494 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.495 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.496 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.496 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.496 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.496 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.496 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "iostat", 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.497 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.498 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.498 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.498 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.498 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.498 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.499 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.499 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.499 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.499 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.499 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.500 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.501 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.501 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.501 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.501 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.501 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.502 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.503 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.503 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.503 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.503 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.503 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.504 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.504 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.504 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.504 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.504 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.505 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.506 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.506 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.506 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.506 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.506 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.507 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.508 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.509 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.509 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.509 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.509 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.509 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.510 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.510 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.510 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.510 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.510 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "k8sevents", 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.512 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.512 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_event_retention_days": { 2023-10-16T14:45:08.512 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "ceph_event_retention_days", 2023-10-16T14:45:08.512 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.512 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "7", 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.513 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Days to hold ceph event information within local cache", 2023-10-16T14:45:08.514 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.514 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.514 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.514 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.514 INFO:teuthology.orchestra.run.smithi178.stdout: "config_check_secs": { 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "config_check_secs", 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.515 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "10", 2023-10-16T14:45:08.516 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.516 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.516 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-10-16T14:45:08.516 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.516 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.517 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.518 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.518 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.518 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.518 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.518 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.519 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.520 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.521 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.521 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.521 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.522 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.522 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.522 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.522 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.522 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.523 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.523 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.523 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.523 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.523 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.524 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.524 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.524 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.524 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.524 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.525 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.526 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.526 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.526 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.526 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.526 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.527 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.528 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.528 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.528 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.528 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.528 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.529 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.530 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.530 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.530 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.530 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.530 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.531 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.531 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.531 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.531 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.531 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.532 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.532 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "localpool", 2023-10-16T14:45:08.532 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.532 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.532 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "failure_domain": { 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "failure_domain", 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.533 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "host", 2023-10-16T14:45:08.534 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.534 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.534 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.534 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "failure domain for any created local pool", 2023-10-16T14:45:08.534 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.535 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.536 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.536 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.536 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.536 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.536 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.537 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.538 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.538 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.538 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.538 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.538 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.539 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.539 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.539 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.539 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.539 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.540 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.541 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.542 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.542 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.542 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.542 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.542 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.543 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.544 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.544 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.544 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.544 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.544 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.545 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.546 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.546 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.546 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.546 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.546 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.547 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.547 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.547 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.547 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.547 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: "min_size": { 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "min_size", 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.548 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.549 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.549 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.549 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.549 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.549 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.550 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default min_size for any created local pool", 2023-10-16T14:45:08.550 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-10-16T14:45:08.550 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.550 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.550 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "num_rep": { 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "num_rep", 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.551 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3", 2023-10-16T14:45:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default replica count for any created local pool", 2023-10-16T14:45:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_num": { 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_num", 2023-10-16T14:45:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "128", 2023-10-16T14:45:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "default pg_num for any created local pool", 2023-10-16T14:45:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: "prefix": { 2023-10-16T14:45:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prefix", 2023-10-16T14:45:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.557 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "name prefix for any created local pool", 2023-10-16T14:45:08.558 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.558 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.558 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.558 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.558 INFO:teuthology.orchestra.run.smithi178.stdout: "subtree": { 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "subtree", 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "rack", 2023-10-16T14:45:08.559 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.560 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.560 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.560 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "CRUSH level for which to create a local pool", 2023-10-16T14:45:08.560 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-10-16T14:45:08.560 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.561 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.561 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.561 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.561 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.561 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mds_autoscaler", 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.562 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.563 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.563 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.563 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.563 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.563 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.564 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.564 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.564 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.564 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.564 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.565 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.566 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.566 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.566 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.566 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.566 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.567 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.568 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.568 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.568 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.568 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.568 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.569 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.570 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.570 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.570 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.570 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.570 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.571 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.572 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.572 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.572 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.572 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.572 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.573 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.574 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.574 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.574 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.574 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.574 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.575 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.576 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirroring", 2023-10-16T14:45:08.576 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.576 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.576 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.576 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.577 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.578 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.578 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.578 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.578 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.578 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.579 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.580 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.581 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.581 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.581 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.581 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.581 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.582 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.582 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.582 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.582 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.582 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.583 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.584 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.584 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.584 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.584 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.584 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.585 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.586 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.586 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.586 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.586 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.586 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.587 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.587 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.587 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.587 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.587 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.589 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.589 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.589 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.589 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "nfs", 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.590 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.591 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.591 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.591 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.591 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.591 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.592 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.593 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.593 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.593 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.593 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.593 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.594 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.595 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.596 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.597 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.597 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.597 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.597 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.597 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.598 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.599 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.599 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.599 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.599 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.599 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.600 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.600 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.600 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.600 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.600 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.601 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.602 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.602 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.602 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.602 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.602 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.603 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.604 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.604 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:08.604 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.604 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.604 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "fail_fs": { 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "fail_fs", 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.605 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.606 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.606 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.606 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.606 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-10-16T14:45:08.606 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.607 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.608 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.608 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.608 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.608 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.608 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.609 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.610 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.610 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.610 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.610 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.610 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.611 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.612 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.612 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.612 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.612 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.612 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.613 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.614 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.614 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.614 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.614 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.614 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.615 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.616 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.616 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.616 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.616 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.616 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.617 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.618 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.618 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.618 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.618 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.618 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.619 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.619 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.619 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.619 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.619 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator": { 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "orchestrator", 2023-10-16T14:45:08.620 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.621 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.622 INFO:teuthology.orchestra.run.smithi178.stdout: "cephadm", 2023-10-16T14:45:08.622 INFO:teuthology.orchestra.run.smithi178.stdout: "rook", 2023-10-16T14:45:08.622 INFO:teuthology.orchestra.run.smithi178.stdout: "test_orchestrator" 2023-10-16T14:45:08.622 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.622 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Orchestrator backend", 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.623 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.624 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.624 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_perf_query", 2023-10-16T14:45:08.624 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.624 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.624 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.625 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.625 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.625 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.625 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.625 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.626 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.627 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.627 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.627 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.627 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.627 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.628 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.629 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.629 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.629 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.629 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.629 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.630 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.631 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.631 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.631 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.631 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.631 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.632 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.633 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.633 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.633 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.633 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.633 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.634 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.634 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.634 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.634 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.634 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.635 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.636 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.637 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.637 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.637 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.637 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.637 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "osd_support", 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.638 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.639 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.639 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.639 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.639 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.639 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.640 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.641 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.641 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.641 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.641 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.641 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.642 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.643 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.643 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.643 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.643 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.643 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.644 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.644 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.644 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.644 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.644 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.645 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.646 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.646 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.646 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.646 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.646 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.647 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.648 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.648 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.648 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.648 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.648 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.649 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.650 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.650 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.650 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.650 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.650 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.651 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.652 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.652 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.652 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "pg_autoscaler", 2023-10-16T14:45:08.652 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.652 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.653 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.654 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.654 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.654 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.654 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.654 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.655 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.656 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.657 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.658 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.658 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.658 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.658 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.658 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.659 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.659 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.659 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.659 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.659 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.660 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.661 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.661 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.661 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.661 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.661 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.662 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.663 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.663 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.663 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.663 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.663 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.664 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: "threshold": { 2023-10-16T14:45:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "threshold", 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "3.0", 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1.0", 2023-10-16T14:45:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "scaling threshold", 2023-10-16T14:45:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-10-16T14:45:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "progress", 2023-10-16T14:45:08.671 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.672 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.672 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.672 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_pg_recovery_event": { 2023-10-16T14:45:08.672 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_pg_recovery_event", 2023-10-16T14:45:08.672 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.673 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.674 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow the module to show pg recovery progress", 2023-10-16T14:45:08.674 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.674 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.674 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.674 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.675 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.676 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.676 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.676 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.676 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.676 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: "max_completed_events": { 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_completed_events", 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.691 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.691 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "50", 2023-10-16T14:45:08.691 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.691 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.691 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.692 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "number of past completed events to remember", 2023-10-16T14:45:08.692 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.692 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.692 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.692 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "sleep_interval": { 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "sleep_interval", 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.693 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "5", 2023-10-16T14:45:08.694 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.694 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.694 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.694 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "how long the module is going to sleep", 2023-10-16T14:45:08.694 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.695 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.696 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "prometheus", 2023-10-16T14:45:08.696 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.696 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.696 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.696 INFO:teuthology.orchestra.run.smithi178.stdout: "cache": { 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "cache", 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.697 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.698 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.698 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.698 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.698 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.698 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.699 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.699 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.699 INFO:teuthology.orchestra.run.smithi178.stdout: "exclude_perf_counters": { 2023-10-16T14:45:08.699 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "exclude_perf_counters", 2023-10-16T14:45:08.699 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.700 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.701 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-10-16T14:45:08.701 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-10-16T14:45:08.701 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.701 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.701 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.702 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.703 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.703 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.703 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.703 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.703 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.704 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.704 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.704 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.704 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.704 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.705 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.706 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.706 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.706 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.706 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.706 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.707 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.708 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.709 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.710 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.710 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.710 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.710 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.710 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.711 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.712 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.712 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.712 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.712 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.712 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.713 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.714 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools": { 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools", 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.715 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.716 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_stats_pools_refresh_interval": { 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.717 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300", 2023-10-16T14:45:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.718 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.719 INFO:teuthology.orchestra.run.smithi178.stdout: "scrape_interval": { 2023-10-16T14:45:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "scrape_interval", 2023-10-16T14:45:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.720 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15.0", 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.721 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:08.722 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "::", 2023-10-16T14:45:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.723 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.724 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.725 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "9283", 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-10-16T14:45:08.726 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.727 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.727 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.727 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.727 INFO:teuthology.orchestra.run.smithi178.stdout: "stale_cache_strategy": { 2023-10-16T14:45:08.727 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stale_cache_strategy", 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "log", 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.728 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.729 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.729 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.729 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.729 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.729 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.730 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.730 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_behaviour": { 2023-10-16T14:45:08.730 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_behaviour", 2023-10-16T14:45:08.730 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.730 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "default", 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.731 INFO:teuthology.orchestra.run.smithi178.stdout: "default", 2023-10-16T14:45:08.732 INFO:teuthology.orchestra.run.smithi178.stdout: "error" 2023-10-16T14:45:08.732 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.732 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.732 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.732 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: "standby_error_status_code": { 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "standby_error_status_code", 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.733 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.734 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.734 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "500", 2023-10-16T14:45:08.734 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "400", 2023-10-16T14:45:08.734 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "599", 2023-10-16T14:45:08.734 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.735 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.736 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.736 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.736 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_support", 2023-10-16T14:45:08.736 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.736 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.737 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.738 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.738 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.738 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.738 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.738 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.739 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.739 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.739 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.739 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.739 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.740 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.740 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.740 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.740 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.740 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.741 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.742 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.742 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.742 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.742 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.742 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.743 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.744 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.745 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.745 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.745 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.745 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.745 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.746 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.747 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.747 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.747 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.747 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.747 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.748 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.749 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.749 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.749 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.749 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.749 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_snap_create": { 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_snap_create", 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.750 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.751 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10", 2023-10-16T14:45:08.751 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.751 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.751 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.751 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: "mirror_snapshot_schedule": { 2023-10-16T14:45:08.752 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "mirror_snapshot_schedule", 2023-10-16T14:45:08.753 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.753 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.753 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.753 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.753 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.754 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: "trash_purge_schedule": { 2023-10-16T14:45:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "trash_purge_schedule", 2023-10-16T14:45:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.755 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.756 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.757 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.757 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.757 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.757 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "restful", 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.758 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.759 INFO:teuthology.orchestra.run.smithi178.stdout: "enable_auth": { 2023-10-16T14:45:08.759 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enable_auth", 2023-10-16T14:45:08.759 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.759 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.759 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.760 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.761 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.761 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.761 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.761 INFO:teuthology.orchestra.run.smithi178.stdout: "key_file": { 2023-10-16T14:45:08.761 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "key_file", 2023-10-16T14:45:08.762 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.762 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.762 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.762 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.762 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.763 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.764 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.764 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.764 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.764 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.764 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.765 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.766 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.766 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.766 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.766 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.766 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.767 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.768 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.768 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.768 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.768 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.768 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.769 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.770 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.770 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.770 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.770 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.770 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.771 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.772 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.772 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.772 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.772 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.772 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.773 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.774 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.774 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.774 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.774 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.774 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.775 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.775 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.775 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.775 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.775 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.776 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "server_addr": { 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_addr", 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.777 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.778 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.778 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.778 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.778 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.778 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.779 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.779 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.779 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.779 INFO:teuthology.orchestra.run.smithi178.stdout: "server_port": { 2023-10-16T14:45:08.779 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "server_port", 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.780 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.781 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.781 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.781 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.781 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.781 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rgw", 2023-10-16T14:45:08.782 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.783 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.783 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.783 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.783 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.783 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.784 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.785 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.785 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.785 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.785 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.785 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.786 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.787 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.787 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.787 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.787 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.787 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.788 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.789 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.789 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.789 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.789 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.789 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.790 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.791 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.792 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.793 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.794 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.794 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.794 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.794 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.795 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.796 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.796 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.796 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.796 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rook", 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "drive_group_interval": { 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "drive_group_interval", 2023-10-16T14:45:08.797 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.798 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.798 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.798 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "300.0", 2023-10-16T14:45:08.798 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.798 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.799 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.799 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-10-16T14:45:08.799 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.799 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.799 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.800 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.800 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.800 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.800 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.800 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.801 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.801 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.801 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.801 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.801 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.802 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.803 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.803 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.803 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.803 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.803 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.804 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.805 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.805 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.805 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.805 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.805 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.806 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.807 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.807 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.807 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.807 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.807 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.808 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.809 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.809 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.809 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.809 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.809 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.810 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.811 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.811 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.811 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.811 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.811 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.812 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: "storage_class": { 2023-10-16T14:45:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "storage_class", 2023-10-16T14:45:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.813 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "local", 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-10-16T14:45:08.814 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.815 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.816 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "selftest", 2023-10-16T14:45:08.816 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.816 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.816 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.816 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.817 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.817 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.817 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.817 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.817 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.818 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.820 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.821 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.821 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.821 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.821 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.821 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.822 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.823 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.823 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.823 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.823 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.823 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.824 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.825 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.825 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.825 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.825 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.825 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.826 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.827 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.827 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.827 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.827 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.827 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.828 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.829 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.829 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.829 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.829 INFO:teuthology.orchestra.run.smithi178.stdout: "roption1": { 2023-10-16T14:45:08.829 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption1", 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.830 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.831 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.831 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.831 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.831 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.831 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: "roption2": { 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "roption2", 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "xyz", 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.833 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption1": { 2023-10-16T14:45:08.834 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption1", 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.835 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.836 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.836 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.836 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.836 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.836 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption2": { 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption2", 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.837 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.838 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.838 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.838 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.838 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.838 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption3": { 2023-10-16T14:45:08.839 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption3", 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "float", 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.840 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.841 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.841 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.841 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.841 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.841 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption4": { 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption4", 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.842 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.843 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.843 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.843 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.843 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.843 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption5": { 2023-10-16T14:45:08.844 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption5", 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.845 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.846 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.846 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.846 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.846 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.846 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption6": { 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption6", 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.847 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.848 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.848 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.848 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.848 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.848 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: "rwoption7": { 2023-10-16T14:45:08.849 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rwoption7", 2023-10-16T14:45:08.850 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.850 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.850 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.850 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.850 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "1", 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "42", 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.851 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.852 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.852 INFO:teuthology.orchestra.run.smithi178.stdout: "testkey": { 2023-10-16T14:45:08.852 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testkey", 2023-10-16T14:45:08.852 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.852 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.853 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.854 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.854 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.854 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.854 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.854 INFO:teuthology.orchestra.run.smithi178.stdout: "testlkey": { 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testlkey", 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.855 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.856 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.856 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.856 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.856 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.857 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.857 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.858 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.858 INFO:teuthology.orchestra.run.smithi178.stdout: "testnewline": { 2023-10-16T14:45:08.858 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "testnewline", 2023-10-16T14:45:08.858 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.858 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.859 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.859 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.859 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.859 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.859 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.860 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.861 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.861 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.861 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snap_schedule", 2023-10-16T14:45:08.861 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.861 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "allow_m_granularity": { 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "allow_m_granularity", 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.862 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.863 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.863 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.863 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.863 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.863 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "allow minute scheduled snapshots", 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: "dump_on_update": { 2023-10-16T14:45:08.864 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "dump_on_update", 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.865 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.866 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.866 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "dump database to debug log on update", 2023-10-16T14:45:08.866 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.866 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.866 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.867 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.868 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.868 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.868 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.868 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.868 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.869 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.870 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.870 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.870 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.870 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.870 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.871 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.872 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.872 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.872 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.872 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.872 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.873 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.874 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.875 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.875 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.875 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.875 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.875 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.876 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.877 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.877 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.877 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.877 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.877 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.878 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.879 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.879 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.879 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.879 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.879 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "stats", 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.880 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.881 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.882 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.882 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.882 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.882 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.882 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.883 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.884 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.884 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.884 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.884 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.884 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.885 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.886 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.886 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.886 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.886 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.886 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.887 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.888 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.889 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.889 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.889 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.889 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.889 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.890 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.891 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.891 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.891 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.891 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.891 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.892 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.893 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.893 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.893 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.893 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "status", 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.894 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.895 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.895 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.895 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.895 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.895 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.896 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.897 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.898 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.899 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.900 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.900 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.900 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.900 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.900 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.901 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.902 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.902 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.902 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.902 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.902 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.904 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.905 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.905 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.905 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.905 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.905 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.906 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.907 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.907 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.907 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.907 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.907 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telegraf", 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "address": { 2023-10-16T14:45:08.908 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "address", 2023-10-16T14:45:08.909 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.909 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.909 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.909 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-10-16T14:45:08.909 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.910 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.911 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.912 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "15", 2023-10-16T14:45:08.912 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.912 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.912 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.912 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.913 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.914 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.914 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.914 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.914 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.914 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.915 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.916 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.916 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.916 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.916 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.916 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.917 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.918 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.918 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.918 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.918 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.918 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.919 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.920 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.921 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.921 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.921 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.921 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.921 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.922 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.922 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.922 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.922 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.922 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.923 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.924 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.925 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.925 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.925 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.925 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.925 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.926 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "telemetry", 2023-10-16T14:45:08.929 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_basic": { 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_basic", 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.930 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.931 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.931 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.931 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.931 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.931 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share basic cluster information (size, version)", 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.932 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_crash": { 2023-10-16T14:45:08.933 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_crash", 2023-10-16T14:45:08.933 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.933 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.933 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.933 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.934 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.935 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.935 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.935 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_device": { 2023-10-16T14:45:08.935 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_device", 2023-10-16T14:45:08.935 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "True", 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.936 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.937 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-10-16T14:45:08.937 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.937 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.937 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.937 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_ident": { 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_ident", 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.938 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.939 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.939 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.939 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.939 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-10-16T14:45:08.939 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: "channel_perf": { 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "channel_perf", 2023-10-16T14:45:08.940 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.941 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.942 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Share various performance metrics of a cluster", 2023-10-16T14:45:08.942 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.942 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.942 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.942 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "contact": { 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "contact", 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.943 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.944 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.945 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.945 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.945 INFO:teuthology.orchestra.run.smithi178.stdout: "description": { 2023-10-16T14:45:08.945 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "description", 2023-10-16T14:45:08.945 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.946 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.947 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "device_url": { 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "device_url", 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.948 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-10-16T14:45:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.949 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "enabled": { 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "enabled", 2023-10-16T14:45:08.950 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.951 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.952 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.953 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "24", 2023-10-16T14:45:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "8", 2023-10-16T14:45:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.954 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "last_opt_revision": { 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "last_opt_revision", 2023-10-16T14:45:08.955 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "1", 2023-10-16T14:45:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.956 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.957 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard": { 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard", 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.958 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.959 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "leaderboard_description": { 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "leaderboard_description", 2023-10-16T14:45:08.960 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.961 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.962 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.963 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.964 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.965 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.966 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.967 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.968 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.969 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.970 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.970 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.970 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.970 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.970 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.971 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.972 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.972 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.972 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.972 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.972 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.973 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.974 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.974 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.974 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.974 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.974 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: "organization": { 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "organization", 2023-10-16T14:45:08.975 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.976 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.976 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.976 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.976 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.976 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.977 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "proxy": { 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "proxy", 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.978 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.979 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.979 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.979 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.979 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.979 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: "url": { 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "url", 2023-10-16T14:45:08.980 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.981 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.982 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.982 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.982 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.982 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.982 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "test_orchestrator", 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.983 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.984 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.984 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.984 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.984 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.984 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.985 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.986 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.986 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.986 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.986 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.986 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.988 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:08.988 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:08.988 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.988 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.988 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.989 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.990 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.990 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.990 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:08.990 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.991 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.992 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.992 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:08.992 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:08.992 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:08.992 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.993 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:08.994 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.995 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.995 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:08.995 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.995 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.995 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.996 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:08.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:08.997 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:08.997 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "volumes", 2023-10-16T14:45:08.997 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:08.997 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:08.998 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:08.999 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:08.999 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:08.999 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:08.999 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:08.999 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:09.000 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.001 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:09.002 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:09.002 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:09.002 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.002 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.002 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.003 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.004 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.004 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:09.004 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:09.004 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.005 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:09.006 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:09.006 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:09.006 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:09.006 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:09.006 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.007 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.008 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.009 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:09.009 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.009 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.009 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.009 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: "max_concurrent_clones": { 2023-10-16T14:45:09.010 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "max_concurrent_clones", 2023-10-16T14:45:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "4", 2023-10-16T14:45:09.011 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Number of asynchronous cloner threads", 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.012 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: "periodic_async_work": { 2023-10-16T14:45:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "periodic_async_work", 2023-10-16T14:45:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:09.013 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.014 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Periodically check for async work", 2023-10-16T14:45:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.015 INFO:teuthology.orchestra.run.smithi178.stdout: "snapshot_clone_delay": { 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "snapshot_clone_delay", 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "0", 2023-10-16T14:45:09.016 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.017 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.018 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.018 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.018 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.018 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.018 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix", 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "can_run": true, 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "error_string": "", 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "module_options": { 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "discovery_interval": { 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "discovery_interval", 2023-10-16T14:45:09.019 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "uint", 2023-10-16T14:45:09.020 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.020 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.020 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "100", 2023-10-16T14:45:09.020 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.020 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.021 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "identifier": { 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "identifier", 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.022 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:09.023 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.023 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.023 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.023 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.023 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: "interval": { 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "interval", 2023-10-16T14:45:09.024 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "secs", 2023-10-16T14:45:09.025 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.025 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.025 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "60", 2023-10-16T14:45:09.025 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.025 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.026 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "log_level": { 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_level", 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.027 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:09.028 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.028 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.028 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:09.028 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:09.028 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.029 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.030 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.030 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.030 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.030 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.030 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster": { 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster", 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.032 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.033 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.033 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_cluster_level": { 2023-10-16T14:45:09.033 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_cluster_level", 2023-10-16T14:45:09.033 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.033 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "info", 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [ 2023-10-16T14:45:09.034 INFO:teuthology.orchestra.run.smithi178.stdout: "", 2023-10-16T14:45:09.035 INFO:teuthology.orchestra.run.smithi178.stdout: "critical", 2023-10-16T14:45:09.035 INFO:teuthology.orchestra.run.smithi178.stdout: "debug", 2023-10-16T14:45:09.035 INFO:teuthology.orchestra.run.smithi178.stdout: "error", 2023-10-16T14:45:09.035 INFO:teuthology.orchestra.run.smithi178.stdout: "info", 2023-10-16T14:45:09.035 INFO:teuthology.orchestra.run.smithi178.stdout: "warning" 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.036 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.037 INFO:teuthology.orchestra.run.smithi178.stdout: "log_to_file": { 2023-10-16T14:45:09.037 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "log_to_file", 2023-10-16T14:45:09.037 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "bool", 2023-10-16T14:45:09.037 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.037 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 1, 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "False", 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.038 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_host": { 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_host", 2023-10-16T14:45:09.039 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.040 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.040 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.040 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "", 2023-10-16T14:45:09.040 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.040 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.041 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.042 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_port": { 2023-10-16T14:45:09.042 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_port", 2023-10-16T14:45:09.042 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "int", 2023-10-16T14:45:09.042 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.042 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "10051", 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.043 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: "zabbix_sender": { 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "zabbix_sender", 2023-10-16T14:45:09.044 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "str", 2023-10-16T14:45:09.045 INFO:teuthology.orchestra.run.smithi178.stdout: "level": "advanced", 2023-10-16T14:45:09.045 INFO:teuthology.orchestra.run.smithi178.stdout: "flags": 0, 2023-10-16T14:45:09.045 INFO:teuthology.orchestra.run.smithi178.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-10-16T14:45:09.045 INFO:teuthology.orchestra.run.smithi178.stdout: "min": "", 2023-10-16T14:45:09.045 INFO:teuthology.orchestra.run.smithi178.stdout: "max": "", 2023-10-16T14:45:09.046 INFO:teuthology.orchestra.run.smithi178.stdout: "enum_allowed": [], 2023-10-16T14:45:09.046 INFO:teuthology.orchestra.run.smithi178.stdout: "desc": "", 2023-10-16T14:45:09.046 INFO:teuthology.orchestra.run.smithi178.stdout: "long_desc": "", 2023-10-16T14:45:09.046 INFO:teuthology.orchestra.run.smithi178.stdout: "tags": [], 2023-10-16T14:45:09.046 INFO:teuthology.orchestra.run.smithi178.stdout: "see_also": [] 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: "services": { 2023-10-16T14:45:09.047 INFO:teuthology.orchestra.run.smithi178.stdout: "dashboard": "https://172.21.15.178:8443/" 2023-10-16T14:45:09.048 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.048 INFO:teuthology.orchestra.run.smithi178.stdout: "always_on_modules": { 2023-10-16T14:45:09.048 INFO:teuthology.orchestra.run.smithi178.stdout: "octopus": [ 2023-10-16T14:45:09.048 INFO:teuthology.orchestra.run.smithi178.stdout: "balancer", 2023-10-16T14:45:09.048 INFO:teuthology.orchestra.run.smithi178.stdout: "crash", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "devicehealth", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_autoscaler", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "progress", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_support", 2023-10-16T14:45:09.049 INFO:teuthology.orchestra.run.smithi178.stdout: "status", 2023-10-16T14:45:09.050 INFO:teuthology.orchestra.run.smithi178.stdout: "telemetry", 2023-10-16T14:45:09.050 INFO:teuthology.orchestra.run.smithi178.stdout: "volumes" 2023-10-16T14:45:09.050 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.050 INFO:teuthology.orchestra.run.smithi178.stdout: "pacific": [ 2023-10-16T14:45:09.050 INFO:teuthology.orchestra.run.smithi178.stdout: "balancer", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "crash", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "devicehealth", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_autoscaler", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "progress", 2023-10-16T14:45:09.051 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_support", 2023-10-16T14:45:09.052 INFO:teuthology.orchestra.run.smithi178.stdout: "status", 2023-10-16T14:45:09.052 INFO:teuthology.orchestra.run.smithi178.stdout: "telemetry", 2023-10-16T14:45:09.052 INFO:teuthology.orchestra.run.smithi178.stdout: "volumes" 2023-10-16T14:45:09.052 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: "quincy": [ 2023-10-16T14:45:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: "balancer", 2023-10-16T14:45:09.053 INFO:teuthology.orchestra.run.smithi178.stdout: "crash", 2023-10-16T14:45:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: "devicehealth", 2023-10-16T14:45:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator", 2023-10-16T14:45:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_autoscaler", 2023-10-16T14:45:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: "progress", 2023-10-16T14:45:09.054 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_support", 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: "status", 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: "telemetry", 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: "volumes" 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: ], 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: "reef": [ 2023-10-16T14:45:09.055 INFO:teuthology.orchestra.run.smithi178.stdout: "balancer", 2023-10-16T14:45:09.056 INFO:teuthology.orchestra.run.smithi178.stdout: "crash", 2023-10-16T14:45:09.056 INFO:teuthology.orchestra.run.smithi178.stdout: "devicehealth", 2023-10-16T14:45:09.056 INFO:teuthology.orchestra.run.smithi178.stdout: "orchestrator", 2023-10-16T14:45:09.056 INFO:teuthology.orchestra.run.smithi178.stdout: "pg_autoscaler", 2023-10-16T14:45:09.057 INFO:teuthology.orchestra.run.smithi178.stdout: "progress", 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: "rbd_support", 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: "status", 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: "telemetry", 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: "volumes" 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.058 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.059 INFO:teuthology.orchestra.run.smithi178.stdout: "last_failure_osd_epoch": 3, 2023-10-16T14:45:09.059 INFO:teuthology.orchestra.run.smithi178.stdout: "active_clients": [ 2023-10-16T14:45:09.059 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.059 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "devicehealth", 2023-10-16T14:45:09.059 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 3582286393 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.060 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "libcephsqlite", 2023-10-16T14:45:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2023-10-16T14:45:09.061 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 664493085 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.062 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.063 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "rbd_support", 2023-10-16T14:45:09.063 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2023-10-16T14:45:09.063 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.063 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 48642049 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:09.064 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.065 INFO:teuthology.orchestra.run.smithi178.stdout: "name": "volumes", 2023-10-16T14:45:09.065 INFO:teuthology.orchestra.run.smithi178.stdout: "addrvec": [ 2023-10-16T14:45:09.065 INFO:teuthology.orchestra.run.smithi178.stdout: { 2023-10-16T14:45:09.065 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v2", 2023-10-16T14:45:09.065 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "172.21.15.178:0", 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 2750452472 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2023-10-16T14:45:09.066 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:45:09.068 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/814933225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:09.069 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:05 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1033034778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:09.069 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:06 smithi187 ceph-mon[135297]: pgmap v680: 129 pgs: 129 active+clean; 675 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-10-16T14:45:09.069 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:06 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/851248313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-16T14:45:09.069 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:08 smithi187 ceph-mon[135297]: pgmap v681: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 355 op/s 2023-10-16T14:45:09.069 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/814933225' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:09.070 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:05 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1033034778' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:09.070 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:06 smithi186 ceph-mon[134892]: pgmap v680: 129 pgs: 129 active+clean; 675 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-10-16T14:45:09.070 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:06 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/851248313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-16T14:45:09.070 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:08 smithi186 ceph-mon[134892]: pgmap v681: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 355 op/s 2023-10-16T14:45:09.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 msgr2=0x7f29bc0a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:09.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 0x7f29bc0a3a60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f29c000b2e0 tx=0x7f29c0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 msgr2=0x7f29d0198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:09.071 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f29cc00e9a0 tx=0x7f29cc00ecb0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 shutdown_connections 2023-10-16T14:45:09.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f29bc0a15a0 0x7f29bc0a3a60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f29d0108dc0 0x7f29d01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0073910 0x7f29d0198db0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.072 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 --2- 172.21.15.178:0/851248313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29d0074ef0 0x7f29d01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:09.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 >> 172.21.15.178:0/851248313 conn(0x7f29d00fc430 msgr2=0x7f29d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:09.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 shutdown_connections 2023-10-16T14:45:09.073 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:05.307+0000 7f29d73ff700 1 -- 172.21.15.178:0/851248313 wait complete. 2023-10-16T14:45:09.079 DEBUG:tasks.check_counter:Getting stats from a 2023-10-16T14:45:09.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.a perf dump 2023-10-16T14:45:09.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:06 smithi178 ceph-mon[98495]: pgmap v680: 129 pgs: 129 active+clean; 675 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 335 op/s 2023-10-16T14:45:09.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:06 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/851248313' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-10-16T14:45:09.080 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:08 smithi178 ceph-mon[98495]: pgmap v681: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 355 op/s 2023-10-16T14:45:09.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:45:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:09 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:45:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:09 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-10-16T14:45:09.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:09 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:09.829 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:45:10.290 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:10 smithi178 ceph-mon[98495]: pgmap v682: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 817 KiB/s wr, 285 op/s 2023-10-16T14:45:10.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:10 smithi186 ceph-mon[134892]: pgmap v682: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 817 KiB/s wr, 285 op/s 2023-10-16T14:45:10.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:10 smithi187 ceph-mon[135297]: pgmap v682: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 817 KiB/s wr, 285 op/s 2023-10-16T14:45:11.834 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:45:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 0, 2023-10-16T14:45:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2, 2023-10-16T14:45:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 0, 2023-10-16T14:45:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 0, 2023-10-16T14:45:11.835 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.003692213, 2023-10-16T14:45:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000000000, 2023-10-16T14:45:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000718486, 2023-10-16T14:45:11.836 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:11.837 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.838 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 224, 2023-10-16T14:45:11.839 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 221, 2023-10-16T14:45:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 474032, 2023-10-16T14:45:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 132432, 2023-10-16T14:45:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:11.840 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.050602661, 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.022680240, 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.020878945, 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.007413552, 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 220, 2023-10-16T14:45:11.841 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.033439105, 2023-10-16T14:45:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000151995 2023-10-16T14:45:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:11.842 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 474032, 2023-10-16T14:45:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 132432 2023-10-16T14:45:11.843 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 5, 2023-10-16T14:45:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 170, 2023-10-16T14:45:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3104, 2023-10-16T14:45:11.844 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 106464, 2023-10-16T14:45:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.022119483, 2023-10-16T14:45:11.845 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.018227793, 2023-10-16T14:45:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.001800777, 2023-10-16T14:45:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 169, 2023-10-16T14:45:11.846 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.022082916, 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000130668 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:11.847 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-16T14:45:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 106464 2023-10-16T14:45:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.848 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-16T14:45:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2023-10-16T14:45:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-16T14:45:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-16T14:45:11.849 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:11.850 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:11.851 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:11.852 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:11.853 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:11.854 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-16T14:45:11.855 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 148984, 2023-10-16T14:45:11.856 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 30, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:11.857 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2023-10-16T14:45:11.858 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-16T14:45:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:11.859 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2023-10-16T14:45:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-16T14:45:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-16T14:45:11.860 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 225, 2023-10-16T14:45:11.861 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 442362, 2023-10-16T14:45:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 225, 2023-10-16T14:45:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-16T14:45:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-16T14:45:11.862 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 225, 2023-10-16T14:45:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 442362, 2023-10-16T14:45:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-16T14:45:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:11.863 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:11.864 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:11.864 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:45:12.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:11 smithi187 ceph-mon[135297]: pgmap v683: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 804 KiB/s wr, 244 op/s 2023-10-16T14:45:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:11 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:11 smithi178 ceph-mon[98495]: pgmap v683: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 804 KiB/s wr, 244 op/s 2023-10-16T14:45:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:11 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:11 smithi186 ceph-mon[134892]: pgmap v683: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 804 KiB/s wr, 244 op/s 2023-10-16T14:45:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:11 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:12.453 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-10-16T14:45:12.536 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-10-16T14:45:12.537 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-10-16T14:45:12.537 DEBUG:tasks.check_counter:Getting stats from d 2023-10-16T14:45:12.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.d perf dump 2023-10-16T14:45:13.296 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:45:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:13 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:13 smithi187 ceph-mon[135297]: pgmap v684: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 273 KiB/s wr, 151 op/s 2023-10-16T14:45:14.274 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.274 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:13 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.275 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:13 smithi178 ceph-mon[98495]: pgmap v684: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 273 KiB/s wr, 151 op/s 2023-10-16T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:13 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:14.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:13 smithi186 ceph-mon[134892]: pgmap v684: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 273 KiB/s wr, 151 op/s 2023-10-16T14:45:14.980 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:45:14.981 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:14.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 233, 2023-10-16T14:45:14.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 228, 2023-10-16T14:45:14.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 478336, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 136000, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.045869963, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.014291099, 2023-10-16T14:45:14.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.021992197, 2023-10-16T14:45:14.983 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.006851640, 2023-10-16T14:45:14.983 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:14.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 228, 2023-10-16T14:45:14.983 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.033642966, 2023-10-16T14:45:14.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000147556 2023-10-16T14:45:14.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.984 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:14.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:14.984 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:14.984 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 478336, 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 136000 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:14.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1, 2023-10-16T14:45:14.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 172, 2023-10-16T14:45:14.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 160, 2023-10-16T14:45:14.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 106720, 2023-10-16T14:45:14.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:14.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.018526062, 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.014720584, 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.001784610, 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:14.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 170, 2023-10-16T14:45:14.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.024232231, 2023-10-16T14:45:14.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000142542 2023-10-16T14:45:14.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.988 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:14.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:14.989 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:14.989 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:14.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.989 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:14.989 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 106720 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 0, 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2, 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 0, 2023-10-16T14:45:14.990 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 0, 2023-10-16T14:45:14.991 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:14.991 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:14.991 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.001267509, 2023-10-16T14:45:14.991 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000000000, 2023-10-16T14:45:14.991 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.001073915, 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:14.992 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.993 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:14.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:14.993 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:14.993 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:14.993 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.994 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:14.994 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:14.994 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.994 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-16T14:45:14.994 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:14.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:14.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:14.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:14.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:14.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:14.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:14.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:14.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:14.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:14.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:14.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:14.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:14.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:14.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:14.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:15.000 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:15.000 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:15.000 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:15.000 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:15.000 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-16T14:45:15.001 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:15.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:15.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:15.002 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:15.002 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 148967, 2023-10-16T14:45:15.002 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 29, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:15.003 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:15.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:15.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:15.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:15.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2023-10-16T14:45:15.004 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2023-10-16T14:45:15.005 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:15.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-16T14:45:15.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:15.006 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-16T14:45:15.006 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:15.006 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 230, 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 443012, 2023-10-16T14:45:15.007 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:15.008 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 230, 2023-10-16T14:45:15.008 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-16T14:45:15.008 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-16T14:45:15.008 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 230, 2023-10-16T14:45:15.008 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 443012, 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:15.009 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:15.010 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:45:15.010 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:45:15.010 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:45:15.011 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:14 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:15.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:45:15.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:45:15.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:14 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:15.325 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-10-16T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-10-16T14:45:15.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:14 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' 2023-10-16T14:45:15.491 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-10-16T14:45:15.491 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-10-16T14:45:15.492 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-10-16T14:45:15.492 DEBUG:tasks.check_counter:Getting stats from g 2023-10-16T14:45:15.492 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.g perf dump 2023-10-16T14:45:16.174 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:15 smithi187 ceph-mon[135297]: pgmap v685: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 261 KiB/s wr, 105 op/s 2023-10-16T14:45:16.233 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:45:16.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:15 smithi178 ceph-mon[98495]: pgmap v685: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 261 KiB/s wr, 105 op/s 2023-10-16T14:45:16.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:15 smithi186 ceph-mon[134892]: pgmap v685: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 261 KiB/s wr, 105 op/s 2023-10-16T14:45:17.965 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:45:17.966 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:17.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 0, 2023-10-16T14:45:17.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2, 2023-10-16T14:45:17.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 0, 2023-10-16T14:45:17.966 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 0, 2023-10-16T14:45:17.967 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:17.967 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:17.967 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.003561650, 2023-10-16T14:45:17.967 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000000000, 2023-10-16T14:45:17.967 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000725670, 2023-10-16T14:45:17.968 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:17.968 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:17.968 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:17.968 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:17.969 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:17.969 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.969 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:17.969 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:17.969 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:17.970 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:17.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.970 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:17.970 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:17.970 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.971 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:17.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 1, 2023-10-16T14:45:17.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 170, 2023-10-16T14:45:17.971 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 160, 2023-10-16T14:45:17.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 106368, 2023-10-16T14:45:17.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:17.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:17.972 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.020024767, 2023-10-16T14:45:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.015679673, 2023-10-16T14:45:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002098055, 2023-10-16T14:45:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:17.973 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 168, 2023-10-16T14:45:17.974 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.025926265, 2023-10-16T14:45:17.974 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000154323 2023-10-16T14:45:17.974 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.974 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:17.975 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:17.975 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:17.975 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:17.975 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:17.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 106368 2023-10-16T14:45:17.976 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.976 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:17.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 231, 2023-10-16T14:45:17.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 226, 2023-10-16T14:45:17.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 477792, 2023-10-16T14:45:17.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 134752, 2023-10-16T14:45:17.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:17.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:17.978 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.056959189, 2023-10-16T14:45:17.978 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.026716211, 2023-10-16T14:45:17.978 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.018041632, 2023-10-16T14:45:17.978 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.009213718, 2023-10-16T14:45:17.979 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:17.979 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 226, 2023-10-16T14:45:17.979 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.037743637, 2023-10-16T14:45:17.979 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000167007 2023-10-16T14:45:17.980 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.980 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:17.980 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:17.980 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:17.980 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:17.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 477792, 2023-10-16T14:45:17.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 134752 2023-10-16T14:45:17.981 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.981 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-16T14:45:17.982 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2023-10-16T14:45:17.982 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-16T14:45:17.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.982 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-16T14:45:17.983 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:17.983 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:17.983 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:17.983 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:17.983 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:17.984 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:17.985 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:17.985 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:17.985 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:17.985 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:17.985 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:17.986 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:17.986 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:17.986 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:17.986 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:17.986 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:17.987 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:17.988 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:17.988 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:17.988 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:17.988 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:17.988 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 148967, 2023-10-16T14:45:17.989 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 29, 2023-10-16T14:45:17.990 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:17.990 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:17.990 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-16T14:45:17.990 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-16T14:45:17.990 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:17.991 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:17.991 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:17.994 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:17.994 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:17.995 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2023-10-16T14:45:17.995 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:17.995 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:17.995 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:17.995 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-16T14:45:17.996 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:17.996 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2023-10-16T14:45:17.996 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:17.996 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-16T14:45:17.996 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:17.997 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-16T14:45:17.997 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:17.997 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:17.997 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-16T14:45:17.997 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 228, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 442752, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 228, 2023-10-16T14:45:17.998 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-16T14:45:17.999 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-16T14:45:17.999 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 228, 2023-10-16T14:45:17.999 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 442752, 2023-10-16T14:45:17.999 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:18.000 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:45:18.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:18 smithi178 ceph-mon[98495]: pgmap v686: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2023-10-16T14:45:18.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:18 smithi186 ceph-mon[134892]: pgmap v686: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2023-10-16T14:45:18.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:18 smithi187 ceph-mon[135297]: pgmap v686: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.9 KiB/s wr, 50 op/s 2023-10-16T14:45:18.527 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-10-16T14:45:18.528 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-10-16T14:45:18.528 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-10-16T14:45:18.528 DEBUG:tasks.check_counter:Getting stats from j 2023-10-16T14:45:18.529 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.j perf dump 2023-10-16T14:45:19.265 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.a/config 2023-10-16T14:45:20.215 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:20 smithi178 ceph-mon[98495]: pgmap v687: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-10-16T14:45:20.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:20 smithi186 ceph-mon[134892]: pgmap v687: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-10-16T14:45:20.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:20 smithi187 ceph-mon[135297]: pgmap v687: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2023-10-16T14:45:20.975 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2023-10-16T14:45:20.975 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:20.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 232, 2023-10-16T14:45:20.975 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 229, 2023-10-16T14:45:20.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 476208, 2023-10-16T14:45:20.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 137424, 2023-10-16T14:45:20.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:20.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:20.976 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.054927573, 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.026260408, 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.017571866, 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.007692012, 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 228, 2023-10-16T14:45:20.977 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.031008078, 2023-10-16T14:45:20.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000136000 2023-10-16T14:45:20.978 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.978 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:20.978 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:20.978 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:20.979 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:20.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.979 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 476208, 2023-10-16T14:45:20.979 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 137424 2023-10-16T14:45:20.979 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.980 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:20.980 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 5, 2023-10-16T14:45:20.980 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 176, 2023-10-16T14:45:20.980 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 3104, 2023-10-16T14:45:20.980 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 107520, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.022042921, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.017905454, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.002135440, 2023-10-16T14:45:20.981 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:20.982 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:20.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 175, 2023-10-16T14:45:20.982 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.021422523, 2023-10-16T14:45:20.982 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000122414 2023-10-16T14:45:20.982 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.983 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:20.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:20.983 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:20.983 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:20.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.984 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-16T14:45:20.984 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 107520 2023-10-16T14:45:20.984 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.984 INFO:teuthology.orchestra.run.smithi178.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_messages": 0, 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages": 2, 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_bytes": 0, 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_bytes": 0, 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:20.985 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_total_time": 0.000867570, 2023-10-16T14:45:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_send_time": 0.000000000, 2023-10-16T14:45:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_recv_time": 0.000733491, 2023-10-16T14:45:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:20.986 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:20.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:20.987 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:20.987 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:20.987 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.987 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:20.988 INFO:teuthology.orchestra.run.smithi178.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:20.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.989 INFO:teuthology.orchestra.run.smithi178.stdout: "cct": { 2023-10-16T14:45:20.989 INFO:teuthology.orchestra.run.smithi178.stdout: "total_workers": 0, 2023-10-16T14:45:20.989 INFO:teuthology.orchestra.run.smithi178.stdout: "unhealthy_workers": 0 2023-10-16T14:45:20.989 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "mempool": { 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:20.990 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:20.991 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:20.993 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:20.994 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:20.994 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:20.994 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:20.994 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:20.994 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:20.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:20.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:20.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:20.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:20.995 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:20.996 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:20.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:20.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:20.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:20.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:20.997 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:20.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:20.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:20.998 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_items": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:20.999 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_bytes": 149998, 2023-10-16T14:45:21.000 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_anon_items": 30, 2023-10-16T14:45:21.000 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:21.000 INFO:teuthology.orchestra.run.smithi178.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:21.000 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_bytes": 0, 2023-10-16T14:45:21.000 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_items": 0, 2023-10-16T14:45:21.001 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:21.001 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:21.001 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:21.001 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:21.001 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_items": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "pgmap_items": 0, 2023-10-16T14:45:21.002 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:21.003 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_co_items": 2, 2023-10-16T14:45:21.003 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:21.003 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_1_items": 0, 2023-10-16T14:45:21.003 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:21.003 INFO:teuthology.orchestra.run.smithi178.stdout: "unittest_2_items": 0 2023-10-16T14:45:21.004 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2023-10-16T14:45:21.004 INFO:teuthology.orchestra.run.smithi178.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:21.004 INFO:teuthology.orchestra.run.smithi178.stdout: "val": 0, 2023-10-16T14:45:21.004 INFO:teuthology.orchestra.run.smithi178.stdout: "max": 104857600, 2023-10-16T14:45:21.004 INFO:teuthology.orchestra.run.smithi178.stdout: "get_started": 0, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "get": 233, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "get_sum": 443402, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "get_or_fail_success": 233, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "take": 0, 2023-10-16T14:45:21.005 INFO:teuthology.orchestra.run.smithi178.stdout: "take_sum": 0, 2023-10-16T14:45:21.006 INFO:teuthology.orchestra.run.smithi178.stdout: "put": 233, 2023-10-16T14:45:21.006 INFO:teuthology.orchestra.run.smithi178.stdout: "put_sum": 443402, 2023-10-16T14:45:21.006 INFO:teuthology.orchestra.run.smithi178.stdout: "wait": { 2023-10-16T14:45:21.006 INFO:teuthology.orchestra.run.smithi178.stdout: "avgcount": 0, 2023-10-16T14:45:21.006 INFO:teuthology.orchestra.run.smithi178.stdout: "sum": 0.000000000, 2023-10-16T14:45:21.007 INFO:teuthology.orchestra.run.smithi178.stdout: "avgtime": 0.000000000 2023-10-16T14:45:21.007 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:21.007 INFO:teuthology.orchestra.run.smithi178.stdout: } 2023-10-16T14:45:21.007 INFO:teuthology.orchestra.run.smithi178.stdout:} 2023-10-16T14:45:21.459 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-10-16T14:45:21.459 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-10-16T14:45:21.459 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-10-16T14:45:21.459 DEBUG:tasks.check_counter:Getting stats from b 2023-10-16T14:45:21.460 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.b perf dump 2023-10-16T14:45:21.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:21 smithi178 ceph-mon[98495]: pgmap v688: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2023-10-16T14:45:21.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:21 smithi186 ceph-mon[134892]: pgmap v688: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2023-10-16T14:45:21.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:21 smithi187 ceph-mon[135297]: pgmap v688: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1023 B/s wr, 8 op/s 2023-10-16T14:45:22.100 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:45:23.627 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T14:45:23.627 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:23.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 78085, 2023-10-16T14:45:23.627 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 97184, 2023-10-16T14:45:23.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 51636758, 2023-10-16T14:45:23.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 354530291, 2023-10-16T14:45:23.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 63, 2023-10-16T14:45:23.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2023-10-16T14:45:23.628 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 5.780365660, 2023-10-16T14:45:23.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 2.069099872, 2023-10-16T14:45:23.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 1.785049705, 2023-10-16T14:45:23.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.907991116, 2023-10-16T14:45:23.629 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:23.629 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 97185, 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 24.960726652, 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000256837 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 62023, 2023-10-16T14:45:23.630 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.010839981, 2023-10-16T14:45:23.631 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000174 2023-10-16T14:45:23.631 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.631 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-16T14:45:23.631 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-10-16T14:45:23.631 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.632 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:23.632 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 84325, 2023-10-16T14:45:23.632 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 105012, 2023-10-16T14:45:23.632 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 38792041, 2023-10-16T14:45:23.632 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 235483489, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 7, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 6.181763742, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 2.251942666, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 2.175406521, 2023-10-16T14:45:23.633 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.801581903, 2023-10-16T14:45:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 105013, 2023-10-16T14:45:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 28.969341268, 2023-10-16T14:45:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000275864 2023-10-16T14:45:23.634 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.635 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:23.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 83182, 2023-10-16T14:45:23.635 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.016880798, 2023-10-16T14:45:23.635 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000202 2023-10-16T14:45:23.635 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 583888, 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 155520 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 350154, 2023-10-16T14:45:23.636 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 620117, 2023-10-16T14:45:23.637 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 112791935, 2023-10-16T14:45:23.637 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 359284469, 2023-10-16T14:45:23.637 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 30, 2023-10-16T14:45:23.637 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 8, 2023-10-16T14:45:23.637 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 18.154266662, 2023-10-16T14:45:23.638 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 8.424146355, 2023-10-16T14:45:23.638 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 23.165263841, 2023-10-16T14:45:23.638 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.950317506, 2023-10-16T14:45:23.638 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:23.638 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 620111, 2023-10-16T14:45:23.639 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 115.388817574, 2023-10-16T14:45:23.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000186077 2023-10-16T14:45:23.639 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.639 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:23.639 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 396183, 2023-10-16T14:45:23.640 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.109766557, 2023-10-16T14:45:23.640 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000277 2023-10-16T14:45:23.640 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.640 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 57968, 2023-10-16T14:45:23.640 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 412816 2023-10-16T14:45:23.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.641 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-16T14:45:23.641 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-16T14:45:23.641 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-16T14:45:23.641 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-16T14:45:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 36483, 2023-10-16T14:45:23.642 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 187.765024877, 2023-10-16T14:45:23.643 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.005146644 2023-10-16T14:45:23.643 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.643 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.643 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:23.643 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:23.644 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:23.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 86553, 2023-10-16T14:45:23.644 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 20.545141054, 2023-10-16T14:45:23.644 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000237370 2023-10-16T14:45:23.644 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.645 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.645 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-16T14:45:23.645 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 182913, 2023-10-16T14:45:23.645 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 169392, 2023-10-16T14:45:23.645 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-16T14:45:23.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 169392, 2023-10-16T14:45:23.646 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1982.560902462, 2023-10-16T14:45:23.646 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.011703981 2023-10-16T14:45:23.646 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.646 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-16T14:45:23.647 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 13496, 2023-10-16T14:45:23.647 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 33, 2023-10-16T14:45:23.647 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 2, 2023-10-16T14:45:23.647 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 1515, 2023-10-16T14:45:23.647 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 149, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 9, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 26, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 6, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 6, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 14, 2023-10-16T14:45:23.648 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 17, 2023-10-16T14:45:23.649 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 67834, 2023-10-16T14:45:23.649 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 1, 2023-10-16T14:45:23.649 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 1, 2023-10-16T14:45:23.649 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 4, 2023-10-16T14:45:23.649 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 267124, 2023-10-16T14:45:23.650 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 236529, 2023-10-16T14:45:23.650 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 13416, 2023-10-16T14:45:23.650 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 558, 2023-10-16T14:45:23.650 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 1, 2023-10-16T14:45:23.650 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:23.651 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 406, 2023-10-16T14:45:23.651 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 13420, 2023-10-16T14:45:23.651 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-16T14:45:23.651 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 318, 2023-10-16T14:45:23.651 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 49552, 2023-10-16T14:45:23.652 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 222, 2023-10-16T14:45:23.652 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 61860, 2023-10-16T14:45:23.652 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:23.652 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 44006, 2023-10-16T14:45:23.653 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:23.653 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 4, 2023-10-16T14:45:23.653 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 246, 2023-10-16T14:45:23.653 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-16T14:45:23.653 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 44006, 2023-10-16T14:45:23.654 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:23.654 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:23.654 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:23.654 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 456, 2023-10-16T14:45:23.654 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 8, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 448, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 43550, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 219, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 33876, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 98902, 2023-10-16T14:45:23.655 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 1, 2023-10-16T14:45:23.656 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:23.656 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:23.656 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 78753, 2023-10-16T14:45:23.656 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 37422, 2023-10-16T14:45:23.656 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 2888, 2023-10-16T14:45:23.657 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 99334 2023-10-16T14:45:23.657 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.657 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-16T14:45:23.657 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2023-10-16T14:45:23.657 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 591, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 1205, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 2, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:23.658 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:23.659 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 50951, 2023-10-16T14:45:23.659 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 50949, 2023-10-16T14:45:23.659 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:23.659 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-16T14:45:23.659 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-16T14:45:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-16T14:45:23.660 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 8, 2023-10-16T14:45:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 321, 2023-10-16T14:45:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-16T14:45:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 159, 2023-10-16T14:45:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:23.661 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 4, 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 250274, 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 153195, 2023-10-16T14:45:23.662 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 147481, 2023-10-16T14:45:23.663 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 102793, 2023-10-16T14:45:23.663 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-16T14:45:23.663 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 5714, 2023-10-16T14:45:23.663 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 322, 2023-10-16T14:45:23.664 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 194, 2023-10-16T14:45:23.664 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 187, 2023-10-16T14:45:23.664 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 135, 2023-10-16T14:45:23.664 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 9, 2023-10-16T14:45:23.665 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-16T14:45:23.665 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 7, 2023-10-16T14:45:23.665 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 358107610, 2023-10-16T14:45:23.665 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 565192276, 2023-10-16T14:45:23.665 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2023-10-16T14:45:23.666 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-16T14:45:23.666 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4388, 2023-10-16T14:45:23.666 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 660.653210663, 2023-10-16T14:45:23.666 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.150559072 2023-10-16T14:45:23.667 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.667 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2023-10-16T14:45:23.667 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.667 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-16T14:45:23.668 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 28, 2023-10-16T14:45:23.668 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 79860, 2023-10-16T14:45:23.668 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 79832, 2023-10-16T14:45:23.668 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 27, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 3213, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 3186, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 26, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 182688, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 182662, 2023-10-16T14:45:23.669 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 1, 2023-10-16T14:45:23.670 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 82143, 2023-10-16T14:45:23.670 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 82142, 2023-10-16T14:45:23.670 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 125700, 2023-10-16T14:45:23.670 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207152 2023-10-16T14:45:23.670 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.671 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-16T14:45:23.671 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 205893, 2023-10-16T14:45:23.671 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 2652, 2023-10-16T14:45:23.671 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 182913, 2023-10-16T14:45:23.672 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 373, 2023-10-16T14:45:23.672 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 6143, 2023-10-16T14:45:23.672 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-16T14:45:23.672 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 41740, 2023-10-16T14:45:23.673 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 245.300288244, 2023-10-16T14:45:23.673 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.005876863 2023-10-16T14:45:23.673 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.673 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-16T14:45:23.674 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2351, 2023-10-16T14:45:23.674 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 361.226718289, 2023-10-16T14:45:23.674 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.153648114 2023-10-16T14:45:23.674 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.675 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:23.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.675 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.675 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-16T14:45:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.676 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.677 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.677 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-16T14:45:23.677 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 32172, 2023-10-16T14:45:23.678 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 711.220228610, 2023-10-16T14:45:23.678 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.022106808 2023-10-16T14:45:23.678 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.679 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:23.679 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.679 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.679 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.679 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.680 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-16T14:45:23.680 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.680 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.680 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.681 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.681 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-16T14:45:23.681 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.681 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.681 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.682 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.682 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:23.682 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.682 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.682 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.683 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.684 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-16T14:45:23.684 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.684 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.684 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.684 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.685 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-16T14:45:23.685 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 233, 2023-10-16T14:45:23.685 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 15.443287949, 2023-10-16T14:45:23.685 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.066280205 2023-10-16T14:45:23.685 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.686 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-16T14:45:23.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.686 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.686 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.686 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.687 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-16T14:45:23.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 8396, 2023-10-16T14:45:23.688 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 73.604764528, 2023-10-16T14:45:23.688 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.008766646 2023-10-16T14:45:23.688 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.688 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1877, 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2.952668200, 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001573078 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-16T14:45:23.689 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 41721, 2023-10-16T14:45:23.690 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 472.956850931, 2023-10-16T14:45:23.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.011336182 2023-10-16T14:45:23.690 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.690 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:23.690 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.691 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.692 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.692 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-16T14:45:23.692 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 276, 2023-10-16T14:45:23.692 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 35.148048241, 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.127348000 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.693 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.694 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.694 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:23.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.694 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.694 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-16T14:45:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2, 2023-10-16T14:45:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.004616418, 2023-10-16T14:45:23.695 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002308209 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.696 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.697 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.698 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-16T14:45:23.699 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 7, 2023-10-16T14:45:23.699 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.019598898, 2023-10-16T14:45:23.699 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002799842 2023-10-16T14:45:23.699 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.699 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-16T14:45:23.700 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 40617, 2023-10-16T14:45:23.701 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 64.683832154, 2023-10-16T14:45:23.701 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.001592531 2023-10-16T14:45:23.701 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.701 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:23.702 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:23.702 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:23.702 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.702 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.703 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.703 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.703 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.703 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-16T14:45:23.703 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-16T14:45:23.704 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 3, 2023-10-16T14:45:23.704 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 2, 2023-10-16T14:45:23.704 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-16T14:45:23.704 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-16T14:45:23.705 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 22408, 2023-10-16T14:45:23.705 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 22408, 2023-10-16T14:45:23.705 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 2529, 2023-10-16T14:45:23.705 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:23.706 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.706 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-16T14:45:23.706 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 360, 2023-10-16T14:45:23.706 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 360, 2023-10-16T14:45:23.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:23.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:23.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:23.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:23.707 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:23.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:23.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:23.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:23.708 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:23.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:23.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:23.709 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:23.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:23.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:23.710 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:23.711 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:23.711 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:23.711 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:23.711 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:23.712 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:23.712 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:23.712 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:23.712 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:23.713 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:23.713 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:23.713 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:23.713 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:23.714 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:23.714 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:23.714 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-16T14:45:23.714 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:23.715 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:23.715 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:23.715 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:23.715 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 282291, 2023-10-16T14:45:23.716 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 113, 2023-10-16T14:45:23.716 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 88, 2023-10-16T14:45:23.716 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 1, 2023-10-16T14:45:23.716 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-16T14:45:23.716 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-16T14:45:23.717 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:23.717 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:23.717 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:23.717 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:23.717 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 172, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-16T14:45:23.718 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 119249, 2023-10-16T14:45:23.719 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 1004, 2023-10-16T14:45:23.719 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:23.719 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-16T14:45:23.719 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:23.719 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 162985, 2023-10-16T14:45:23.720 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 600363385, 2023-10-16T14:45:23.721 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 37, 2023-10-16T14:45:23.721 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 162985, 2023-10-16T14:45:23.721 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-16T14:45:23.721 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 162985, 2023-10-16T14:45:23.721 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 6713.306989054, 2023-10-16T14:45:23.722 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.041189722 2023-10-16T14:45:23.722 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.722 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-16T14:45:23.722 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-16T14:45:23.722 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 162985, 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 174535 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 162985, 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 45061, 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 117924, 2023-10-16T14:45:23.723 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-16T14:45:23.724 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-16T14:45:23.724 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 145, 2023-10-16T14:45:23.724 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 6204, 2023-10-16T14:45:23.724 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1012, 2023-10-16T14:45:23.724 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 5397, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 430, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 149, 2023-10-16T14:45:23.725 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 104102, 2023-10-16T14:45:23.726 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-16T14:45:23.726 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:23.726 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:23.726 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 44024, 2023-10-16T14:45:23.726 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 9343, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-16T14:45:23.727 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-16T14:45:23.728 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:23.728 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-16T14:45:23.728 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:23.728 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 1, 2023-10-16T14:45:23.728 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-16T14:45:23.729 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-16T14:45:23.730 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-16T14:45:23.730 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-16T14:45:23.730 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-16T14:45:23.730 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-16T14:45:23.730 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 91, 2023-10-16T14:45:23.731 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-16T14:45:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 9, 2023-10-16T14:45:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 12, 2023-10-16T14:45:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2023-10-16T14:45:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-16T14:45:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 3246, 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 70, 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 412 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-16T14:45:23.733 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2023-10-16T14:45:23.734 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:23.734 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-16T14:45:23.734 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-16T14:45:23.734 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.734 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-16T14:45:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 13 2023-10-16T14:45:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-16T14:45:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:23.735 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 24, 2023-10-16T14:45:23.736 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-16T14:45:23.736 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 9, 2023-10-16T14:45:23.736 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 50949, 2023-10-16T14:45:23.736 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:23.737 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.737 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:23.737 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:23.737 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:23.737 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:23.738 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 512560, 2023-10-16T14:45:23.738 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 161719026, 2023-10-16T14:45:23.738 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:23.738 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 512560, 2023-10-16T14:45:23.739 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:23.739 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:23.739 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 512560, 2023-10-16T14:45:23.739 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 161719026, 2023-10-16T14:45:23.740 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:23.740 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.740 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.740 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.740 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.741 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.741 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:23.741 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:23.741 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:23.742 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:23.742 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:23.742 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:23.742 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:23.743 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:23.743 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 162985, 2023-10-16T14:45:23.743 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 605915810, 2023-10-16T14:45:23.743 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 162985, 2023-10-16T14:45:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 605915810, 2023-10-16T14:45:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.744 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.745 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.745 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.745 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:23.745 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:23.745 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-16T14:45:23.746 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:23.746 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:23.746 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:23.746 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:23.746 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:23.747 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 162985, 2023-10-16T14:45:23.747 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 162985, 2023-10-16T14:45:23.747 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 162985, 2023-10-16T14:45:23.748 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 162985, 2023-10-16T14:45:23.748 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:23.748 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.748 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.749 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.749 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.749 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.750 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:23.750 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:23.750 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:23.750 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:23.751 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 50949, 2023-10-16T14:45:23.751 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 5552961, 2023-10-16T14:45:23.751 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:23.751 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 50949, 2023-10-16T14:45:23.752 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:23.752 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:23.752 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 1009, 2023-10-16T14:45:23.752 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 5552961, 2023-10-16T14:45:23.753 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:23.753 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.753 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.753 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.754 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.754 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:23.754 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55bf953926e0": { 2023-10-16T14:45:23.754 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 250274, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 560997972, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:23.755 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 250274, 2023-10-16T14:45:23.756 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:23.756 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:23.756 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 4388, 2023-10-16T14:45:23.756 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 560997972, 2023-10-16T14:45:23.756 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:23.757 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T14:45:24.048 INFO:tasks.check_counter:Daemon mds.b mds.exported=318 2023-10-16T14:45:24.048 INFO:tasks.check_counter:Daemon mds.b mds.imported=222 2023-10-16T14:45:24.048 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=149 2023-10-16T14:45:24.049 DEBUG:tasks.check_counter:Getting stats from e 2023-10-16T14:45:24.049 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.e perf dump 2023-10-16T14:45:24.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:24 smithi178 ceph-mon[98495]: pgmap v689: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:45:24.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:24 smithi186 ceph-mon[134892]: pgmap v689: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:45:24.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:24 smithi187 ceph-mon[135297]: pgmap v689: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 6 op/s 2023-10-16T14:45:24.605 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:45:25.798 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T14:45:25.799 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:25.799 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 953, 2023-10-16T14:45:25.799 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 953, 2023-10-16T14:45:25.799 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 126608854, 2023-10-16T14:45:25.799 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 295651, 2023-10-16T14:45:25.800 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2023-10-16T14:45:25.800 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:25.800 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.141006359, 2023-10-16T14:45:25.800 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.022228789, 2023-10-16T14:45:25.800 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.086070982, 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.025071998, 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 952, 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.069891922, 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000073415 2023-10-16T14:45:25.801 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.802 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:25.802 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.802 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.802 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.802 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.803 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-16T14:45:25.803 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-10-16T14:45:25.803 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.803 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:25.803 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 718, 2023-10-16T14:45:25.804 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 700, 2023-10-16T14:45:25.804 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 71978906, 2023-10-16T14:45:25.804 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 282901, 2023-10-16T14:45:25.804 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 5, 2023-10-16T14:45:25.804 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.112042047, 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.020337483, 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.069985595, 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.018914450, 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:25.805 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 699, 2023-10-16T14:45:25.806 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.062487438, 2023-10-16T14:45:25.806 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000089395 2023-10-16T14:45:25.806 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.806 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:25.806 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.807 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.807 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.807 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.807 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 553392, 2023-10-16T14:45:25.807 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 138592 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1087, 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 1260, 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 101002260, 2023-10-16T14:45:25.808 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 705666, 2023-10-16T14:45:25.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 6, 2023-10-16T14:45:25.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:25.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.165875245, 2023-10-16T14:45:25.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.031617947, 2023-10-16T14:45:25.809 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.086519280, 2023-10-16T14:45:25.810 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.036977905, 2023-10-16T14:45:25.810 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:25.810 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 1258, 2023-10-16T14:45:25.810 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.097048691, 2023-10-16T14:45:25.810 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000077145 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.811 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.812 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:25.812 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 367920 2023-10-16T14:45:25.812 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.812 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-16T14:45:25.812 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-16T14:45:25.813 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-16T14:45:25.813 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.813 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-16T14:45:25.813 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:25.813 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 4189, 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.189069414, 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000045134 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.814 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:25.815 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:25.815 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:25.815 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.815 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.815 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.816 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.816 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.816 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-16T14:45:25.816 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 0, 2023-10-16T14:45:25.816 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 0, 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.817 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-16T14:45:25.818 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 0, 2023-10-16T14:45:25.818 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 0, 2023-10-16T14:45:25.818 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-10-16T14:45:25.818 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 0, 2023-10-16T14:45:25.818 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 0, 2023-10-16T14:45:25.819 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 0, 2023-10-16T14:45:25.819 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 19404, 2023-10-16T14:45:25.819 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 0, 2023-10-16T14:45:25.819 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 0, 2023-10-16T14:45:25.819 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 19404, 2023-10-16T14:45:25.820 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 19404, 2023-10-16T14:45:25.820 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 4454, 2023-10-16T14:45:25.820 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-10-16T14:45:25.820 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-10-16T14:45:25.820 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 1, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 0, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 0, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 0, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 0, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-10-16T14:45:25.821 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:25.822 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 0, 2023-10-16T14:45:25.822 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-10-16T14:45:25.822 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-16T14:45:25.822 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 0, 2023-10-16T14:45:25.822 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 0, 2023-10-16T14:45:25.823 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 0, 2023-10-16T14:45:25.823 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 0, 2023-10-16T14:45:25.823 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:25.823 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 0, 2023-10-16T14:45:25.823 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:25.824 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:25.825 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:25.825 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 0, 2023-10-16T14:45:25.825 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-10-16T14:45:25.825 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 0, 2023-10-16T14:45:25.825 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 0, 2023-10-16T14:45:25.826 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 0, 2023-10-16T14:45:25.826 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 0, 2023-10-16T14:45:25.826 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 0, 2023-10-16T14:45:25.826 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-10-16T14:45:25.826 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 0, 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 0, 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 0, 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 0 2023-10-16T14:45:25.827 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.828 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-16T14:45:25.828 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 0, 2023-10-16T14:45:25.828 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2023-10-16T14:45:25.828 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2023-10-16T14:45:25.828 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 0, 2023-10-16T14:45:25.829 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 0, 2023-10-16T14:45:25.829 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-10-16T14:45:25.829 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:25.829 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:25.829 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 0, 2023-10-16T14:45:25.830 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 0, 2023-10-16T14:45:25.830 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:25.830 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-16T14:45:25.830 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:25.830 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 0, 2023-10-16T14:45:25.831 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-16T14:45:25.832 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 0, 2023-10-16T14:45:25.832 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:25.832 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-16T14:45:25.832 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.832 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-16T14:45:25.833 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 0, 2023-10-16T14:45:25.833 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 0, 2023-10-16T14:45:25.833 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 0, 2023-10-16T14:45:25.833 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 0, 2023-10-16T14:45:25.833 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 139241, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 0, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 0, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 0, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 0, 2023-10-16T14:45:25.834 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 141, 2023-10-16T14:45:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 13, 2023-10-16T14:45:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-16T14:45:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 0, 2023-10-16T14:45:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 55836922, 2023-10-16T14:45:25.835 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 302510166, 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 302501550, 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.836 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.837 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 139241 2023-10-16T14:45:25.837 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.837 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-16T14:45:25.837 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 13, 2023-10-16T14:45:25.837 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 32879, 2023-10-16T14:45:25.838 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 32866, 2023-10-16T14:45:25.838 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 11, 2023-10-16T14:45:25.838 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1246, 2023-10-16T14:45:25.838 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1235, 2023-10-16T14:45:25.838 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 19404, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 72191, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 52787, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 0, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 0, 2023-10-16T14:45:25.839 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 104160, 2023-10-16T14:45:25.840 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 207152 2023-10-16T14:45:25.840 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.840 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-16T14:45:25.840 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 0, 2023-10-16T14:45:25.840 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 0, 2023-10-16T14:45:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 0, 2023-10-16T14:45:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 0, 2023-10-16T14:45:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 0, 2023-10-16T14:45:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-16T14:45:25.841 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.842 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.843 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.843 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:25.843 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.843 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.844 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-16T14:45:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.848 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-16T14:45:25.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.848 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.848 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.848 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.849 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:25.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.849 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.849 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.849 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.850 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-16T14:45:25.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.851 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.851 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.851 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.851 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-16T14:45:25.852 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.852 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.852 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.852 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.852 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-16T14:45:25.853 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.854 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.854 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.854 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.854 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-16T14:45:25.854 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.855 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.855 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.855 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.855 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-16T14:45:25.855 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.856 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.857 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.857 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.857 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:25.857 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.857 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.858 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.858 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.858 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:25.858 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.858 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.859 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.859 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.859 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-16T14:45:25.859 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.859 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.860 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.861 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.861 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:25.861 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.861 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.861 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.862 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.862 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-16T14:45:25.862 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.862 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.862 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.863 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.863 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:25.863 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.863 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.863 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.864 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.865 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-16T14:45:25.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.865 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.865 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.865 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.866 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-16T14:45:25.866 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.866 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.866 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.866 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.867 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-16T14:45:25.868 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.868 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.868 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.868 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.868 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.869 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.870 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.870 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-16T14:45:25.870 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-16T14:45:25.870 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2023-10-16T14:45:25.870 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-10-16T14:45:25.871 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-16T14:45:25.871 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-16T14:45:25.871 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 0, 2023-10-16T14:45:25.871 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 0, 2023-10-16T14:45:25.871 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 2532, 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:25.872 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:25.873 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:25.873 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:25.873 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:25.873 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:25.873 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:25.874 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:25.874 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:25.874 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:25.874 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:25.874 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:25.875 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:25.876 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:25.876 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:25.876 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:25.876 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:25.876 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:25.877 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:25.878 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:25.878 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-16T14:45:25.878 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:25.878 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:25.878 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 231391, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 75, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:25.879 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-16T14:45:25.880 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-16T14:45:25.880 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:25.880 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:25.880 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:25.880 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 172, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:25.881 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-16T14:45:25.882 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 13547944, 2023-10-16T14:45:25.882 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 77744, 2023-10-16T14:45:25.882 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:25.882 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-16T14:45:25.882 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:25.883 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-16T14:45:25.883 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.883 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-16T14:45:25.883 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-16T14:45:25.883 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 2499, 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 21, 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 3, 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 2499, 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-16T14:45:25.884 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2499, 2023-10-16T14:45:25.885 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 73.049720967, 2023-10-16T14:45:25.885 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.029231581 2023-10-16T14:45:25.885 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.885 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-16T14:45:25.885 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-16T14:45:25.886 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2499, 2023-10-16T14:45:25.886 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 2500 2023-10-16T14:45:25.886 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.886 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 2499, 2023-10-16T14:45:25.886 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 2499, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 0, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 1400, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 0, 2023-10-16T14:45:25.887 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 1097, 2023-10-16T14:45:25.888 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 0, 2023-10-16T14:45:25.888 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 0, 2023-10-16T14:45:25.888 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-16T14:45:25.888 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-16T14:45:25.888 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:25.889 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 1, 2023-10-16T14:45:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 0, 2023-10-16T14:45:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:25.890 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-16T14:45:25.894 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-16T14:45:25.894 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-16T14:45:25.894 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:25.894 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-16T14:45:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-10-16T14:45:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-16T14:45:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-16T14:45:25.895 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-16T14:45:25.896 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-16T14:45:25.896 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-16T14:45:25.896 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-16T14:45:25.896 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-16T14:45:25.896 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-16T14:45:25.897 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-16T14:45:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-16T14:45:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-16T14:45:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 91, 2023-10-16T14:45:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-16T14:45:25.898 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 10, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 10, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 0, 2023-10-16T14:45:25.899 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 2, 2023-10-16T14:45:25.900 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 0 2023-10-16T14:45:25.900 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.900 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-16T14:45:25.900 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 0, 2023-10-16T14:45:25.900 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:25.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-16T14:45:25.901 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-16T14:45:25.901 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.901 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-16T14:45:25.901 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 0 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 0, 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-16T14:45:25.902 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 0, 2023-10-16T14:45:25.903 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 0, 2023-10-16T14:45:25.903 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:25.903 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.903 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:25.903 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 2754, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 299326214, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:25.904 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 2754, 2023-10-16T14:45:25.905 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:25.905 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:25.905 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2754, 2023-10-16T14:45:25.905 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 299326214, 2023-10-16T14:45:25.905 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:25.906 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.906 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.906 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.906 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.906 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:25.907 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:25.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:25.908 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:25.908 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2499, 2023-10-16T14:45:25.908 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 298335326, 2023-10-16T14:45:25.908 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2499, 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 298335326, 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.909 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.910 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.910 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:25.910 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:25.910 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-16T14:45:25.910 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:25.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:25.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:25.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:25.911 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:25.911 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 2499, 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 2499, 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2499, 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2499, 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.912 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.913 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.913 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.913 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.913 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:25.913 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:25.914 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:25.915 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:25.915 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:25.915 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-10-16T14:45:25.915 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-10-16T14:45:25.915 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:25.916 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x560c8272a960": { 2023-10-16T14:45:25.917 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:25.917 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:25.917 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:25.917 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:25.917 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:25.918 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:25.918 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:25.918 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:25.918 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:25.918 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 0, 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 0, 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:25.919 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.920 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:25.920 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T14:45:26.247 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:26 smithi186 ceph-mon[134892]: pgmap v690: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-16T14:45:26.249 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-10-16T14:45:26.249 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-10-16T14:45:26.249 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-10-16T14:45:26.250 DEBUG:tasks.check_counter:Getting stats from h 2023-10-16T14:45:26.250 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.h perf dump 2023-10-16T14:45:26.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:26 smithi178 ceph-mon[98495]: pgmap v690: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-16T14:45:26.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:26 smithi187 ceph-mon[135297]: pgmap v690: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 767 B/s wr, 6 op/s 2023-10-16T14:45:26.806 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:45:28.120 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T14:45:28.120 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:28.121 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 268188, 2023-10-16T14:45:28.121 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 380530, 2023-10-16T14:45:28.121 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 88189743, 2023-10-16T14:45:28.121 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 262642541, 2023-10-16T14:45:28.121 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 29, 2023-10-16T14:45:28.122 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 8, 2023-10-16T14:45:28.122 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 12.294521898, 2023-10-16T14:45:28.122 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 5.477880125, 2023-10-16T14:45:28.122 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 13.959069826, 2023-10-16T14:45:28.122 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.538329958, 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 380526, 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 94.553134690, 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000248480 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.123 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:28.124 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 242568, 2023-10-16T14:45:28.124 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.067645328, 2023-10-16T14:45:28.124 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000278 2023-10-16T14:45:28.124 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.124 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:28.125 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:28.125 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.125 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:28.125 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 53386, 2023-10-16T14:45:28.125 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 53337, 2023-10-16T14:45:28.126 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 18020511, 2023-10-16T14:45:28.126 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 131410728, 2023-10-16T14:45:28.126 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 8, 2023-10-16T14:45:28.126 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:28.126 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 3.650839226, 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 1.083916249, 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 1.189811674, 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.425649258, 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 53337, 2023-10-16T14:45:28.127 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 23.223859361, 2023-10-16T14:45:28.128 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000435417 2023-10-16T14:45:28.128 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.128 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:28.128 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 39464, 2023-10-16T14:45:28.128 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.006665050, 2023-10-16T14:45:28.129 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000168 2023-10-16T14:45:28.129 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.129 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:28.129 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 370224 2023-10-16T14:45:28.129 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.130 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:28.130 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 84526, 2023-10-16T14:45:28.130 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 63799, 2023-10-16T14:45:28.130 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 39927900, 2023-10-16T14:45:28.130 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 122277104, 2023-10-16T14:45:28.131 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 9, 2023-10-16T14:45:28.131 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 6, 2023-10-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 5.068131987, 2023-10-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 1.780721337, 2023-10-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 2.413106689, 2023-10-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 1.421302427, 2023-10-16T14:45:28.132 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:28.133 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 63799, 2023-10-16T14:45:28.133 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 30.146803445, 2023-10-16T14:45:28.133 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000472527 2023-10-16T14:45:28.133 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.134 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:28.134 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 85648, 2023-10-16T14:45:28.134 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.012341253, 2023-10-16T14:45:28.135 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000144 2023-10-16T14:45:28.135 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.135 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 567360, 2023-10-16T14:45:28.135 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 153792 2023-10-16T14:45:28.135 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 1, 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-MDSRank": { 2023-10-16T14:45:28.136 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:28.137 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:28.137 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 31771, 2023-10-16T14:45:28.137 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 105.065165200, 2023-10-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.003306951 2023-10-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi186.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:28.138 INFO:teuthology.orchestra.run.smithi186.stdout: "queue_len": 0, 2023-10-16T14:45:28.139 INFO:teuthology.orchestra.run.smithi186.stdout: "complete_latency": { 2023-10-16T14:45:28.139 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 36803, 2023-10-16T14:45:28.139 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 9.615809730, 2023-10-16T14:45:28.139 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000261277 2023-10-16T14:45:28.139 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.140 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.140 INFO:teuthology.orchestra.run.smithi186.stdout: "mds": { 2023-10-16T14:45:28.140 INFO:teuthology.orchestra.run.smithi186.stdout: "request": 109447, 2023-10-16T14:45:28.140 INFO:teuthology.orchestra.run.smithi186.stdout: "reply": 107920, 2023-10-16T14:45:28.141 INFO:teuthology.orchestra.run.smithi186.stdout: "reply_latency": { 2023-10-16T14:45:28.141 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 107920, 2023-10-16T14:45:28.141 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1975.904908260, 2023-10-16T14:45:28.141 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.018308978 2023-10-16T14:45:28.141 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.142 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_reply": 0, 2023-10-16T14:45:28.142 INFO:teuthology.orchestra.run.smithi186.stdout: "forward": 1525, 2023-10-16T14:45:28.142 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_complete": 16, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_fetch_keys": 0, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_commit": 1038, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_split": 90, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_merge": 2, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes": 17, 2023-10-16T14:45:28.143 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_top": 1, 2023-10-16T14:45:28.144 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_bottom": 6, 2023-10-16T14:45:28.144 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pin_tail": 10, 2023-10-16T14:45:28.144 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_pinned": 13, 2023-10-16T14:45:28.144 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_expired": 33531, 2023-10-16T14:45:28.144 INFO:teuthology.orchestra.run.smithi186.stdout: "inodes_with_caps": 0, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "caps": 0, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "subtrees": 3, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse": 181630, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_hit": 153714, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_forward": 1402, 2023-10-16T14:45:28.145 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_discover": 1237, 2023-10-16T14:45:28.146 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_dir_fetch": 0, 2023-10-16T14:45:28.146 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:28.146 INFO:teuthology.orchestra.run.smithi186.stdout: "traverse_lock": 778, 2023-10-16T14:45:28.146 INFO:teuthology.orchestra.run.smithi186.stdout: "load_cent": 0, 2023-10-16T14:45:28.146 INFO:teuthology.orchestra.run.smithi186.stdout: "q": 0, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "exported": 148, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "exported_inodes": 21486, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "imported": 226, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "imported_inodes": 17905, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:28.147 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_backtrace_fetch": 41250, 2023-10-16T14:45:28.148 INFO:teuthology.orchestra.run.smithi186.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:28.148 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rfiles": 1623, 2023-10-16T14:45:28.148 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rbytes": 35380239, 2023-10-16T14:45:28.148 INFO:teuthology.orchestra.run.smithi186.stdout: "root_rsnaps": 0, 2023-10-16T14:45:28.149 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_fetch": 41250, 2023-10-16T14:45:28.149 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:28.149 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:28.150 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:28.150 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_inodes": 254, 2023-10-16T14:45:28.150 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dir_base_inodes": 0, 2023-10-16T14:45:28.150 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_dirfrag_rstats": 254, 2023-10-16T14:45:28.151 INFO:teuthology.orchestra.run.smithi186.stdout: "scrub_file_inodes": 40996, 2023-10-16T14:45:28.151 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_inode_file_caps": 317, 2023-10-16T14:45:28.151 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_revoke": 26102, 2023-10-16T14:45:28.151 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_grant": 47076, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_trunc": 0, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps": 65458, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_caps_dirty": 26523, 2023-10-16T14:45:28.152 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_cap_release": 867, 2023-10-16T14:45:28.153 INFO:teuthology.orchestra.run.smithi186.stdout: "process_request_cap_release": 53147 2023-10-16T14:45:28.153 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.153 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_cache": { 2023-10-16T14:45:28.153 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update": 2, 2023-10-16T14:45:28.153 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_update_receipt": 0, 2023-10-16T14:45:28.154 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_try_discover": 0, 2023-10-16T14:45:28.154 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_send_discover": 1392, 2023-10-16T14:45:28.154 INFO:teuthology.orchestra.run.smithi186.stdout: "dir_handle_discover": 711, 2023-10-16T14:45:28.154 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays": 0, 2023-10-16T14:45:28.154 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_created": 21380, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_enqueued": 21380, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "strays_migrated": 0, 2023-10-16T14:45:28.155 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:28.156 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:28.156 INFO:teuthology.orchestra.run.smithi186.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:28.156 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_started": 0, 2023-10-16T14:45:28.156 INFO:teuthology.orchestra.run.smithi186.stdout: "recovery_completed": 0, 2023-10-16T14:45:28.156 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_enqueue_scrub": 0, 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_exportdir": 151, 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_flush": 0, 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragmentdir": 92, 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: "ireq_inodestats": 0 2023-10-16T14:45:28.157 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.158 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_log": { 2023-10-16T14:45:28.158 INFO:teuthology.orchestra.run.smithi186.stdout: "evlrg": 2, 2023-10-16T14:45:28.158 INFO:teuthology.orchestra.run.smithi186.stdout: "evadd": 139241, 2023-10-16T14:45:28.158 INFO:teuthology.orchestra.run.smithi186.stdout: "evex": 35569, 2023-10-16T14:45:28.158 INFO:teuthology.orchestra.run.smithi186.stdout: "evtrm": 24581, 2023-10-16T14:45:28.159 INFO:teuthology.orchestra.run.smithi186.stdout: "ev": 114660, 2023-10-16T14:45:28.159 INFO:teuthology.orchestra.run.smithi186.stdout: "evexg": 0, 2023-10-16T14:45:28.159 INFO:teuthology.orchestra.run.smithi186.stdout: "evexd": 10988, 2023-10-16T14:45:28.159 INFO:teuthology.orchestra.run.smithi186.stdout: "segadd": 172, 2023-10-16T14:45:28.159 INFO:teuthology.orchestra.run.smithi186.stdout: "segex": 44, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "segtrm": 31, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "seg": 141, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "segmjr": 10, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "segexg": 0, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "segexd": 13, 2023-10-16T14:45:28.160 INFO:teuthology.orchestra.run.smithi186.stdout: "expos": 55836922, 2023-10-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi186.stdout: "wrpos": 302510166, 2023-10-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi186.stdout: "rdpos": 0, 2023-10-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi186.stdout: "jlat": { 2023-10-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 2872, 2023-10-16T14:45:28.161 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 638.272181160, 2023-10-16T14:45:28.162 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.222239617 2023-10-16T14:45:28.162 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.162 INFO:teuthology.orchestra.run.smithi186.stdout: "replayed": 0 2023-10-16T14:45:28.162 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.162 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_mem": { 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "ino": 22, 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "ino+": 34123, 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "ino-": 34101, 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "dir": 20, 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "dir+": 1363, 2023-10-16T14:45:28.163 INFO:teuthology.orchestra.run.smithi186.stdout: "dir-": 1343, 2023-10-16T14:45:28.164 INFO:teuthology.orchestra.run.smithi186.stdout: "dn": 17, 2023-10-16T14:45:28.164 INFO:teuthology.orchestra.run.smithi186.stdout: "dn+": 79462, 2023-10-16T14:45:28.164 INFO:teuthology.orchestra.run.smithi186.stdout: "dn-": 79445, 2023-10-16T14:45:28.164 INFO:teuthology.orchestra.run.smithi186.stdout: "cap": 0, 2023-10-16T14:45:28.164 INFO:teuthology.orchestra.run.smithi186.stdout: "cap+": 35388, 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: "cap-": 35388, 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: "rss": 149120, 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: "heap": 198960 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_server": { 2023-10-16T14:45:28.165 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_client_request": 119703, 2023-10-16T14:45:28.166 INFO:teuthology.orchestra.run.smithi186.stdout: "dispatch_server_request": 4622, 2023-10-16T14:45:28.166 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_request": 109447, 2023-10-16T14:45:28.166 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_client_session": 272, 2023-10-16T14:45:28.166 INFO:teuthology.orchestra.run.smithi186.stdout: "handle_peer_request": 7069, 2023-10-16T14:45:28.166 INFO:teuthology.orchestra.run.smithi186.stdout: "req_create_latency": { 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 22810, 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 108.444337851, 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.004754245 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getattr_latency": { 2023-10-16T14:45:28.167 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 95, 2023-10-16T14:45:28.168 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 26.427922515, 2023-10-16T14:45:28.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.278188658 2023-10-16T14:45:28.168 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.168 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:28.168 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: "req_link_latency": { 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.169 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookup_latency": { 2023-10-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 40140, 2023-10-16T14:45:28.170 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1332.687962628, 2023-10-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.033200995 2023-10-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.171 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupino_latency": { 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.172 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupname_latency": { 2023-10-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.173 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.174 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.175 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:28.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.175 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.175 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.175 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.176 INFO:teuthology.orchestra.run.smithi186.stdout: "req_lssnap_latency": { 2023-10-16T14:45:28.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.176 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.176 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.176 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mkdir_latency": { 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 108, 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 30.437223666, 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.281826145 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.177 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mknod_latency": { 2023-10-16T14:45:28.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.178 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.178 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.178 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.178 INFO:teuthology.orchestra.run.smithi186.stdout: "req_mksnap_latency": { 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: "req_open_latency": { 2023-10-16T14:45:28.179 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 6518, 2023-10-16T14:45:28.180 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 32.901626295, 2023-10-16T14:45:28.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.005047810 2023-10-16T14:45:28.180 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.180 INFO:teuthology.orchestra.run.smithi186.stdout: "req_readdir_latency": { 2023-10-16T14:45:28.180 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 623, 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 1.343567561, 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002156609 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rename_latency": { 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 22830, 2023-10-16T14:45:28.181 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 393.830268550, 2023-10-16T14:45:28.182 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.017250559 2023-10-16T14:45:28.182 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.182 INFO:teuthology.orchestra.run.smithi186.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:28.182 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.182 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.183 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.183 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.183 INFO:teuthology.orchestra.run.smithi186.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:28.183 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.183 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmdir_latency": { 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 100, 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 12.301375329, 2023-10-16T14:45:28.184 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.123013753 2023-10-16T14:45:28.185 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.185 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:28.185 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.185 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.185 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.186 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.186 INFO:teuthology.orchestra.run.smithi186.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:28.186 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.186 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.186 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setattr_latency": { 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.187 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.188 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:28.188 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.188 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.188 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.188 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.189 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setlayout_latency": { 2023-10-16T14:45:28.190 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.190 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.190 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.190 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.190 INFO:teuthology.orchestra.run.smithi186.stdout: "req_setxattr_latency": { 2023-10-16T14:45:28.191 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.191 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.191 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.191 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.191 INFO:teuthology.orchestra.run.smithi186.stdout: "req_symlink_latency": { 2023-10-16T14:45:28.192 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.192 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.192 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.192 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.192 INFO:teuthology.orchestra.run.smithi186.stdout: "req_unlink_latency": { 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 14696, 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 37.530623865, 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.002553798 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:28.193 INFO:teuthology.orchestra.run.smithi186.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:28.194 INFO:teuthology.orchestra.run.smithi186.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:28.194 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.194 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.194 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.194 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_sessions": { 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: "session_count": 1, 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: "session_add": 1, 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: "session_remove": 0, 2023-10-16T14:45:28.195 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_open": 1, 2023-10-16T14:45:28.196 INFO:teuthology.orchestra.run.smithi186.stdout: "sessions_stale": 0, 2023-10-16T14:45:28.196 INFO:teuthology.orchestra.run.smithi186.stdout: "total_load": 12636, 2023-10-16T14:45:28.196 INFO:teuthology.orchestra.run.smithi186.stdout: "average_load": 12636, 2023-10-16T14:45:28.196 INFO:teuthology.orchestra.run.smithi186.stdout: "avg_session_uptime": 2533, 2023-10-16T14:45:28.196 INFO:teuthology.orchestra.run.smithi186.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:28.197 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:28.198 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:28.198 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:28.198 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:28.198 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:28.198 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:28.199 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:28.200 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:28.200 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:28.200 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:28.200 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:28.200 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:28.201 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:28.202 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:28.202 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:28.202 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:28.202 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:28.202 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:28.203 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 305900, 2023-10-16T14:45:28.204 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 137, 2023-10-16T14:45:28.204 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 792, 2023-10-16T14:45:28.204 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 9, 2023-10-16T14:45:28.204 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-16T14:45:28.204 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:28.205 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 172, 2023-10-16T14:45:28.206 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:28.206 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:28.206 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:28.206 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-16T14:45:28.206 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 87493, 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 486, 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-16T14:45:28.207 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.208 INFO:teuthology.orchestra.run.smithi186.stdout: "objecter": { 2023-10-16T14:45:28.208 INFO:teuthology.orchestra.run.smithi186.stdout: "op_active": 0, 2023-10-16T14:45:28.208 INFO:teuthology.orchestra.run.smithi186.stdout: "op_laggy": 0, 2023-10-16T14:45:28.208 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send": 100236, 2023-10-16T14:45:28.208 INFO:teuthology.orchestra.run.smithi186.stdout: "op_send_bytes": 326065407, 2023-10-16T14:45:28.209 INFO:teuthology.orchestra.run.smithi186.stdout: "op_resend": 27, 2023-10-16T14:45:28.209 INFO:teuthology.orchestra.run.smithi186.stdout: "op_reply": 100236, 2023-10-16T14:45:28.209 INFO:teuthology.orchestra.run.smithi186.stdout: "op_latency": { 2023-10-16T14:45:28.209 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 100236, 2023-10-16T14:45:28.209 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 6879.320897803, 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.068631239 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: "op_inflight": 0, 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: "oplen_avg": { 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 100236, 2023-10-16T14:45:28.210 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 116946 2023-10-16T14:45:28.211 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.211 INFO:teuthology.orchestra.run.smithi186.stdout: "op": 100236, 2023-10-16T14:45:28.211 INFO:teuthology.orchestra.run.smithi186.stdout: "op_r": 41885, 2023-10-16T14:45:28.211 INFO:teuthology.orchestra.run.smithi186.stdout: "op_w": 58351, 2023-10-16T14:45:28.211 INFO:teuthology.orchestra.run.smithi186.stdout: "op_rmw": 0, 2023-10-16T14:45:28.212 INFO:teuthology.orchestra.run.smithi186.stdout: "op_pg": 0, 2023-10-16T14:45:28.212 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_stat": 212, 2023-10-16T14:45:28.212 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_create": 9958, 2023-10-16T14:45:28.212 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_read": 619, 2023-10-16T14:45:28.212 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_write": 3493, 2023-10-16T14:45:28.213 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writefull": 300, 2023-10-16T14:45:28.213 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_writesame": 0, 2023-10-16T14:45:28.213 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_append": 0, 2023-10-16T14:45:28.213 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_zero": 0, 2023-10-16T14:45:28.213 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_truncate": 90, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_delete": 43372, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_mapext": 0, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_getxattr": 41250, 2023-10-16T14:45:28.214 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_setxattr": 14972, 2023-10-16T14:45:28.215 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:28.215 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:28.215 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:28.215 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_call": 0, 2023-10-16T14:45:28.215 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_watch": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_notify": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "osdop_other": 0, 2023-10-16T14:45:28.216 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_active": 0, 2023-10-16T14:45:28.217 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_send": 0, 2023-10-16T14:45:28.217 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_resend": 0, 2023-10-16T14:45:28.217 INFO:teuthology.orchestra.run.smithi186.stdout: "linger_ping": 0, 2023-10-16T14:45:28.217 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_active": 0, 2023-10-16T14:45:28.217 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_send": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "poolop_resend": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_active": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_send": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "poolstat_resend": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_active": 0, 2023-10-16T14:45:28.218 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_send": 0, 2023-10-16T14:45:28.219 INFO:teuthology.orchestra.run.smithi186.stdout: "statfs_resend": 0, 2023-10-16T14:45:28.219 INFO:teuthology.orchestra.run.smithi186.stdout: "command_active": 0, 2023-10-16T14:45:28.219 INFO:teuthology.orchestra.run.smithi186.stdout: "command_send": 0, 2023-10-16T14:45:28.219 INFO:teuthology.orchestra.run.smithi186.stdout: "command_resend": 0, 2023-10-16T14:45:28.219 INFO:teuthology.orchestra.run.smithi186.stdout: "map_epoch": 91, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "map_full": 0, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "map_inc": 2, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_sessions": 12, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_open": 12, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_session_close": 0, 2023-10-16T14:45:28.220 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_laggy": 0, 2023-10-16T14:45:28.221 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_wr": 2252, 2023-10-16T14:45:28.221 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_rd": 32, 2023-10-16T14:45:28.221 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_del": 396 2023-10-16T14:45:28.221 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.221 INFO:teuthology.orchestra.run.smithi186.stdout: "oft": { 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_objs": 1, 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_updates": 0, 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: "omap_total_removes": 0 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.222 INFO:teuthology.orchestra.run.smithi186.stdout: "osd-slow-ops": { 2023-10-16T14:45:28.223 INFO:teuthology.orchestra.run.smithi186.stdout: "slow_ops_count": 69 2023-10-16T14:45:28.223 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.223 INFO:teuthology.orchestra.run.smithi186.stdout: "purge_queue": { 2023-10-16T14:45:28.223 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:28.223 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_ops_high_water": 24, 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing": 0, 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executing_high_water": 9, 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_executed": 21380, 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.224 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:28.225 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:28.225 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:28.225 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:28.225 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 406096, 2023-10-16T14:45:28.225 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 113263140, 2023-10-16T14:45:28.226 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:28.226 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 406096, 2023-10-16T14:45:28.226 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:28.226 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:28.226 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 406096, 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 113263140, 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.227 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.228 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.228 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:28.228 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:28.228 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:28.228 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 100236, 2023-10-16T14:45:28.229 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 328395403, 2023-10-16T14:45:28.230 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 100236, 2023-10-16T14:45:28.230 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 328395403, 2023-10-16T14:45:28.230 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:28.230 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.230 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:28.231 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 1024, 2023-10-16T14:45:28.232 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:28.232 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 0, 2023-10-16T14:45:28.232 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 0, 2023-10-16T14:45:28.232 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:28.232 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 100236, 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 100236, 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 100236, 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 100236, 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:28.233 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.234 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.234 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.234 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.234 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.234 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 21380, 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 2330252, 2023-10-16T14:45:28.235 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:28.236 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 21380, 2023-10-16T14:45:28.236 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:28.236 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:28.236 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 621, 2023-10-16T14:45:28.236 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 2330252, 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.237 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:28.238 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-write_buf_throttle-0x55b3fb5086e0": { 2023-10-16T14:45:28.238 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:28.238 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 3758096384, 2023-10-16T14:45:28.238 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:28.238 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 139241, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 298315862, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 139241, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:28.239 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 2872, 2023-10-16T14:45:28.240 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 298315862, 2023-10-16T14:45:28.240 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:28.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:28.240 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:28.240 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:28.241 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.241 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:28.241 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T14:45:28.277 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:28 smithi186 ceph-mon[134892]: pgmap v691: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-16T14:45:28.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:28 smithi178 ceph-mon[98495]: pgmap v691: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-16T14:45:28.424 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:28 smithi187 ceph-mon[135297]: pgmap v691: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2023-10-16T14:45:28.563 INFO:tasks.check_counter:Daemon mds.h mds.exported=148 2023-10-16T14:45:28.564 INFO:tasks.check_counter:Daemon mds.h mds.imported=226 2023-10-16T14:45:28.564 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=90 2023-10-16T14:45:28.564 DEBUG:tasks.check_counter:Getting stats from k 2023-10-16T14:45:28.565 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.k perf dump 2023-10-16T14:45:29.121 INFO:teuthology.orchestra.run.smithi186.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.b/config 2023-10-16T14:45:30.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:30 smithi178 ceph-mon[98495]: pgmap v692: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:45:30.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:30 smithi186 ceph-mon[134892]: pgmap v692: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:45:30.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:30 smithi187 ceph-mon[135297]: pgmap v692: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-10-16T14:45:30.448 INFO:teuthology.orchestra.run.smithi186.stdout:{ 2023-10-16T14:45:30.448 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:30.448 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 4, 2023-10-16T14:45:30.449 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 4, 2023-10-16T14:45:30.449 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 2944, 2023-10-16T14:45:30.449 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 448, 2023-10-16T14:45:30.449 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.001572223, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.000044042, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000709812, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:30.450 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:30.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 3, 2023-10-16T14:45:30.451 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000839910, 2023-10-16T14:45:30.451 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000279970 2023-10-16T14:45:30.451 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.451 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:30.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:30.452 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:30.452 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:30.452 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.452 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 448 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 1, 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 176, 2023-10-16T14:45:30.453 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 160, 2023-10-16T14:45:30.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 107424, 2023-10-16T14:45:30.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:30.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:30.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.008653506, 2023-10-16T14:45:30.454 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.006021621, 2023-10-16T14:45:30.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.000980918, 2023-10-16T14:45:30.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:30.455 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:30.455 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 174, 2023-10-16T14:45:30.455 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.028886912, 2023-10-16T14:45:30.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000166016 2023-10-16T14:45:30.456 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.456 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:30.456 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:30.456 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 107424 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.457 INFO:teuthology.orchestra.run.smithi186.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:30.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_messages": 235, 2023-10-16T14:45:30.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages": 232, 2023-10-16T14:45:30.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_bytes": 477024, 2023-10-16T14:45:30.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_bytes": 139280, 2023-10-16T14:45:30.458 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_total_time": 0.031894391, 2023-10-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_send_time": 0.011684268, 2023-10-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_recv_time": 0.013823457, 2023-10-16T14:45:30.459 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_running_fast_dispatch_time": 0.005898364, 2023-10-16T14:45:30.460 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:30.460 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 231, 2023-10-16T14:45:30.460 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.018958659, 2023-10-16T14:45:30.460 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000082072 2023-10-16T14:45:30.460 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.461 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_recv_encrypted_bytes": 477024, 2023-10-16T14:45:30.462 INFO:teuthology.orchestra.run.smithi186.stdout: "msgr_send_encrypted_bytes": 139280 2023-10-16T14:45:30.462 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.462 INFO:teuthology.orchestra.run.smithi186.stdout: "cct": { 2023-10-16T14:45:30.462 INFO:teuthology.orchestra.run.smithi186.stdout: "total_workers": 0, 2023-10-16T14:45:30.462 INFO:teuthology.orchestra.run.smithi186.stdout: "unhealthy_workers": 0 2023-10-16T14:45:30.463 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.463 INFO:teuthology.orchestra.run.smithi186.stdout: "mempool": { 2023-10-16T14:45:30.463 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:30.463 INFO:teuthology.orchestra.run.smithi186.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:30.463 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:30.464 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:30.464 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:30.464 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:30.464 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:30.464 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:30.465 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:30.466 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:30.466 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:30.466 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:30.466 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:30.466 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:30.467 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:30.467 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:30.467 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:30.467 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:30.468 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:30.469 INFO:teuthology.orchestra.run.smithi186.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:30.469 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:30.469 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_items": 0, 2023-10-16T14:45:30.469 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:30.469 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:30.470 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:30.470 INFO:teuthology.orchestra.run.smithi186.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:30.470 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_bytes": 149007, 2023-10-16T14:45:30.470 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_anon_items": 30, 2023-10-16T14:45:30.470 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:30.471 INFO:teuthology.orchestra.run.smithi186.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:30.471 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_bytes": 0, 2023-10-16T14:45:30.471 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_items": 0, 2023-10-16T14:45:30.471 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:30.471 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_items": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:30.472 INFO:teuthology.orchestra.run.smithi186.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:30.473 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:30.473 INFO:teuthology.orchestra.run.smithi186.stdout: "pgmap_items": 0, 2023-10-16T14:45:30.473 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:30.473 INFO:teuthology.orchestra.run.smithi186.stdout: "mds_co_items": 2, 2023-10-16T14:45:30.473 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_1_items": 0, 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: "unittest_2_items": 0 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: }, 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:30.474 INFO:teuthology.orchestra.run.smithi186.stdout: "val": 0, 2023-10-16T14:45:30.475 INFO:teuthology.orchestra.run.smithi186.stdout: "max": 104857600, 2023-10-16T14:45:30.475 INFO:teuthology.orchestra.run.smithi186.stdout: "get_started": 0, 2023-10-16T14:45:30.475 INFO:teuthology.orchestra.run.smithi186.stdout: "get": 236, 2023-10-16T14:45:30.475 INFO:teuthology.orchestra.run.smithi186.stdout: "get_sum": 443792, 2023-10-16T14:45:30.475 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "get_or_fail_success": 236, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "take": 0, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "take_sum": 0, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "put": 236, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "put_sum": 443792, 2023-10-16T14:45:30.476 INFO:teuthology.orchestra.run.smithi186.stdout: "wait": { 2023-10-16T14:45:30.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgcount": 0, 2023-10-16T14:45:30.477 INFO:teuthology.orchestra.run.smithi186.stdout: "sum": 0.000000000, 2023-10-16T14:45:30.477 INFO:teuthology.orchestra.run.smithi186.stdout: "avgtime": 0.000000000 2023-10-16T14:45:30.477 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:30.477 INFO:teuthology.orchestra.run.smithi186.stdout: } 2023-10-16T14:45:30.478 INFO:teuthology.orchestra.run.smithi186.stdout:} 2023-10-16T14:45:30.765 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-10-16T14:45:30.766 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-10-16T14:45:30.766 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-10-16T14:45:30.767 DEBUG:tasks.check_counter:Getting stats from c 2023-10-16T14:45:30.767 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.c perf dump 2023-10-16T14:45:31.415 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:45:31.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:31 smithi178 ceph-mon[98495]: pgmap v693: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-10-16T14:45:31.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:31 smithi186 ceph-mon[134892]: pgmap v693: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-10-16T14:45:31.875 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:31 smithi187 ceph-mon[135297]: pgmap v693: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2023-10-16T14:45:32.955 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-10-16T14:45:32.956 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:32.956 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1794, 2023-10-16T14:45:32.956 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1796, 2023-10-16T14:45:32.956 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 213411103, 2023-10-16T14:45:32.956 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 557944, 2023-10-16T14:45:32.957 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2023-10-16T14:45:32.957 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:32.957 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.245703049, 2023-10-16T14:45:32.957 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.041573502, 2023-10-16T14:45:32.957 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.134340825, 2023-10-16T14:45:32.958 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.055026885, 2023-10-16T14:45:32.958 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:32.958 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1794, 2023-10-16T14:45:32.958 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.094684682, 2023-10-16T14:45:32.958 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000052778 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.959 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.960 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:32.960 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:32.960 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.960 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:32.960 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 600, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 771, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 172128867, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 555063, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 7, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2023-10-16T14:45:32.961 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.131583365, 2023-10-16T14:45:32.962 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.019214270, 2023-10-16T14:45:32.962 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.091605320, 2023-10-16T14:45:32.962 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.015492686, 2023-10-16T14:45:32.962 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:32.962 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 770, 2023-10-16T14:45:32.963 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.054881734, 2023-10-16T14:45:32.963 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000071274 2023-10-16T14:45:32.963 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.963 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:32.963 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 369952 2023-10-16T14:45:32.964 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1107, 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1089, 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 176956692, 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 403725, 2023-10-16T14:45:32.965 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2023-10-16T14:45:32.966 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:32.966 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.188406135, 2023-10-16T14:45:32.966 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.032549686, 2023-10-16T14:45:32.966 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.119350170, 2023-10-16T14:45:32.966 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.030642026, 2023-10-16T14:45:32.967 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:32.967 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1088, 2023-10-16T14:45:32.967 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.087523320, 2023-10-16T14:45:32.967 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000080444 2023-10-16T14:45:32.967 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.968 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 553552, 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 138752 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-10-16T14:45:32.969 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.970 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-10-16T14:45:32.970 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:32.970 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:32.970 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5104, 2023-10-16T14:45:32.970 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.348922119, 2023-10-16T14:45:32.971 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000068362 2023-10-16T14:45:32.971 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:32.971 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.971 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:32.971 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:32.972 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.973 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-10-16T14:45:32.973 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2023-10-16T14:45:32.973 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2023-10-16T14:45:32.973 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-10-16T14:45:32.973 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2023-10-16T14:45:32.974 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-10-16T14:45:32.975 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-10-16T14:45:32.975 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2023-10-16T14:45:32.975 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-10-16T14:45:32.975 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-10-16T14:45:32.975 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 38382, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 1052, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 37310, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 20, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 36879, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 13352, 2023-10-16T14:45:32.976 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 2, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2023-10-16T14:45:32.977 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2023-10-16T14:45:32.978 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-10-16T14:45:32.978 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:32.978 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2023-10-16T14:45:32.978 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2023-10-16T14:45:32.978 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:32.979 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2023-10-16T14:45:32.980 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:32.980 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 4, 2023-10-16T14:45:32.980 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 246, 2023-10-16T14:45:32.980 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-10-16T14:45:32.980 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2023-10-16T14:45:32.981 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:32.981 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:32.981 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:32.981 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2023-10-16T14:45:32.981 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2023-10-16T14:45:32.982 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-10-16T14:45:32.983 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:32.983 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:32.983 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2023-10-16T14:45:32.983 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2023-10-16T14:45:32.983 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2023-10-16T14:45:32.984 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2023-10-16T14:45:32.985 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2023-10-16T14:45:32.985 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2023-10-16T14:45:32.985 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-10-16T14:45:32.985 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:32.985 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:32.986 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2023-10-16T14:45:32.986 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2023-10-16T14:45:32.986 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:32.986 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-10-16T14:45:32.986 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-10-16T14:45:32.987 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.988 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-10-16T14:45:32.989 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-10-16T14:45:32.989 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2023-10-16T14:45:32.989 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-10-16T14:45:32.989 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-10-16T14:45:32.989 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 250274, 2023-10-16T14:45:32.990 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-10-16T14:45:32.990 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-10-16T14:45:32.990 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2023-10-16T14:45:32.990 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-10-16T14:45:32.990 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 135, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 22, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 358107610, 2023-10-16T14:45:32.991 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 565192276, 2023-10-16T14:45:32.992 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 565190288, 2023-10-16T14:45:32.992 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-10-16T14:45:32.992 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.992 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.992 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 250274 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 25, 2023-10-16T14:45:32.993 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 79361, 2023-10-16T14:45:32.994 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 79336, 2023-10-16T14:45:32.994 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 20, 2023-10-16T14:45:32.994 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 3185, 2023-10-16T14:45:32.994 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 3165, 2023-10-16T14:45:32.994 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 38382, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 176661, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 138279, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2023-10-16T14:45:32.995 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 94744, 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 207152 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2023-10-16T14:45:32.996 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2023-10-16T14:45:32.997 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2023-10-16T14:45:32.997 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2023-10-16T14:45:32.997 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-10-16T14:45:32.997 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.997 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.998 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:32.999 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:32.999 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:32.999 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:32.999 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:32.999 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.000 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.001 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:33.002 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.002 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.002 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.002 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.002 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-10-16T14:45:33.003 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.004 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.004 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.004 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.004 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:33.004 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.005 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.005 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.005 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.005 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:33.005 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.006 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.007 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.007 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-10-16T14:45:33.007 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.007 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.008 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.008 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-10-16T14:45:33.008 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.008 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.009 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.010 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.010 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-10-16T14:45:33.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.010 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.010 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.011 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.012 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-10-16T14:45:33.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.012 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.012 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.012 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.013 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:33.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.014 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.014 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.014 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.014 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:33.015 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.016 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.016 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.016 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:33.016 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.017 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.018 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.018 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:33.018 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.018 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.019 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.020 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.020 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-10-16T14:45:33.020 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.020 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.020 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.021 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.022 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-10-16T14:45:33.022 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.022 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.022 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.022 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.023 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.024 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.025 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.025 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-10-16T14:45:33.025 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-10-16T14:45:33.025 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2023-10-16T14:45:33.025 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 2, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 2559, 2023-10-16T14:45:33.026 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:33.027 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.027 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-10-16T14:45:33.027 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:33.027 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:33.027 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:33.028 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:33.028 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:33.028 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:33.028 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:33.028 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:33.029 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:33.030 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:33.030 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:33.030 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:33.030 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:33.030 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:33.031 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:33.032 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:33.032 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:33.032 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:33.032 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:33.032 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:33.033 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 238693, 2023-10-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 78, 2023-10-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:33.034 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:33.035 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 172, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-10-16T14:45:33.036 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 26793889, 2023-10-16T14:45:33.037 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 154022, 2023-10-16T14:45:33.037 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:33.037 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-10-16T14:45:33.037 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:33.037 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 3241, 2023-10-16T14:45:33.038 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 26, 2023-10-16T14:45:33.039 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 4, 2023-10-16T14:45:33.039 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 3241, 2023-10-16T14:45:33.039 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-10-16T14:45:33.039 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3241, 2023-10-16T14:45:33.039 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 45.013258419, 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.013888694 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3241, 2023-10-16T14:45:33.040 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 3242 2023-10-16T14:45:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 3241, 2023-10-16T14:45:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 3241, 2023-10-16T14:45:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2023-10-16T14:45:33.041 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-10-16T14:45:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-10-16T14:45:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 1928, 2023-10-16T14:45:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2023-10-16T14:45:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1310, 2023-10-16T14:45:33.042 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-10-16T14:45:33.043 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 2, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2023-10-16T14:45:33.044 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-10-16T14:45:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-10-16T14:45:33.045 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-10-16T14:45:33.046 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-10-16T14:45:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-10-16T14:45:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-10-16T14:45:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-10-16T14:45:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-10-16T14:45:33.047 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-10-16T14:45:33.048 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-10-16T14:45:33.049 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 2, 2023-10-16T14:45:33.050 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2023-10-16T14:45:33.050 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 11, 2023-10-16T14:45:33.050 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2023-10-16T14:45:33.050 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-10-16T14:45:33.050 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2023-10-16T14:45:33.051 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2023-10-16T14:45:33.051 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2023-10-16T14:45:33.051 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.051 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-10-16T14:45:33.051 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-10-16T14:45:33.052 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-10-16T14:45:33.053 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.053 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-10-16T14:45:33.053 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:33.053 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2023-10-16T14:45:33.053 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:33.054 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:33.055 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:33.055 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:33.055 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 3497, 2023-10-16T14:45:33.055 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 562165940, 2023-10-16T14:45:33.055 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 3497, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 3497, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 562165940, 2023-10-16T14:45:33.056 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.057 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:33.058 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:33.058 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:33.058 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:33.058 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:33.058 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 3241, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 561050360, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 3241, 2023-10-16T14:45:33.059 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 561050360, 2023-10-16T14:45:33.060 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:33.060 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.060 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.060 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.060 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:33.061 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:33.062 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:33.062 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:33.062 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:33.062 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 3241, 2023-10-16T14:45:33.062 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 3241, 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 3241, 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 3241, 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.063 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.064 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.064 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.064 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:33.064 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:33.064 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:33.065 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:33.066 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:33.066 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-10-16T14:45:33.066 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-10-16T14:45:33.066 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:33.066 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55a223466960": { 2023-10-16T14:45:33.067 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:33.068 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:33.069 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:33.069 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:33.069 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-10-16T14:45:33.069 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-10-16T14:45:33.069 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:33.070 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-10-16T14:45:33.383 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-10-16T14:45:33.384 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-10-16T14:45:33.384 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-10-16T14:45:33.384 DEBUG:tasks.check_counter:Getting stats from f 2023-10-16T14:45:33.385 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.f perf dump 2023-10-16T14:45:33.946 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:45:34.303 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:34 smithi178 ceph-mon[98495]: pgmap v694: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-10-16T14:45:34.312 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:34 smithi187 ceph-mon[135297]: pgmap v694: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-10-16T14:45:34.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:34 smithi186 ceph-mon[134892]: pgmap v694: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2023-10-16T14:45:35.144 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-10-16T14:45:35.144 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:35.144 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1937, 2023-10-16T14:45:35.145 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1939, 2023-10-16T14:45:35.145 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 104181868, 2023-10-16T14:45:35.145 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 602414, 2023-10-16T14:45:35.145 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2023-10-16T14:45:35.145 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:35.146 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.204785765, 2023-10-16T14:45:35.146 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.046274140, 2023-10-16T14:45:35.146 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.086487250, 2023-10-16T14:45:35.146 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.058290359, 2023-10-16T14:45:35.146 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:35.147 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1937, 2023-10-16T14:45:35.147 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.110059259, 2023-10-16T14:45:35.147 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000056819 2023-10-16T14:45:35.147 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.147 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:35.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.148 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.148 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.148 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 309, 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 481, 2023-10-16T14:45:35.149 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 71387836, 2023-10-16T14:45:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 466318, 2023-10-16T14:45:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2023-10-16T14:45:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.070763563, 2023-10-16T14:45:35.150 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.011724890, 2023-10-16T14:45:35.151 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.047490267, 2023-10-16T14:45:35.151 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.007922405, 2023-10-16T14:45:35.151 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:35.151 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 480, 2023-10-16T14:45:35.151 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.035582654, 2023-10-16T14:45:35.152 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000074130 2023-10-16T14:45:35.152 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.152 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:35.152 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.152 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3104, 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 371712 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.153 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:35.154 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 691, 2023-10-16T14:45:35.154 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 673, 2023-10-16T14:45:35.154 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 109729444, 2023-10-16T14:45:35.154 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 274812, 2023-10-16T14:45:35.154 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 5, 2023-10-16T14:45:35.155 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:35.155 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.124637611, 2023-10-16T14:45:35.155 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.021055297, 2023-10-16T14:45:35.155 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.082366864, 2023-10-16T14:45:35.155 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.017769853, 2023-10-16T14:45:35.156 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:35.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 672, 2023-10-16T14:45:35.156 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.083973359, 2023-10-16T14:45:35.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000124960 2023-10-16T14:45:35.156 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.157 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 553664, 2023-10-16T14:45:35.158 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 139216 2023-10-16T14:45:35.158 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.158 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-10-16T14:45:35.158 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-10-16T14:45:35.159 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-10-16T14:45:35.159 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.159 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-10-16T14:45:35.159 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:35.160 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:35.160 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4415, 2023-10-16T14:45:35.160 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.177235496, 2023-10-16T14:45:35.160 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000040143 2023-10-16T14:45:35.161 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.161 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.161 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:35.161 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:35.162 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:35.162 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.162 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.162 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.163 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.163 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.163 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-10-16T14:45:35.163 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2023-10-16T14:45:35.164 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2023-10-16T14:45:35.164 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-10-16T14:45:35.164 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.164 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-10-16T14:45:35.165 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-10-16T14:45:35.166 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2023-10-16T14:45:35.166 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2023-10-16T14:45:35.166 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2023-10-16T14:45:35.166 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 13553, 2023-10-16T14:45:35.166 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 0, 2023-10-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 0, 2023-10-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 13553, 2023-10-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 13553, 2023-10-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 4222, 2023-10-16T14:45:35.167 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2023-10-16T14:45:35.168 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2023-10-16T14:45:35.168 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 1, 2023-10-16T14:45:35.168 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2023-10-16T14:45:35.168 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2023-10-16T14:45:35.168 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2023-10-16T14:45:35.169 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2023-10-16T14:45:35.169 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-10-16T14:45:35.169 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:35.169 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2023-10-16T14:45:35.170 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2023-10-16T14:45:35.170 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2023-10-16T14:45:35.170 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2023-10-16T14:45:35.170 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2023-10-16T14:45:35.171 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2023-10-16T14:45:35.171 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2023-10-16T14:45:35.171 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:35.172 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2023-10-16T14:45:35.172 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:35.172 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2023-10-16T14:45:35.172 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2023-10-16T14:45:35.173 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-10-16T14:45:35.173 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2023-10-16T14:45:35.173 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:35.173 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:35.174 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:35.174 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2023-10-16T14:45:35.174 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-10-16T14:45:35.174 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2023-10-16T14:45:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:35.176 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:35.176 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2023-10-16T14:45:35.176 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2023-10-16T14:45:35.176 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2023-10-16T14:45:35.176 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2023-10-16T14:45:35.177 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2023-10-16T14:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2023-10-16T14:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-10-16T14:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:35.178 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2023-10-16T14:45:35.179 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2023-10-16T14:45:35.179 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:35.179 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-10-16T14:45:35.179 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:35.179 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2023-10-16T14:45:35.180 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-10-16T14:45:35.181 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2023-10-16T14:45:35.181 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:35.181 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-10-16T14:45:35.181 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.181 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 129692, 2023-10-16T14:45:35.182 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-10-16T14:45:35.183 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2023-10-16T14:45:35.183 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2023-10-16T14:45:35.183 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2023-10-16T14:45:35.183 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2023-10-16T14:45:35.183 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 144, 2023-10-16T14:45:35.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 12, 2023-10-16T14:45:35.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-10-16T14:45:35.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2023-10-16T14:45:35.184 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 54122398, 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 288173220, 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 288172621, 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.185 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.186 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.186 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 129692 2023-10-16T14:45:35.186 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.186 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-10-16T14:45:35.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 13, 2023-10-16T14:45:35.187 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 45931, 2023-10-16T14:45:35.187 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 45918, 2023-10-16T14:45:35.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 11, 2023-10-16T14:45:35.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 1676, 2023-10-16T14:45:35.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 1665, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 13553, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 80944, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 67391, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2023-10-16T14:45:35.188 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2023-10-16T14:45:35.189 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 93976, 2023-10-16T14:45:35.189 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 207152 2023-10-16T14:45:35.189 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.189 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-10-16T14:45:35.189 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-10-16T14:45:35.190 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.191 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.191 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.191 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-10-16T14:45:35.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.192 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.192 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.192 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:35.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.193 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.194 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.194 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.194 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-10-16T14:45:35.194 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.195 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.195 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.195 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.195 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:35.196 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.196 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.196 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.196 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.197 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-10-16T14:45:35.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.197 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.197 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.198 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-10-16T14:45:35.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.198 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.199 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.199 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:35.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.199 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.200 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.200 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:35.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.200 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.201 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.201 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-10-16T14:45:35.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.202 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.202 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.203 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-10-16T14:45:35.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.203 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.204 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.204 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-10-16T14:45:35.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.204 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.205 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.205 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-10-16T14:45:35.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.206 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.206 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.206 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-10-16T14:45:35.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.207 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.207 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.207 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-10-16T14:45:35.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.208 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.208 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.209 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-10-16T14:45:35.209 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.209 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.210 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.210 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.210 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:35.210 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.211 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.211 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.211 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:35.211 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.212 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.212 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.212 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.212 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-10-16T14:45:35.212 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.213 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.213 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.213 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.213 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:35.213 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.214 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.214 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.214 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.214 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:35.214 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.215 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.216 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.216 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:35.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.216 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.217 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.218 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.219 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-10-16T14:45:35.219 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.219 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.219 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.219 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.220 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-10-16T14:45:35.221 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.221 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.221 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.221 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.221 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-10-16T14:45:35.222 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.222 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.222 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.222 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.222 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.223 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.224 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.224 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-10-16T14:45:35.224 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-10-16T14:45:35.224 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2023-10-16T14:45:35.224 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 2562, 2023-10-16T14:45:35.225 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:35.226 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.226 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-10-16T14:45:35.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:35.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:35.226 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:35.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:35.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:35.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:35.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:35.227 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:35.228 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:35.229 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:35.229 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:35.229 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:35.229 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:35.230 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:35.230 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:35.230 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:35.230 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:35.231 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:35.231 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:35.231 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:35.231 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:35.232 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:35.232 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:35.232 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:35.232 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:35.232 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:35.233 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-10-16T14:45:35.233 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:35.233 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:35.233 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:35.234 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:35.234 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 231391, 2023-10-16T14:45:35.235 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 75, 2023-10-16T14:45:35.235 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:35.235 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:35.235 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-10-16T14:45:35.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-10-16T14:45:35.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:35.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:35.236 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:35.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:35.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:35.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 172, 2023-10-16T14:45:35.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:35.237 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:35.238 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:35.238 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-10-16T14:45:35.238 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 9475648, 2023-10-16T14:45:35.238 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 54340, 2023-10-16T14:45:35.238 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-10-16T14:45:35.239 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-10-16T14:45:35.240 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-10-16T14:45:35.240 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 2677, 2023-10-16T14:45:35.240 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2023-10-16T14:45:35.240 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2023-10-16T14:45:35.240 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 2677, 2023-10-16T14:45:35.241 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-10-16T14:45:35.241 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2677, 2023-10-16T14:45:35.241 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 72.149808533, 2023-10-16T14:45:35.241 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.026951740 2023-10-16T14:45:35.242 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.242 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-10-16T14:45:35.242 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-10-16T14:45:35.242 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2677, 2023-10-16T14:45:35.243 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2678 2023-10-16T14:45:35.243 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.243 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 2677, 2023-10-16T14:45:35.243 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 2677, 2023-10-16T14:45:35.244 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2023-10-16T14:45:35.244 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-10-16T14:45:35.244 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-10-16T14:45:35.244 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 1570, 2023-10-16T14:45:35.245 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2023-10-16T14:45:35.245 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1105, 2023-10-16T14:45:35.245 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2023-10-16T14:45:35.245 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2023-10-16T14:45:35.246 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-10-16T14:45:35.246 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-10-16T14:45:35.246 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-10-16T14:45:35.246 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2023-10-16T14:45:35.247 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2023-10-16T14:45:35.247 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-10-16T14:45:35.247 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:35.247 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:35.247 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-10-16T14:45:35.248 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-10-16T14:45:35.249 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-10-16T14:45:35.249 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:35.249 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-10-16T14:45:35.249 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:35.249 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-10-16T14:45:35.250 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-10-16T14:45:35.251 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-10-16T14:45:35.251 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-10-16T14:45:35.251 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-10-16T14:45:35.251 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-10-16T14:45:35.251 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-10-16T14:45:35.252 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-10-16T14:45:35.252 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-10-16T14:45:35.252 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-10-16T14:45:35.252 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-10-16T14:45:35.252 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-10-16T14:45:35.253 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-10-16T14:45:35.253 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-10-16T14:45:35.253 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 2, 2023-10-16T14:45:35.253 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 10, 2023-10-16T14:45:35.253 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 10, 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2023-10-16T14:45:35.254 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.255 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-10-16T14:45:35.255 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2023-10-16T14:45:35.255 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:35.255 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-10-16T14:45:35.255 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-10-16T14:45:35.256 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.256 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-10-16T14:45:35.256 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2023-10-16T14:45:35.256 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.256 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2023-10-16T14:45:35.257 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:35.258 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.258 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:35.258 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:35.258 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:35.258 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 2933, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 285019180, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 2933, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:35.259 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:35.260 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2933, 2023-10-16T14:45:35.260 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 285019180, 2023-10-16T14:45:35.260 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:35.260 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.260 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:35.261 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:35.262 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:35.262 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:35.262 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:35.262 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:35.262 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:35.263 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2677, 2023-10-16T14:45:35.263 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 284000058, 2023-10-16T14:45:35.263 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2677, 2023-10-16T14:45:35.265 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 284000058, 2023-10-16T14:45:35.265 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:35.265 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.265 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.266 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.266 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.266 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.266 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:35.266 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:35.267 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-10-16T14:45:35.267 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:35.267 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:35.267 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:35.268 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:35.268 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:35.268 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 2677, 2023-10-16T14:45:35.268 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 2677, 2023-10-16T14:45:35.268 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2677, 2023-10-16T14:45:35.269 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2677, 2023-10-16T14:45:35.269 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:35.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.269 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.270 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.270 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.270 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.270 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:35.271 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:35.271 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:35.271 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:35.272 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:35.272 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:35.272 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:35.273 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:35.273 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:35.273 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:35.274 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-10-16T14:45:35.274 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-10-16T14:45:35.274 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:35.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.275 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.276 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.276 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:35.276 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55c9f34ba960": { 2023-10-16T14:45:35.277 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:35.277 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:35.277 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:35.278 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:35.278 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:35.278 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:35.278 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:35.279 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:35.279 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:35.279 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2023-10-16T14:45:35.279 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2023-10-16T14:45:35.280 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:35.280 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:35.280 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:35.280 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:35.281 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.281 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:35.281 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-10-16T14:45:35.488 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-10-16T14:45:35.488 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-10-16T14:45:35.489 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-10-16T14:45:35.489 DEBUG:tasks.check_counter:Getting stats from i 2023-10-16T14:45:35.490 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.i perf dump 2023-10-16T14:45:36.048 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:45:36.280 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:36 smithi187 ceph-mon[135297]: pgmap v695: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:45:36.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:36 smithi178 ceph-mon[98495]: pgmap v695: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:45:36.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:36 smithi186 ceph-mon[134892]: pgmap v695: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-10-16T14:45:37.265 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-10-16T14:45:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 82965, 2023-10-16T14:45:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 64069, 2023-10-16T14:45:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 53581827, 2023-10-16T14:45:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 96263101, 2023-10-16T14:45:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 22, 2023-10-16T14:45:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2023-10-16T14:45:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 3.836611950, 2023-10-16T14:45:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 1.364272183, 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 3.570775091, 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.719468698, 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 64068, 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 14.071170306, 2023-10-16T14:45:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000219628 2023-10-16T14:45:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 105856, 2023-10-16T14:45:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.013720256, 2023-10-16T14:45:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000129 2023-10-16T14:45:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 573600, 2023-10-16T14:45:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 311296 2023-10-16T14:45:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 273107, 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 386688, 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 91675120, 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 284194887, 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 12, 2023-10-16T14:45:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 10.781871061, 2023-10-16T14:45:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 5.091135155, 2023-10-16T14:45:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 13.429549189, 2023-10-16T14:45:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.890583496, 2023-10-16T14:45:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 386686, 2023-10-16T14:45:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 123.619315739, 2023-10-16T14:45:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000319689 2023-10-16T14:45:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 237762, 2023-10-16T14:45:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.072091809, 2023-10-16T14:45:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000303 2023-10-16T14:45:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1728, 2023-10-16T14:45:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 377360 2023-10-16T14:45:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 55244, 2023-10-16T14:45:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 55244, 2023-10-16T14:45:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 16101194, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 122349545, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 13, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 3.387506192, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 1.032961149, 2023-10-16T14:45:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.677522320, 2023-10-16T14:45:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.480908527, 2023-10-16T14:45:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 55243, 2023-10-16T14:45:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 27.186336370, 2023-10-16T14:45:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000492122 2023-10-16T14:45:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4, 2023-10-16T14:45:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000505, 2023-10-16T14:45:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000126 2023-10-16T14:45:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 2944, 2023-10-16T14:45:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 448 2023-10-16T14:45:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-10-16T14:45:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2023-10-16T14:45:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-10-16T14:45:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2023-10-16T14:45:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 38446, 2023-10-16T14:45:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 100.486047943, 2023-10-16T14:45:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002613693 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2023-10-16T14:45:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 28717, 2023-10-16T14:45:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 6.412606459, 2023-10-16T14:45:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000223303 2023-10-16T14:45:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2023-10-16T14:45:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 109961, 2023-10-16T14:45:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 108482, 2023-10-16T14:45:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2023-10-16T14:45:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 108482, 2023-10-16T14:45:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2909.051864419, 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.026815986 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 1479, 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2023-10-16T14:45:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2023-10-16T14:45:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 943, 2023-10-16T14:45:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 88, 2023-10-16T14:45:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 4, 2023-10-16T14:45:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 17, 2023-10-16T14:45:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 2, 2023-10-16T14:45:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 2, 2023-10-16T14:45:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 13, 2023-10-16T14:45:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 13, 2023-10-16T14:45:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 60139, 2023-10-16T14:45:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 2, 2023-10-16T14:45:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 2, 2023-10-16T14:45:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 3, 2023-10-16T14:45:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 185541, 2023-10-16T14:45:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 157548, 2023-10-16T14:45:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 828, 2023-10-16T14:45:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 503, 2023-10-16T14:45:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2023-10-16T14:45:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2023-10-16T14:45:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 700, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 1000, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 1, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 238, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 46915, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 256, 2023-10-16T14:45:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 38188, 2023-10-16T14:45:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2023-10-16T14:45:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 50606, 2023-10-16T14:45:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2023-10-16T14:45:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 1623, 2023-10-16T14:45:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 35380239, 2023-10-16T14:45:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2023-10-16T14:45:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 50606, 2023-10-16T14:45:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2023-10-16T14:45:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2023-10-16T14:45:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 301, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 301, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 50305, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 302, 2023-10-16T14:45:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 24585, 2023-10-16T14:45:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 41750, 2023-10-16T14:45:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 4, 2023-10-16T14:45:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-10-16T14:45:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2023-10-16T14:45:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 74617, 2023-10-16T14:45:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 26273, 2023-10-16T14:45:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 1352, 2023-10-16T14:45:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 50556 2023-10-16T14:45:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2023-10-16T14:45:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2023-10-16T14:45:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 6, 2023-10-16T14:45:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 4, 2023-10-16T14:45:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 592, 2023-10-16T14:45:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 953, 2023-10-16T14:45:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2023-10-16T14:45:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2023-10-16T14:45:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2023-10-16T14:45:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 15828, 2023-10-16T14:45:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 15828, 2023-10-16T14:45:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2023-10-16T14:45:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2023-10-16T14:45:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2023-10-16T14:45:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 240, 2023-10-16T14:45:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2023-10-16T14:45:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 92, 2023-10-16T14:45:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2023-10-16T14:45:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2023-10-16T14:45:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2023-10-16T14:45:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 3, 2023-10-16T14:45:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 129692, 2023-10-16T14:45:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 25695, 2023-10-16T14:45:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 24581, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 105111, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 13404, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 159, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 31, 2023-10-16T14:45:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 29, 2023-10-16T14:45:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 130, 2023-10-16T14:45:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 9, 2023-10-16T14:45:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2023-10-16T14:45:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 16, 2023-10-16T14:45:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 54122398, 2023-10-16T14:45:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 288173220, 2023-10-16T14:45:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2023-10-16T14:45:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2023-10-16T14:45:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2924, 2023-10-16T14:45:37.307 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 601.706746860, 2023-10-16T14:45:37.307 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.205782061 2023-10-16T14:45:37.307 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.307 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2023-10-16T14:45:37.307 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 22, 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 46049, 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 46027, 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 20, 2023-10-16T14:45:37.308 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 1730, 2023-10-16T14:45:37.309 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 1710, 2023-10-16T14:45:37.309 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 17, 2023-10-16T14:45:37.309 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 84587, 2023-10-16T14:45:37.309 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 84570, 2023-10-16T14:45:37.309 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 2, 2023-10-16T14:45:37.310 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 45610, 2023-10-16T14:45:37.310 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 45608, 2023-10-16T14:45:37.310 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 151720, 2023-10-16T14:45:37.310 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 198960 2023-10-16T14:45:37.310 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.311 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2023-10-16T14:45:37.311 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 118651, 2023-10-16T14:45:37.311 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 2236, 2023-10-16T14:45:37.311 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 109961, 2023-10-16T14:45:37.311 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 338, 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 4579, 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 23195, 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 133.631202696, 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005761207 2023-10-16T14:45:37.312 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.313 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2023-10-16T14:45:37.313 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 489, 2023-10-16T14:45:37.313 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 128.739888861, 2023-10-16T14:45:37.313 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.263271756 2023-10-16T14:45:37.314 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.314 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2023-10-16T14:45:37.314 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.314 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.314 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.315 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.315 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2023-10-16T14:45:37.315 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.315 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.315 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.316 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.316 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2023-10-16T14:45:37.316 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 47395, 2023-10-16T14:45:37.316 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2250.455743826, 2023-10-16T14:45:37.316 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.047482978 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.317 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.318 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2023-10-16T14:45:37.318 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 2, 2023-10-16T14:45:37.318 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000416046, 2023-10-16T14:45:37.318 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000208023 2023-10-16T14:45:37.318 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.319 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2023-10-16T14:45:37.319 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.319 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.319 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.319 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.320 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2023-10-16T14:45:37.321 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.321 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.321 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.321 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.321 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2023-10-16T14:45:37.322 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.322 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.322 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.322 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.322 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 80, 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 8.320992086, 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.104012401 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2023-10-16T14:45:37.323 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.324 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.324 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.324 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.324 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2023-10-16T14:45:37.324 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.325 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.325 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.325 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.325 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2023-10-16T14:45:37.325 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 8060, 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 49.161756335, 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.006099473 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 301, 2023-10-16T14:45:37.326 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.840234620, 2023-10-16T14:45:37.327 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002791477 2023-10-16T14:45:37.327 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.327 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2023-10-16T14:45:37.327 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 22802, 2023-10-16T14:45:37.328 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 322.548598564, 2023-10-16T14:45:37.328 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.014145627 2023-10-16T14:45:37.328 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.328 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2023-10-16T14:45:37.328 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.329 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.329 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.329 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.329 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2023-10-16T14:45:37.329 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 39, 2023-10-16T14:45:37.330 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 6.838845949, 2023-10-16T14:45:37.331 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.175355024 2023-10-16T14:45:37.331 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.331 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2023-10-16T14:45:37.331 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.331 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.332 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.332 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.332 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2023-10-16T14:45:37.332 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.332 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5, 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.024056068, 2023-10-16T14:45:37.333 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.004811213 2023-10-16T14:45:37.334 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.334 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2023-10-16T14:45:37.334 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.334 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.334 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.335 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.336 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2023-10-16T14:45:37.336 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.336 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.336 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.336 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.337 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2023-10-16T14:45:37.338 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3, 2023-10-16T14:45:37.338 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.002298411, 2023-10-16T14:45:37.338 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000766137 2023-10-16T14:45:37.338 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.338 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2023-10-16T14:45:37.339 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6111, 2023-10-16T14:45:37.339 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 8.487830957, 2023-10-16T14:45:37.339 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001388943 2023-10-16T14:45:37.339 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.339 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2023-10-16T14:45:37.340 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2023-10-16T14:45:37.340 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2023-10-16T14:45:37.340 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.340 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.340 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 1, 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 2, 2023-10-16T14:45:37.341 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2023-10-16T14:45:37.342 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 1, 2023-10-16T14:45:37.342 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2023-10-16T14:45:37.342 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 3217, 2023-10-16T14:45:37.342 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 3217, 2023-10-16T14:45:37.342 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 681, 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:37.343 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:37.344 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:37.344 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:37.344 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:37.344 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:37.344 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:37.345 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:37.346 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:37.346 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:37.346 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:37.346 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:37.346 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:37.347 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:37.347 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:37.347 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:37.347 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:37.347 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:37.348 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:37.348 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:37.348 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:37.348 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:37.348 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:37.349 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:37.350 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:37.350 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 315828, 2023-10-16T14:45:37.350 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 144, 2023-10-16T14:45:37.350 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 792, 2023-10-16T14:45:37.350 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 9, 2023-10-16T14:45:37.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-10-16T14:45:37.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-10-16T14:45:37.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:37.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:37.351 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 8984, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 172, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:37.352 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:37.353 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-10-16T14:45:37.353 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 87997, 2023-10-16T14:45:37.353 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 521, 2023-10-16T14:45:37.353 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:37.353 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2023-10-16T14:45:37.354 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2023-10-16T14:45:37.355 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 94256, 2023-10-16T14:45:37.355 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 308759868, 2023-10-16T14:45:37.355 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 33, 2023-10-16T14:45:37.355 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 94256, 2023-10-16T14:45:37.355 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 94256, 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4562.287822806, 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.048403155 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2023-10-16T14:45:37.356 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2023-10-16T14:45:37.357 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 94256, 2023-10-16T14:45:37.357 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 105490 2023-10-16T14:45:37.357 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.357 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 94256, 2023-10-16T14:45:37.357 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 51059, 2023-10-16T14:45:37.358 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 43197, 2023-10-16T14:45:37.358 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2023-10-16T14:45:37.358 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2023-10-16T14:45:37.358 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 138, 2023-10-16T14:45:37.358 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 6489, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 453, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 3382, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 224, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2023-10-16T14:45:37.359 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2023-10-16T14:45:37.360 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 88, 2023-10-16T14:45:37.360 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 31960, 2023-10-16T14:45:37.360 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2023-10-16T14:45:37.360 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2023-10-16T14:45:37.360 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 50606, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 9759, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2023-10-16T14:45:37.361 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2023-10-16T14:45:37.362 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2023-10-16T14:45:37.362 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2023-10-16T14:45:37.362 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2023-10-16T14:45:37.362 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2023-10-16T14:45:37.363 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2023-10-16T14:45:37.363 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 4, 2023-10-16T14:45:37.363 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2023-10-16T14:45:37.363 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2023-10-16T14:45:37.363 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2023-10-16T14:45:37.364 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2023-10-16T14:45:37.365 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2023-10-16T14:45:37.365 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2023-10-16T14:45:37.365 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2023-10-16T14:45:37.365 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2023-10-16T14:45:37.365 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 91, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 3, 2023-10-16T14:45:37.366 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2023-10-16T14:45:37.367 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2023-10-16T14:45:37.367 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2023-10-16T14:45:37.367 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2023-10-16T14:45:37.367 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 2071, 2023-10-16T14:45:37.367 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2023-10-16T14:45:37.368 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 316 2023-10-16T14:45:37.368 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.368 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2023-10-16T14:45:37.368 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2023-10-16T14:45:37.368 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 915 2023-10-16T14:45:37.369 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2023-10-16T14:45:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2023-10-16T14:45:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 23, 2023-10-16T14:45:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2023-10-16T14:45:37.370 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 9, 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 15828, 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:37.371 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:37.372 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:37.372 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 411312, 2023-10-16T14:45:37.372 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 128361028, 2023-10-16T14:45:37.372 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:37.372 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 411312, 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 411312, 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 128361028, 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:37.373 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.374 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.374 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.374 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.374 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.374 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2023-10-16T14:45:37.375 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:37.375 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:37.375 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:37.375 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:37.375 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:37.376 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:37.376 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:37.376 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 94256, 2023-10-16T14:45:37.376 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 310485008, 2023-10-16T14:45:37.376 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 94256, 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 310485008, 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.377 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.378 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.378 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2023-10-16T14:45:37.378 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:37.378 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2023-10-16T14:45:37.378 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 94256, 2023-10-16T14:45:37.379 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 94256, 2023-10-16T14:45:37.380 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 94256, 2023-10-16T14:45:37.380 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 94256, 2023-10-16T14:45:37.380 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:37.380 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.380 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:37.381 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 15828, 2023-10-16T14:45:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 1725140, 2023-10-16T14:45:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:37.382 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 15828, 2023-10-16T14:45:37.383 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:37.383 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:37.383 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 458, 2023-10-16T14:45:37.383 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 1725140, 2023-10-16T14:45:37.383 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:37.384 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55ce4bb1e6e0": { 2023-10-16T14:45:37.385 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:37.385 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2023-10-16T14:45:37.385 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:37.385 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 129692, 2023-10-16T14:45:37.385 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 283978916, 2023-10-16T14:45:37.386 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:37.386 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 129692, 2023-10-16T14:45:37.386 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:37.386 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:37.386 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 2924, 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 283978916, 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:37.387 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.388 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:37.388 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-10-16T14:45:37.700 INFO:tasks.check_counter:Daemon mds.i mds.exported=238 2023-10-16T14:45:37.700 INFO:tasks.check_counter:Daemon mds.i mds.imported=256 2023-10-16T14:45:37.700 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=88 2023-10-16T14:45:37.701 DEBUG:tasks.check_counter:Getting stats from l 2023-10-16T14:45:37.701 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:4ba8a190d7130cb1f1e7ceedfc0d129df5dd29b0 shell --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 -- ceph daemon mds.l perf dump 2023-10-16T14:45:38.259 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/mon.c/config 2023-10-16T14:45:38.302 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:38 smithi178 ceph-mon[98495]: pgmap v696: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:45:38.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:38 smithi186 ceph-mon[134892]: pgmap v696: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:45:38.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:38 smithi187 ceph-mon[135297]: pgmap v696: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-10-16T14:45:39.461 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2023-10-16T14:45:39.462 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2023-10-16T14:45:39.462 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 0, 2023-10-16T14:45:39.463 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 2, 2023-10-16T14:45:39.463 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 0, 2023-10-16T14:45:39.463 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 0, 2023-10-16T14:45:39.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 4, 2023-10-16T14:45:39.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 0, 2023-10-16T14:45:39.464 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.001240920, 2023-10-16T14:45:39.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.000000000, 2023-10-16T14:45:39.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.000353229, 2023-10-16T14:45:39.465 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:39.466 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:39.466 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:39.467 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:39.467 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:39.468 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.468 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:39.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:39.470 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:39.470 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:39.471 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.471 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2023-10-16T14:45:39.472 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2023-10-16T14:45:39.472 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.472 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2023-10-16T14:45:39.472 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 242, 2023-10-16T14:45:39.473 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 237, 2023-10-16T14:45:39.473 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 480784, 2023-10-16T14:45:39.473 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 141600, 2023-10-16T14:45:39.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 2, 2023-10-16T14:45:39.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 1, 2023-10-16T14:45:39.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.035297971, 2023-10-16T14:45:39.474 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.013076451, 2023-10-16T14:45:39.475 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.015291567, 2023-10-16T14:45:39.475 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.006388980, 2023-10-16T14:45:39.475 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:39.475 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 237, 2023-10-16T14:45:39.476 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.022018698, 2023-10-16T14:45:39.476 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000092905 2023-10-16T14:45:39.476 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.477 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:39.477 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:39.477 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:39.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:39.478 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.478 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 480784, 2023-10-16T14:45:39.479 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 141600 2023-10-16T14:45:39.479 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.479 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2023-10-16T14:45:39.480 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1, 2023-10-16T14:45:39.480 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 179, 2023-10-16T14:45:39.480 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 160, 2023-10-16T14:45:39.480 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 107952, 2023-10-16T14:45:39.481 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 3, 2023-10-16T14:45:39.481 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 2, 2023-10-16T14:45:39.481 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.008747846, 2023-10-16T14:45:39.482 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.006415199, 2023-10-16T14:45:39.482 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.000703190, 2023-10-16T14:45:39.482 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-10-16T14:45:39.482 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2023-10-16T14:45:39.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 177, 2023-10-16T14:45:39.483 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.015902609, 2023-10-16T14:45:39.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000089845 2023-10-16T14:45:39.484 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.484 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2023-10-16T14:45:39.484 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:39.484 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:39.485 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:39.485 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.485 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2023-10-16T14:45:39.486 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 107952 2023-10-16T14:45:39.486 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.486 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2023-10-16T14:45:39.487 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 0, 2023-10-16T14:45:39.487 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2023-10-16T14:45:39.487 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.487 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2023-10-16T14:45:39.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2023-10-16T14:45:39.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2023-10-16T14:45:39.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2023-10-16T14:45:39.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2023-10-16T14:45:39.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2023-10-16T14:45:39.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2023-10-16T14:45:39.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2023-10-16T14:45:39.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2023-10-16T14:45:39.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2023-10-16T14:45:39.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2023-10-16T14:45:39.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2023-10-16T14:45:39.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2023-10-16T14:45:39.492 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2023-10-16T14:45:39.492 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2023-10-16T14:45:39.492 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2023-10-16T14:45:39.493 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2023-10-16T14:45:39.493 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2023-10-16T14:45:39.493 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2023-10-16T14:45:39.493 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2023-10-16T14:45:39.494 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2023-10-16T14:45:39.494 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2023-10-16T14:45:39.494 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2023-10-16T14:45:39.495 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2023-10-16T14:45:39.495 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2023-10-16T14:45:39.495 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2023-10-16T14:45:39.495 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2023-10-16T14:45:39.496 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2023-10-16T14:45:39.496 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2023-10-16T14:45:39.496 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2023-10-16T14:45:39.497 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2023-10-16T14:45:39.497 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2023-10-16T14:45:39.497 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2023-10-16T14:45:39.498 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2023-10-16T14:45:39.498 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2023-10-16T14:45:39.498 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2023-10-16T14:45:39.498 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2023-10-16T14:45:39.499 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 149007, 2023-10-16T14:45:39.499 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 30, 2023-10-16T14:45:39.500 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2023-10-16T14:45:39.500 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2023-10-16T14:45:39.500 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2023-10-16T14:45:39.500 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2023-10-16T14:45:39.501 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2023-10-16T14:45:39.501 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2023-10-16T14:45:39.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2023-10-16T14:45:39.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2023-10-16T14:45:39.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 0, 2023-10-16T14:45:39.503 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 0, 2023-10-16T14:45:39.503 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2023-10-16T14:45:39.503 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2023-10-16T14:45:39.503 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2023-10-16T14:45:39.504 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2023-10-16T14:45:39.504 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 928, 2023-10-16T14:45:39.504 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 2, 2023-10-16T14:45:39.504 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2023-10-16T14:45:39.505 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2023-10-16T14:45:39.505 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2023-10-16T14:45:39.505 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2023-10-16T14:45:39.505 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2023-10-16T14:45:39.506 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-10-16T14:45:39.506 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2023-10-16T14:45:39.506 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2023-10-16T14:45:39.507 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2023-10-16T14:45:39.507 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 239, 2023-10-16T14:45:39.507 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 444182, 2023-10-16T14:45:39.507 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2023-10-16T14:45:39.508 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 239, 2023-10-16T14:45:39.508 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2023-10-16T14:45:39.508 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2023-10-16T14:45:39.509 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 239, 2023-10-16T14:45:39.509 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 444182, 2023-10-16T14:45:39.509 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2023-10-16T14:45:39.510 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2023-10-16T14:45:39.510 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2023-10-16T14:45:39.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2023-10-16T14:45:39.512 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:39.512 INFO:teuthology.orchestra.run.smithi187.stdout: } 2023-10-16T14:45:39.513 INFO:teuthology.orchestra.run.smithi187.stdout:} 2023-10-16T14:45:39.904 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-10-16T14:45:39.905 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-10-16T14:45:39.905 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-10-16T14:45:39.905 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-16T14:45:40.624 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-16T14:45:40.625 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:45:40.625 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:45:40.626 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:40 smithi186 ceph-mon[134892]: pgmap v697: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:45:40.626 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:40 smithi178 ceph-mon[98495]: pgmap v697: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:45:40.627 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:40 smithi187 ceph-mon[135297]: pgmap v697: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-10-16T14:45:40.683 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:45:40.683 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:45:40.779 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:45:41.027 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 -- 172.21.15.178:0/793248902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0100ea0 msgr2=0x7ffbe01012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/793248902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0100ea0 0x7ffbe01012c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009990 tx=0x7ffbd0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 -- 172.21.15.178:0/793248902 shutdown_connections 2023-10-16T14:45:41.028 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/793248902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102a40 0x7ffbe0109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/793248902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0100ea0 0x7ffbe01012c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/793248902 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0102080 0x7ffbe0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 -- 172.21.15.178:0/793248902 >> 172.21.15.178:0/793248902 conn(0x7ffbe00fc420 msgr2=0x7ffbe00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:41.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 -- 172.21.15.178:0/793248902 shutdown_connections 2023-10-16T14:45:41.029 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.023+0000 7ffbe69ff700 1 -- 172.21.15.178:0/793248902 wait complete. 2023-10-16T14:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 Processor -- start 2023-10-16T14:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 -- start start 2023-10-16T14:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0100ea0 0x7ffbe0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102080 0x7ffbe01992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.032 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a08f0 con 0x7ffbe0102a40 2023-10-16T14:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a0a30 con 0x7ffbe0100ea0 2023-10-16T14:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe69ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01a0bc0 con 0x7ffbe0102080 2023-10-16T14:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.024+0000 7ffbe4f9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.033 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40746/0 (socket says 172.21.15.178:40746) 2023-10-16T14:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 -- 172.21.15.178:0/3480025419 learned_addr learned my addr 172.21.15.178:0/3480025419 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbdffff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0100ea0 0x7ffbe0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbdf7fe700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102080 0x7ffbe01992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 -- 172.21.15.178:0/3480025419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102080 msgr2=0x7ffbe01992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.034 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102080 0x7ffbe01992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 -- 172.21.15.178:0/3480025419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0100ea0 msgr2=0x7ffbe0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0100ea0 0x7ffbe0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 -- 172.21.15.178:0/3480025419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd00095f0 con 0x7ffbe0102a40 2023-10-16T14:45:41.035 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe4f9c700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800ef30 tx=0x7ffbd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8018070 con 0x7ffbe0102a40 2023-10-16T14:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe01a0e40 con 0x7ffbe0102a40 2023-10-16T14:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbd800f040 con 0x7ffbe0102a40 2023-10-16T14:45:41.036 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.025+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd800cd00 con 0x7ffbe0102a40 2023-10-16T14:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.026+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe01a1270 con 0x7ffbe0102a40 2023-10-16T14:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.026+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe010c760 con 0x7ffbe0102a40 2023-10-16T14:45:41.037 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.028+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7ffbd8003a00 con 0x7ffbe0102a40 2023-10-16T14:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.030+0000 7ffbdd7fa700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 0x7ffbc80a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.030+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ffbd80cd550 con 0x7ffbe0102a40 2023-10-16T14:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.030+0000 7ffbdffff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 0x7ffbc80a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.038 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.031+0000 7ffbdffff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 0x7ffbc80a39e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b2e0 tx=0x7ffbd0005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.039 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.033+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7ffbd8096870 con 0x7ffbe0102a40 2023-10-16T14:45:41.244 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.240+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbe01a1460 con 0x7ffbe0102a40 2023-10-16T14:45:41.245 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.242+0000 7ffbdd7fa700 1 -- 172.21.15.178:0/3480025419 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7ffbd8099e90 con 0x7ffbe0102a40 2023-10-16T14:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:41.246 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:45:41.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 msgr2=0x7ffbc80a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 0x7ffbc80a39e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000b2e0 tx=0x7ffbd0005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 msgr2=0x7ffbe01a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.249 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffbd800ef30 tx=0x7ffbd800c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 shutdown_connections 2023-10-16T14:45:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7ffbc80a1520 0x7ffbc80a39e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffbe0102a40 0x7ffbe01a0320 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7ffbe0100ea0 0x7ffbe0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.250 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 --2- 172.21.15.178:0/3480025419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbe0102080 0x7ffbe01992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 >> 172.21.15.178:0/3480025419 conn(0x7ffbe00fc420 msgr2=0x7ffbe00fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:41.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 shutdown_connections 2023-10-16T14:45:41.251 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.245+0000 7ffbe69ff700 1 -- 172.21.15.178:0/3480025419 wait complete. 2023-10-16T14:45:41.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T14:27:25.107672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T14:31:34.812918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T14:31:36.013811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-10-16T14:31:39.561433+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-16T14:45:41.269 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T14:45:41.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 -- 172.21.15.178:0/3574234978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc073910 msgr2=0x7f37dc073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.471 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 --2- 172.21.15.178:0/3574234978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc073910 0x7f37dc073d90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009990 tx=0x7f37c8009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 -- 172.21.15.178:0/3574234978 shutdown_connections 2023-10-16T14:45:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 --2- 172.21.15.178:0/3574234978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 --2- 172.21.15.178:0/3574234978 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc074ef0 0x7f37dc073340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 --2- 172.21.15.178:0/3574234978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc073910 0x7f37dc073d90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.472 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.467+0000 7f37e2dff700 1 -- 172.21.15.178:0/3574234978 >> 172.21.15.178:0/3574234978 conn(0x7f37dc0fc400 msgr2=0x7f37dc0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:41.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.468+0000 7f37e2dff700 1 -- 172.21.15.178:0/3574234978 shutdown_connections 2023-10-16T14:45:41.473 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.468+0000 7f37e2dff700 1 -- 172.21.15.178:0/3574234978 wait complete. 2023-10-16T14:45:41.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 Processor -- start 2023-10-16T14:45:41.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 -- start start 2023-10-16T14:45:41.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 0x7f37dc19d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.475 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1a4db0 con 0x7f37dc10d4a0 2023-10-16T14:45:41.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1a4ef0 con 0x7f37dc073910 2023-10-16T14:45:41.476 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e2dff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37dc1a5080 con 0x7f37dc074ef0 2023-10-16T14:45:41.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e139c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e0b9b700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 0x7f37dc19d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37dbfff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.477 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e139c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40762/0 (socket says 172.21.15.178:40762) 2023-10-16T14:45:41.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37e139c700 1 -- 172.21.15.178:0/1357776841 learned_addr learned my addr 172.21.15.178:0/1357776841 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:41.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.469+0000 7f37dbfff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:56924/0 (socket says 172.21.15.178:56924) 2023-10-16T14:45:41.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 -- 172.21.15.178:0/1357776841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 msgr2=0x7f37dc19d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.478 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 -- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 msgr2=0x7f37dc1a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 -- 172.21.15.178:0/1357776841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80095f0 con 0x7f37dc073910 2023-10-16T14:45:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e139c700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:41.479 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37dbfff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e0b9b700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 0x7f37dc19d230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f37d000e9f0 tx=0x7f37d000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000c9c0 con 0x7f37dc073910 2023-10-16T14:45:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37dc1a5360 con 0x7f37dc073910 2023-10-16T14:45:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37d0010430 con 0x7f37dc073910 2023-10-16T14:45:41.480 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.470+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0018680 con 0x7f37dc073910 2023-10-16T14:45:41.481 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.471+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37dc1a58c0 con 0x7f37dc073910 2023-10-16T14:45:41.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.472+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37dc102ff0 con 0x7f37dc073910 2023-10-16T14:45:41.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.474+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f37d0010860 con 0x7f37dc073910 2023-10-16T14:45:41.482 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.475+0000 7f37d9ffb700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 0x7f37cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.475+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f37d0014070 con 0x7f37dc073910 2023-10-16T14:45:41.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.475+0000 7f37dbfff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 0x7f37cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.476+0000 7f37dbfff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 0x7f37cc0a3940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b2e0 tx=0x7f37c8005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.483 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.477+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f37d0096640 con 0x7f37dc073910 2023-10-16T14:45:41.684 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.681+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37dc112ea0 con 0x7f37dc073910 2023-10-16T14:45:41.685 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.682+0000 7f37d9ffb700 1 -- 172.21.15.178:0/1357776841 <== mon.2 v2:172.21.15.186:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37d0026020 con 0x7f37dc073910 2023-10-16T14:45:41.685 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 1 2023-10-16T14:45:41.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 msgr2=0x7f37cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 0x7f37cc0a3940 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37c800b2e0 tx=0x7f37c8005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 msgr2=0x7f37dc19d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 0x7f37dc19d230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f37d000e9f0 tx=0x7f37d000ed00 comp rx=0 tx=0).stop 2023-10-16T14:45:41.688 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 shutdown_connections 2023-10-16T14:45:41.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f37cc0a1480 0x7f37cc0a3940 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37dc10d4a0 0x7f37dc1a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f37dc073910 0x7f37dc19d230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 --2- 172.21.15.178:0/1357776841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37dc074ef0 0x7f37dc19d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.689 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 >> 172.21.15.178:0/1357776841 conn(0x7f37dc0fc400 msgr2=0x7f37dc0fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:41.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 shutdown_connections 2023-10-16T14:45:41.690 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.684+0000 7f37e2dff700 1 -- 172.21.15.178:0/1357776841 wait complete. 2023-10-16T14:45:41.707 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T14:45:41.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:41 smithi178 ceph-mon[98495]: pgmap v698: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:45:41.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:41 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3480025419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:41 smithi186 ceph-mon[134892]: pgmap v698: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:45:41.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:41 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3480025419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:41.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.904+0000 7fba4e3ff700 1 -- 172.21.15.178:0/3414347380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 msgr2=0x7fba481012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.904+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/3414347380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba481012c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fba38009990 tx=0x7fba38009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 -- 172.21.15.178:0/3414347380 shutdown_connections 2023-10-16T14:45:41.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/3414347380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 0x7fba48109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/3414347380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba481012c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/3414347380 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 0x7fba48102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 -- 172.21.15.178:0/3414347380 >> 172.21.15.178:0/3414347380 conn(0x7fba480fc420 msgr2=0x7fba480fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 -- 172.21.15.178:0/3414347380 shutdown_connections 2023-10-16T14:45:41.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.905+0000 7fba4e3ff700 1 -- 172.21.15.178:0/3414347380 wait complete. 2023-10-16T14:45:41.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 Processor -- start 2023-10-16T14:45:41.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 -- start start 2023-10-16T14:45:41.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 0x7fba480732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 0x7fba480737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48073f80 con 0x7fba48102a40 2023-10-16T14:45:41.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480740f0 con 0x7fba48102080 2023-10-16T14:45:41.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4e3ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba481a0ce0 con 0x7fba48100ea0 2023-10-16T14:45:41.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba47fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba47fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:56954/0 (socket says 172.21.15.178:56954) 2023-10-16T14:45:41.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba47fff700 1 -- 172.21.15.178:0/1028016092 learned_addr learned my addr 172.21.15.178:0/1028016092 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:41.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba4c99c700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 0x7fba480737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.906+0000 7fba477fe700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 0x7fba480732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 -- 172.21.15.178:0/1028016092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 msgr2=0x7fba480732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 0x7fba480732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 -- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 msgr2=0x7fba480737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 0x7fba480737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 -- 172.21.15.178:0/1028016092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba380095f0 con 0x7fba48100ea0 2023-10-16T14:45:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba47fff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba38005720 tx=0x7fba38009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3801d070 con 0x7fba48100ea0 2023-10-16T14:45:41.916 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba38021410 con 0x7fba48100ea0 2023-10-16T14:45:41.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba481a0f60 con 0x7fba48100ea0 2023-10-16T14:45:41.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38017650 con 0x7fba48100ea0 2023-10-16T14:45:41.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.907+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481a12d0 con 0x7fba48100ea0 2023-10-16T14:45:41.917 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.908+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4810c760 con 0x7fba48100ea0 2023-10-16T14:45:41.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.910+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fba38021580 con 0x7fba48100ea0 2023-10-16T14:45:41.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.912+0000 7fba457fa700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 0x7fba300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:41.918 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.912+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fba380ce740 con 0x7fba48100ea0 2023-10-16T14:45:41.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.912+0000 7fba477fe700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 0x7fba300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:41.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.912+0000 7fba477fe700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 0x7fba300a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba48100bc0 tx=0x7fba3c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:41.919 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:41.914+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fba38097a60 con 0x7fba48100ea0 2023-10-16T14:45:41.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:41 smithi187 ceph-mon[135297]: pgmap v698: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-10-16T14:45:41.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:41 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3480025419' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:42.121 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.118+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba48107ed0 con 0x7fba48100ea0 2023-10-16T14:45:42.122 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.118+0000 7fba457fa700 1 -- 172.21.15.178:0/1028016092 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba3809b080 con 0x7fba48100ea0 2023-10-16T14:45:42.122 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:45:42.124 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 msgr2=0x7fba300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 0x7fba300a3940 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fba48100bc0 tx=0x7fba3c009260 comp rx=0 tx=0).stop 2023-10-16T14:45:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 msgr2=0x7fba48074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fba38005720 tx=0x7fba38009580 comp rx=0 tx=0).stop 2023-10-16T14:45:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 shutdown_connections 2023-10-16T14:45:42.125 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fba300a1480 0x7fba300a3940 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba48102a40 0x7fba480737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba48100ea0 0x7fba48074c60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 --2- 172.21.15.178:0/1028016092 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fba48102080 0x7fba480732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.126 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 >> 172.21.15.178:0/1028016092 conn(0x7fba480fc420 msgr2=0x7fba480fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 shutdown_connections 2023-10-16T14:45:42.127 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.121+0000 7fba4e3ff700 1 -- 172.21.15.178:0/1028016092 wait complete. 2023-10-16T14:45:42.143 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T14:45:42.344 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 -- 172.21.15.178:0/3712757461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c074ef0 msgr2=0x7f653c073340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 --2- 172.21.15.178:0/3712757461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c074ef0 0x7f653c073340 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f652c009990 tx=0x7f652c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 -- 172.21.15.178:0/3712757461 shutdown_connections 2023-10-16T14:45:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 --2- 172.21.15.178:0/3712757461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c108db0 0x7f653c10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.345 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 --2- 172.21.15.178:0/3712757461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c074ef0 0x7f653c073340 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 --2- 172.21.15.178:0/3712757461 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 -- 172.21.15.178:0/3712757461 >> 172.21.15.178:0/3712757461 conn(0x7f653c0fc420 msgr2=0x7f653c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 -- 172.21.15.178:0/3712757461 shutdown_connections 2023-10-16T14:45:42.346 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.341+0000 7f65415ff700 1 -- 172.21.15.178:0/3712757461 wait complete. 2023-10-16T14:45:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 Processor -- start 2023-10-16T14:45:42.348 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 -- start start 2023-10-16T14:45:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c108db0 0x7f653c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1a09a0 con 0x7f653c074ef0 2023-10-16T14:45:42.349 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1a0ae0 con 0x7f653c073910 2023-10-16T14:45:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.342+0000 7f65415ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c1a0c70 con 0x7f653c108db0 2023-10-16T14:45:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653affd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40800/0 (socket says 172.21.15.178:40800) 2023-10-16T14:45:42.350 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 -- 172.21.15.178:0/1475065826 learned_addr learned my addr 172.21.15.178:0/1475065826 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653b7fe700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c108db0 0x7f653c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 -- 172.21.15.178:0/1475065826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c108db0 msgr2=0x7f653c1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c108db0 0x7f653c1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 -- 172.21.15.178:0/1475065826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 msgr2=0x7f653c198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.351 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 -- 172.21.15.178:0/1475065826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c0095f0 con 0x7f653c074ef0 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653affd700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.343+0000 7f653a7fc700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f653000e970 tx=0x7f653000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.344+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530009790 con 0x7f653c074ef0 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.344+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f653000c7e0 con 0x7f653c074ef0 2023-10-16T14:45:42.352 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.344+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c1a0f50 con 0x7f653c074ef0 2023-10-16T14:45:42.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.344+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530003c70 con 0x7f653c074ef0 2023-10-16T14:45:42.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.344+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1a13d0 con 0x7f653c074ef0 2023-10-16T14:45:42.353 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.345+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c107460 con 0x7f653c074ef0 2023-10-16T14:45:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.347+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f653000cc50 con 0x7f653c074ef0 2023-10-16T14:45:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.348+0000 7f6523fff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 0x7f65240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.348+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6530014070 con 0x7f653c074ef0 2023-10-16T14:45:42.354 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.348+0000 7f653affd700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 0x7f65240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.349+0000 7f653affd700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 0x7f65240a3940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b2e0 tx=0x7f652c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:42.355 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.352+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f65300964f0 con 0x7f653c074ef0 2023-10-16T14:45:42.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.553+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f653c02cf70 con 0x7f653c074ef0 2023-10-16T14:45:42.557 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.554+0000 7f6523fff700 1 -- 172.21.15.178:0/1475065826 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6530003dd0 con 0x7f653c074ef0 2023-10-16T14:45:42.558 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 64 2023-10-16T14:45:42.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.556+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 msgr2=0x7f65240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.556+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 0x7f65240a3940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b2e0 tx=0x7f652c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.560 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.556+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 msgr2=0x7f653c199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.556+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f653000e970 tx=0x7f653000ec80 comp rx=0 tx=0).stop 2023-10-16T14:45:42.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 shutdown_connections 2023-10-16T14:45:42.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f65240a1480 0x7f65240a3940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f653c108db0 0x7f653c1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.561 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f653c073910 0x7f653c198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 --2- 172.21.15.178:0/1475065826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f653c074ef0 0x7f653c199360 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 >> 172.21.15.178:0/1475065826 conn(0x7f653c0fc420 msgr2=0x7f653c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:42.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 shutdown_connections 2023-10-16T14:45:42.562 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.557+0000 7f65415ff700 1 -- 172.21.15.178:0/1475065826 wait complete. 2023-10-16T14:45:42.579 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-10-16T14:45:42.779 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.775+0000 7f98acdff700 1 -- 172.21.15.178:0/1049292363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 msgr2=0x7f98a81024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.775+0000 7f98acdff700 1 --2- 172.21.15.178:0/1049292363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a81024e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f989c0098f0 tx=0x7f989c009c00 comp rx=0 tx=0).stop 2023-10-16T14:45:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 -- 172.21.15.178:0/1049292363 shutdown_connections 2023-10-16T14:45:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 --2- 172.21.15.178:0/1049292363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8102a20 0x7f98a8109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.780 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 --2- 172.21.15.178:0/1049292363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8100e80 0x7f98a81012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 --2- 172.21.15.178:0/1049292363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a81024e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 -- 172.21.15.178:0/1049292363 >> 172.21.15.178:0/1049292363 conn(0x7f98a80fc400 msgr2=0x7f98a80fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 -- 172.21.15.178:0/1049292363 shutdown_connections 2023-10-16T14:45:42.781 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.776+0000 7f98acdff700 1 -- 172.21.15.178:0/1049292363 wait complete. 2023-10-16T14:45:42.783 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 Processor -- start 2023-10-16T14:45:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 -- start start 2023-10-16T14:45:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8100e80 0x7f98a8198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.784 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8102a20 0x7f98a81a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81a0a40 con 0x7f98a8102060 2023-10-16T14:45:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81a0b80 con 0x7f98a8102a20 2023-10-16T14:45:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98acdff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a81a0d10 con 0x7f98a8100e80 2023-10-16T14:45:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98a5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.785 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98a5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40818/0 (socket says 172.21.15.178:40818) 2023-10-16T14:45:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98a5d9b700 1 -- 172.21.15.178:0/522646310 learned_addr learned my addr 172.21.15.178:0/522646310 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.777+0000 7f98a659c700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8100e80 0x7f98a8198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a6d9d700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8102a20 0x7f98a81a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 -- 172.21.15.178:0/522646310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8100e80 msgr2=0x7f98a8198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.786 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8100e80 0x7f98a8198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 -- 172.21.15.178:0/522646310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8102a20 msgr2=0x7f98a81a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8102a20 0x7f98a81a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.787 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 -- 172.21.15.178:0/522646310 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009520 con 0x7f98a8102060 2023-10-16T14:45:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98a5d9b700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f989c009fd0 tx=0x7f989c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c01c070 con 0x7f98a8102060 2023-10-16T14:45:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f989c00f7d0 con 0x7f98a8102060 2023-10-16T14:45:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a81a0f90 con 0x7f98a8102060 2023-10-16T14:45:42.788 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.778+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c003a40 con 0x7f98a8102060 2023-10-16T14:45:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.779+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a81a1300 con 0x7f98a8102060 2023-10-16T14:45:42.789 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.780+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a810c7b0 con 0x7f98a8102060 2023-10-16T14:45:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.782+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f989c020030 con 0x7f98a8102060 2023-10-16T14:45:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.783+0000 7f98977fe700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 0x7f98900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:42.790 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.783+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f989c0cdfe0 con 0x7f98a8102060 2023-10-16T14:45:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.783+0000 7f98a659c700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 0x7f98900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.784+0000 7f98a659c700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 0x7f98900a3940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98a8101bd0 tx=0x7f9898009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:42.791 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.787+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f989c097360 con 0x7f98a8102060 2023-10-16T14:45:42.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:42 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1357776841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:42 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1028016092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:42 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1357776841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:42 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1028016092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:42 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1357776841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:42 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1028016092' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-10-16T14:45:42.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.989+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f98a81a14f0 con 0x7f98a8102060 2023-10-16T14:45:42.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.990+0000 7f98977fe700 1 -- 172.21.15.178:0/522646310 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f989c09a980 con 0x7f98a8102060 2023-10-16T14:45:42.994 INFO:teuthology.orchestra.run.smithi178.stdout:pg_num: 32 2023-10-16T14:45:42.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 msgr2=0x7f98900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 0x7f98900a3940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f98a8101bd0 tx=0x7f9898009260 comp rx=0 tx=0).stop 2023-10-16T14:45:42.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 msgr2=0x7f98a8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f989c009fd0 tx=0x7f989c00b790 comp rx=0 tx=0).stop 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 shutdown_connections 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f98900a1480 0x7f98900a3940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f98a8102a20 0x7f98a81a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98a8100e80 0x7f98a8198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.997 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 --2- 172.21.15.178:0/522646310 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a8102060 0x7f98a8199400 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:42.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.992+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 >> 172.21.15.178:0/522646310 conn(0x7f98a80fc400 msgr2=0x7f98a80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:42.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.993+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 shutdown_connections 2023-10-16T14:45:42.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:42.993+0000 7f98acdff700 1 -- 172.21.15.178:0/522646310 wait complete. 2023-10-16T14:45:43.016 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T14:45:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/1000254742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 msgr2=0x7f8b58109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/1000254742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b58109360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000b0a0 tx=0x7f8b5000b3b0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/1000254742 shutdown_connections 2023-10-16T14:45:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/1000254742 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b58109360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/1000254742 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58100ea0 0x7f8b581012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/1000254742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58102080 0x7f8b58102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.210+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/1000254742 >> 172.21.15.178:0/1000254742 conn(0x7f8b580fc420 msgr2=0x7f8b580fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:43.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.211+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/1000254742 shutdown_connections 2023-10-16T14:45:43.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.211+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/1000254742 wait complete. 2023-10-16T14:45:43.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.212+0000 7f8b5e9ff700 1 Processor -- start 2023-10-16T14:45:43.218 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.212+0000 7f8b5e9ff700 1 -- start start 2023-10-16T14:45:43.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:43.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58102080 0x7f8b580732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:43.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b580737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:43.219 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b58073f80 con 0x7f8b58102a40 2023-10-16T14:45:43.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580740f0 con 0x7f8b58102080 2023-10-16T14:45:43.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5e9ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b581a0d00 con 0x7f8b58100ea0 2023-10-16T14:45:43.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:43.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5cf9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b580737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:43.220 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:57016/0 (socket says 172.21.15.178:57016) 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 -- 172.21.15.178:0/680285410 learned_addr learned my addr 172.21.15.178:0/680285410 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b577fe700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58102080 0x7f8b580732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 -- 172.21.15.178:0/680285410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58102080 msgr2=0x7f8b580732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58102080 0x7f8b580732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 -- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 msgr2=0x7f8b580737e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:43.221 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b580737e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b57fff700 1 -- 172.21.15.178:0/680285410 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50009d00 con 0x7f8b58100ea0 2023-10-16T14:45:43.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.213+0000 7f8b5cf9c700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b580737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:43.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b57fff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800e970 tx=0x7f8b4800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:43.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48009790 con 0x7f8b58100ea0 2023-10-16T14:45:43.222 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b4800eec0 con 0x7f8b58100ea0 2023-10-16T14:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b581a0fe0 con 0x7f8b58100ea0 2023-10-16T14:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48003cf0 con 0x7f8b58100ea0 2023-10-16T14:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.214+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b581a13b0 con 0x7f8b58100ea0 2023-10-16T14:45:43.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.215+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b581083b0 con 0x7f8b58100ea0 2023-10-16T14:45:43.224 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.217+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f8b4800ccd0 con 0x7f8b58100ea0 2023-10-16T14:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.218+0000 7f8b557fa700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 0x7f8b400a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.218+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8b48014070 con 0x7f8b58100ea0 2023-10-16T14:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.218+0000 7f8b577fe700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 0x7f8b400a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.219+0000 7f8b577fe700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 0x7f8b400a3990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8b58100bc0 tx=0x7f8b4c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:43.225 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.220+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f8b48096730 con 0x7f8b58100ea0 2023-10-16T14:45:43.446 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.442+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b5810ca40 con 0x7f8b58100ea0 2023-10-16T14:45:43.447 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.443+0000 7f8b557fa700 1 -- 172.21.15.178:0/680285410 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) v1 ==== 81+0+602 (secure 0 0 0) 0x7f8b5810ca40 con 0x7f8b58100ea0 2023-10-16T14:45:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:45:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2364174490 2023-10-17T14:25:06.501803+0000 2023-10-16T14:45:43.447 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/2438067538 2023-10-17T14:25:06.501803+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6801/3157107959 2023-10-17T14:25:06.501803+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3729267496 2023-10-17T14:24:39.404169+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3093526104 2023-10-17T14:24:39.404169+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/1552969994 2023-10-17T14:25:06.501803+0000 2023-10-16T14:45:43.448 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:6800/2960529415 2023-10-17T14:24:39.404169+0000 2023-10-16T14:45:43.449 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.15.178:0/3372273467 2023-10-17T14:24:39.404169+0000 2023-10-16T14:45:43.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.445+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 msgr2=0x7f8b400a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:43.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.445+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 0x7f8b400a3990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8b58100bc0 tx=0x7f8b4c009260 comp rx=0 tx=0).stop 2023-10-16T14:45:43.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.445+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 msgr2=0x7f8b58074c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:43.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.445+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8b4800e970 tx=0x7f8b4800ec80 comp rx=0 tx=0).stop 2023-10-16T14:45:43.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.445+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 shutdown_connections 2023-10-16T14:45:43.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f8b400a14d0 0x7f8b400a3990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b58102a40 0x7f8b580737e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b58100ea0 0x7f8b58074c60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 --2- 172.21.15.178:0/680285410 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f8b58102080 0x7f8b580732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:43.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 >> 172.21.15.178:0/680285410 conn(0x7f8b580fc420 msgr2=0x7f8b581054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:43.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 shutdown_connections 2023-10-16T14:45:43.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:43.446+0000 7f8b5e9ff700 1 -- 172.21.15.178:0/680285410 wait complete. 2023-10-16T14:45:43.452 INFO:teuthology.orchestra.run.smithi178.stderr:listed 10 entries 2023-10-16T14:45:43.468 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-16T14:45:43.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:43.469 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-16T14:45:43.567 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:45:43.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:43.568 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T14:45:43.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:43.669 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-10-16T14:45:43.766 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T14:45:43.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:43.767 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T14:45:43.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:43 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1475065826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:43 smithi178 ceph-mon[98495]: pgmap v699: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2023-10-16T14:45:43.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:43 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/522646310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.803 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:43 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/680285410' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:45:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:43 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1475065826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:43 smithi186 ceph-mon[134892]: pgmap v699: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2023-10-16T14:45:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:43 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/522646310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.826 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:43 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/680285410' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:45:43.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:43.878 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-10-16T14:45:43.924 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:43 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1475065826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:43 smithi187 ceph-mon[135297]: pgmap v699: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 5 op/s 2023-10-16T14:45:43.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:43 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/522646310' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-10-16T14:45:43.925 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:43 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/680285410' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-10-16T14:45:44.027 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-16T14:45:44.027 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-10-16T14:45:44.027 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T14:45:44.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16348244k,nr_inodes=4087061,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275604k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/c349f995cf4d8a1f4f026e9bd0be70033f397a264cce7788f74d0976bc19b854/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/c349f995cf4d8a1f4f026e9bd0be70033f397a264cce7788f74d0976bc19b854/diff,workdir=/var/lib/containers/storage/overlay/c349f995cf4d8a1f4f026e9bd0be70033f397a264cce7788f74d0976bc19b854/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cc66a8e9bf3f4f8566710fcc0d4eaa140067152f9fa41ea1e3440533e2771ac8/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/cc66a8e9bf3f4f8566710fcc0d4eaa140067152f9fa41ea1e3440533e2771ac8/diff,workdir=/var/lib/containers/storage/overlay/cc66a8e9bf3f4f8566710fcc0d4eaa140067152f9fa41ea1e3440533e2771ac8/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275604k,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,relatime 0 0 overlay /var/lib/containers/storage/overlay/201374a8fede183aba3f846de2fcd1b62826df48458b3f7290d86ce79a733fb0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/201374a8fede183aba3f846de2fcd1b62826df48458b3f7290d86ce79a733fb0/diff,workdir=/var/lib/containers/storage/overlay/201374a8fede183aba3f846de2fcd1b62826df48458b3f7290d86ce79a733fb0/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a8119eca76a8d78ab2fceb3c5c4b2b729b9ae2cff1fd77afa8755486e681eef1/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/a8119eca76a8d78ab2fceb3c5c4b2b729b9ae2cff1fd77afa8755486e681eef1/diff,workdir=/var/lib/containers/storage/overlay/a8119eca76a8d78ab2fceb3c5c4b2b729b9ae2cff1fd77afa8755486e681eef1/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c15b4e19cbd9162c444ce11ef9964e338f20cc38a27d0173891cfa8bbdc9d25d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/c15b4e19cbd9162c444ce11ef9964e338f20cc38a27d0173891cfa8bbdc9d25d/diff,workdir=/var/lib/containers/storage/overlay/c15b4e19cbd9162c444ce11ef9964e338f20cc38a27d0173891cfa8bbdc9d25d/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c43aac3828067c8e145ff6b365a92c8f71815e7d091d26bf62672d994e57b94c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/c43aac3828067c8e145ff6b365a92c8f71815e7d091d26bf62672d994e57b94c/diff,workdir=/var/lib/containers/storage/overlay/c43aac3828067c8e145ff6b365a92c8f71815e7d091d26bf62672d994e57b94c/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/26dc8dae4056abed73b308f2975f639e76a482e2bc95eb1adfbfd2e72983b075/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/26dc8dae4056abed73b308f2975f639e76a482e2bc95eb1adfbfd2e72983b075/diff,workdir=/var/lib/containers/storage/overlay/26dc8dae4056abed73b308f2975f639e76a482e2bc95eb1adfbfd2e72983b075/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/1b0665d5469466142301d1a9d39fac95acd758c32cf83378420665bf90543ceb/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/1b0665d5469466142301d1a9d39fac95acd758c32cf83378420665bf90543ceb/diff,workdir=/var/lib/containers/storage/overlay/1b0665d5469466142301d1a9d39fac95acd758c32cf83378420665bf90543ceb/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fd215243147bbebc87b425b857be4bddace79b4dab1e5b7dd13e071ecb8ac95b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/fd215243147bbebc87b425b857be4bddace79b4dab1e5b7dd13e071ecb8ac95b/diff,workdir=/var/lib/containers/storage/overlay/fd215243147bbebc87b425b857be4bddace79b4dab1e5b7dd13e071ecb8ac95b/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/6892876b869dfc27dbf949837a8624554834e4b099986fad3a8782a58d6945bd/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/LG2HCUT5FDEJ7BM6SJLAZ36ILR:/var/lib/containers/storage/overlay/l/X75BAPBQFB6LWDB5S5J4EXLOB6,upperdir=/var/lib/containers/storage/overlay/6892876b869dfc27dbf949837a8624554834e4b099986fad3a8782a58d6945bd/diff,workdir=/var/lib/containers/storage/overlay/6892876b869dfc27dbf949837a8624554834e4b099986fad3a8782a58d6945bd/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 2023-10-16T14:45:44.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:44.076 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-10-16T14:45:44.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T14:45:44.160 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-10-16T14:45:44.273 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2023-10-16T14:45:44.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:45:44.277 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-10-16T14:45:44.287 INFO:tasks.cephadm:Teardown begin 2023-10-16T14:45:44.287 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T14:45:44.287 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:45:44.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1538088347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac102080 msgr2=0x7f3fac102500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.487 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1538088347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac102080 0x7f3fac102500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009990 tx=0x7f3fa0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1538088347 shutdown_connections 2023-10-16T14:45:44.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1538088347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1538088347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac100ea0 0x7f3fac1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.488 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1538088347 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac102080 0x7f3fac102500 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1538088347 >> 172.21.15.178:0/1538088347 conn(0x7f3fac0fc420 msgr2=0x7f3fac0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:44.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1538088347 shutdown_connections 2023-10-16T14:45:44.489 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.483+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1538088347 wait complete. 2023-10-16T14:45:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.484+0000 7f3fb17ff700 1 Processor -- start 2023-10-16T14:45:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.484+0000 7f3fb17ff700 1 -- start start 2023-10-16T14:45:44.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 0x7f3fac196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac102080 0x7f3fac197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19e6d0 con 0x7f3fac102a40 2023-10-16T14:45:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19e810 con 0x7f3fac100ea0 2023-10-16T14:45:44.492 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fb17ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac19e9a0 con 0x7f3fac102080 2023-10-16T14:45:44.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 0x7f3fac196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 0x7f3fac196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:59318/0 (socket says 172.21.15.178:59318) 2023-10-16T14:45:44.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40852/0 (socket says 172.21.15.178:40852) 2023-10-16T14:45:44.493 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3faaffd700 1 -- 172.21.15.178:0/1994450103 learned_addr learned my addr 172.21.15.178:0/1994450103 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:44.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3faa7fc700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac102080 0x7f3fac197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 -- 172.21.15.178:0/1994450103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac102080 msgr2=0x7f3fac197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac102080 0x7f3fac197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 -- 172.21.15.178:0/1994450103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 msgr2=0x7f3fac196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 0x7f3fac196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.485+0000 7f3fab7fe700 1 -- 172.21.15.178:0/1994450103 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00095f0 con 0x7f3fac102a40 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3fab7fe700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400efc0 tx=0x7f3fa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4018070 con 0x7f3fac102a40 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa400f040 con 0x7f3fac102a40 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac19ec20 con 0x7f3fac102a40 2023-10-16T14:45:44.495 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa400cd40 con 0x7f3fac102a40 2023-10-16T14:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.486+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac19f130 con 0x7f3fac102a40 2023-10-16T14:45:44.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.487+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac107db0 con 0x7f3fac102a40 2023-10-16T14:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.489+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3fa4003a00 con 0x7f3fac102a40 2023-10-16T14:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.491+0000 7f3f93fff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 0x7f3f940a3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.491+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3fa40ccbf0 con 0x7f3fac102a40 2023-10-16T14:45:44.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.491+0000 7f3faaffd700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 0x7f3f940a3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.491+0000 7f3faaffd700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 0x7f3f940a3a60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005950 tx=0x7f3f9c009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:44.498 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.495+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3fa4095f10 con 0x7f3fac102a40 2023-10-16T14:45:44.742 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.739+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fac10c760 con 0x7f3fac102a40 2023-10-16T14:45:44.743 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.739+0000 7f3f93fff700 1 -- 172.21.15.178:0/1994450103 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f3fa401b320 con 0x7f3fac102a40 2023-10-16T14:45:44.743 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:44.745 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:44:25.377452+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":128,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:45:44.746 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 msgr2=0x7f3f940a3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 0x7f3f940a3a60 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c005950 tx=0x7f3f9c009260 comp rx=0 tx=0).stop 2023-10-16T14:45:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 msgr2=0x7f3fac19e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400efc0 tx=0x7f3fa400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 shutdown_connections 2023-10-16T14:45:44.747 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f3f940a15a0 0x7f3f940a3a60 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fac102a40 0x7f3fac19e100 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3fac100ea0 0x7f3fac196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 --2- 172.21.15.178:0/1994450103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fac102080 0x7f3fac197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.748 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 >> 172.21.15.178:0/1994450103 conn(0x7f3fac0fc420 msgr2=0x7f3fac0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:44.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 shutdown_connections 2023-10-16T14:45:44.749 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.742+0000 7f3fb17ff700 1 -- 172.21.15.178:0/1994450103 wait complete. 2023-10-16T14:45:44.749 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 40 2023-10-16T14:45:44.765 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T14:45:44.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 -- 172.21.15.178:0/3042411027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 msgr2=0x7f32781012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 --2- 172.21.15.178:0/3042411027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f32781012d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3274009990 tx=0x7f3274009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 -- 172.21.15.178:0/3042411027 shutdown_connections 2023-10-16T14:45:44.964 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 --2- 172.21.15.178:0/3042411027 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102a50 0x7f3278109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 --2- 172.21.15.178:0/3042411027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f32781012d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 --2- 172.21.15.178:0/3042411027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102090 0x7f3278102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 -- 172.21.15.178:0/3042411027 >> 172.21.15.178:0/3042411027 conn(0x7f32780fc430 msgr2=0x7f32780fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:44.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 -- 172.21.15.178:0/3042411027 shutdown_connections 2023-10-16T14:45:44.965 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.960+0000 7f327f7ff700 1 -- 172.21.15.178:0/3042411027 wait complete. 2023-10-16T14:45:44.967 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.961+0000 7f327f7ff700 1 Processor -- start 2023-10-16T14:45:44.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.961+0000 7f327f7ff700 1 -- start start 2023-10-16T14:45:44.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 0x7f32781970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.968 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102a50 0x7f327819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327819e6e0 con 0x7f3278100eb0 2023-10-16T14:45:44.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327819e820 con 0x7f3278102090 2023-10-16T14:45:44.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327f7ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327819e9b0 con 0x7f3278102a50 2023-10-16T14:45:44.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.969 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40868/0 (socket says 172.21.15.178:40868) 2023-10-16T14:45:44.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 -- 172.21.15.178:0/2135454286 learned_addr learned my addr 172.21.15.178:0/2135454286 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:44.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327dd9c700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102a50 0x7f327819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327cd9a700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 0x7f32781970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 -- 172.21.15.178:0/2135454286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102a50 msgr2=0x7f327819e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.970 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102a50 0x7f327819e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 -- 172.21.15.178:0/2135454286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 msgr2=0x7f32781970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 0x7f32781970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327d59b700 1 -- 172.21.15.178:0/2135454286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740095f0 con 0x7f3278100eb0 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.962+0000 7f327cd9a700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 0x7f32781970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f327d59b700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3274005720 tx=0x7f3274009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:44.971 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327401d070 con 0x7f3278100eb0 2023-10-16T14:45:44.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3274021410 con 0x7f3278100eb0 2023-10-16T14:45:44.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327819ec30 con 0x7f3278100eb0 2023-10-16T14:45:44.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274017650 con 0x7f3278100eb0 2023-10-16T14:45:44.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.963+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327819f050 con 0x7f3278100eb0 2023-10-16T14:45:44.972 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.964+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3278107dc0 con 0x7f3278100eb0 2023-10-16T14:45:44.973 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.966+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f3274021580 con 0x7f3278100eb0 2023-10-16T14:45:44.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.969+0000 7f326e7fc700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 0x7f32640a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:44.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.969+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f32740ce270 con 0x7f3278100eb0 2023-10-16T14:45:44.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.969+0000 7f327cd9a700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 0x7f32640a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:44.974 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.969+0000 7f327cd9a700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 0x7f32640a39e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3278100bd0 tx=0x7f3268009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:44.975 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:44.972+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f3274097590 con 0x7f3278100eb0 2023-10-16T14:45:45.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:44 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1994450103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:45.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:44 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1994450103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:45.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:44 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1994450103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:45.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.223+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f327802d090 con 0x7f3278100eb0 2023-10-16T14:45:45.227 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.223+0000 7f326e7fc700 1 -- 172.21.15.178:0/2135454286 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+11350 (secure 0 0 0) 0x7f3274095e60 con 0x7f3278100eb0 2023-10-16T14:45:45.227 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:45.227 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14475,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1662555012","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6834","nonce":1662555012},{"type":"v1","addr":"172.21.15.178:6835","nonce":1662555012}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14535,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6839/519971243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6838","nonce":519971243},{"type":"v1","addr":"172.21.15.178:6839","nonce":519971243}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14547,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6841/2221064860","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6840","nonce":2221064860},{"type":"v1","addr":"172.21.15.178:6841","nonce":2221064860}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24466,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1854237656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1854237656},{"type":"v1","addr":"172.21.15.187:6833","nonce":1854237656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24479,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.186:6833/3571012275","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6832","nonce":3571012275},{"type":"v1","addr":"172.21.15.186:6833","nonce":3571012275}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24503,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6837/3662037973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6836","nonce":3662037973},{"type":"v1","addr":"172.21.15.178:6837","nonce":3662037973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-10-16T14:31:36.554542+0000","modified":"2023-10-16T14:44:25.377452+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24593,"mds_1":24550,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24515":{"gid":24515,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.186:6835/1810043383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6834","nonce":1810043383},{"type":"v1","addr":"172.21.15.186:6835","nonce":1810043383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24520":{"gid":24520,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/2797759968","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":2797759968},{"type":"v1","addr":"172.21.15.187:6835","nonce":2797759968}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24532":{"gid":24532,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6837/4191845467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":4191845467},{"type":"v1","addr":"172.21.15.187:6837","nonce":4191845467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.186:6837/1524095342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6836","nonce":1524095342},{"type":"v1","addr":"172.21.15.186:6837","nonce":1524095342}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24550":{"gid":24550,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":128,"addr":"172.21.15.187:6839/397706812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":397706812},{"type":"v1","addr":"172.21.15.187:6839","nonce":397706812}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24593":{"gid":24593,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.186:6839/2475230127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6838","nonce":2475230127},{"type":"v1","addr":"172.21.15.186:6839","nonce":2475230127}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T14:45:45.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 msgr2=0x7f32640a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 0x7f32640a39e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3278100bd0 tx=0x7f3268009260 comp rx=0 tx=0).stop 2023-10-16T14:45:45.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 msgr2=0x7f3278196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3274005720 tx=0x7f3274009580 comp rx=0 tx=0).stop 2023-10-16T14:45:45.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 shutdown_connections 2023-10-16T14:45:45.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f32640a1520 0x7f32640a39e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3278102a50 0x7f327819e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.231 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3278100eb0 0x7f3278196b60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 --2- 172.21.15.178:0/2135454286 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f3278102090 0x7f32781970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 >> 172.21.15.178:0/2135454286 conn(0x7f32780fc430 msgr2=0x7f32780fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:45.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 shutdown_connections 2023-10-16T14:45:45.232 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.227+0000 7f327f7ff700 1 -- 172.21.15.178:0/2135454286 wait complete. 2023-10-16T14:45:45.233 INFO:teuthology.orchestra.run.smithi178.stderr:dumped fsmap epoch 40 2023-10-16T14:45:45.250 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T14:45:45.449 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 -- 172.21.15.178:0/924945881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c100ea0 msgr2=0x7fae8c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 --2- 172.21.15.178:0/924945881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c100ea0 0x7fae8c1012c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009990 tx=0x7fae7c009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 -- 172.21.15.178:0/924945881 shutdown_connections 2023-10-16T14:45:45.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 --2- 172.21.15.178:0/924945881 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102a40 0x7fae8c109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.450 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 --2- 172.21.15.178:0/924945881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c100ea0 0x7fae8c1012c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 --2- 172.21.15.178:0/924945881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c102080 0x7fae8c102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 -- 172.21.15.178:0/924945881 >> 172.21.15.178:0/924945881 conn(0x7fae8c0fc420 msgr2=0x7fae8c0fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:45.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 -- 172.21.15.178:0/924945881 shutdown_connections 2023-10-16T14:45:45.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.446+0000 7fae919ff700 1 -- 172.21.15.178:0/924945881 wait complete. 2023-10-16T14:45:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 Processor -- start 2023-10-16T14:45:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 -- start start 2023-10-16T14:45:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c100ea0 0x7fae8c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 0x7fae8c1992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1a08f0 con 0x7fae8c102a40 2023-10-16T14:45:45.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1a0a60 con 0x7fae8c102080 2023-10-16T14:45:45.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.447+0000 7fae919ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c1a0c20 con 0x7fae8c100ea0 2023-10-16T14:45:45.454 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 0x7fae8c1992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c100ea0 0x7fae8c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40882/0 (socket says 172.21.15.178:40882) 2023-10-16T14:45:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 0x7fae8c1992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.186:3300/0 says I am v2:172.21.15.178:59354/0 (socket says 172.21.15.178:59354) 2023-10-16T14:45:45.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 -- 172.21.15.178:0/4063773363 learned_addr learned my addr 172.21.15.178:0/4063773363 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 -- 172.21.15.178:0/4063773363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c100ea0 msgr2=0x7fae8c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c100ea0 0x7fae8c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 -- 172.21.15.178:0/4063773363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 msgr2=0x7fae8c1992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 0x7fae8c1992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 -- 172.21.15.178:0/4063773363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0095f0 con 0x7fae8c102a40 2023-10-16T14:45:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae8b7fe700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ef30 tx=0x7fae8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae84018070 con 0x7fae8c102a40 2023-10-16T14:45:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae8400f040 con 0x7fae8c102a40 2023-10-16T14:45:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c1a0ea0 con 0x7fae8c102a40 2023-10-16T14:45:45.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.448+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8400cd00 con 0x7fae8c102a40 2023-10-16T14:45:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.449+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c1a12c0 con 0x7fae8c102a40 2023-10-16T14:45:45.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.449+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c10c760 con 0x7fae8c102a40 2023-10-16T14:45:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.451+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fae84003a00 con 0x7fae8c102a40 2023-10-16T14:45:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.454+0000 7fae909fd700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 0x7fae740a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.454+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fae840ccc10 con 0x7fae8c102a40 2023-10-16T14:45:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.454+0000 7fae8affd700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 0x7fae740a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.454+0000 7fae8affd700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 0x7fae740a39e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000c00 tx=0x7fae7c005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:45.460 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.457+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fae84095f30 con 0x7fae8c102a40 2023-10-16T14:45:45.662 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.659+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae8c1a14b0 con 0x7fae8c102a40 2023-10-16T14:45:45.664 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.662+0000 7fae909fd700 1 -- 172.21.15.178:0/4063773363 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+24688 (secure 0 0 0) 0x7fae84099550 con 0x7fae8c102a40 2023-10-16T14:45:45.665 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-10-16T14:45:45.665 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":91,"fsid":"8c31bde4-6c2f-11ee-8db6-212e2dc638e7","created":"2023-10-16T14:24:07.673050+0000","modified":"2023-10-16T14:33:08.029307+0000","last_up_change":"2023-10-16T14:30:48.834869+0000","last_in_change":"2023-10-16T14:30:30.632389+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T14:27:25.107672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T14:31:34.812918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T14:31:36.013811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-10-16T14:31:39.561433+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9a72b3ff-3836-4ade-969c-531e6616f47e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6803","nonce":1189154044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6805","nonce":1189154044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6809","nonce":1189154044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1189154044},{"type":"v1","addr":"172.21.15.178:6807","nonce":1189154044}]},"public_addr":"172.21.15.178:6803/1189154044","cluster_addr":"172.21.15.178:6805/1189154044","heartbeat_back_addr":"172.21.15.178:6809/1189154044","heartbeat_front_addr":"172.21.15.178:6807/1189154044","state":["exists","up"]},{"osd":1,"uuid":"0811af9a-0421-41e8-be52-d60c284536d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6801","nonce":4108518172}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6803","nonce":4108518172}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6807","nonce":4108518172}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":4108518172},{"type":"v1","addr":"172.21.15.186:6805","nonce":4108518172}]},"public_addr":"172.21.15.186:6801/4108518172","cluster_addr":"172.21.15.186:6803/4108518172","heartbeat_back_addr":"172.21.15.186:6807/4108518172","heartbeat_front_addr":"172.21.15.186:6805/4108518172","state":["exists","up"]},{"osd":2,"uuid":"f3786153-58c5-4fec-a44b-d3e5b81e9748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6801","nonce":2818376874}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6803","nonce":2818376874}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6807","nonce":2818376874}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2818376874},{"type":"v1","addr":"172.21.15.187:6805","nonce":2818376874}]},"public_addr":"172.21.15.187:6801/2818376874","cluster_addr":"172.21.15.187:6803/2818376874","heartbeat_back_addr":"172.21.15.187:6807/2818376874","heartbeat_front_addr":"172.21.15.187:6805/2818376874","state":["exists","up"]},{"osd":3,"uuid":"2b6e4a7c-b16b-4da0-a854-7d054ac16422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6811","nonce":4190845983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6813","nonce":4190845983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6817","nonce":4190845983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":4190845983},{"type":"v1","addr":"172.21.15.178:6815","nonce":4190845983}]},"public_addr":"172.21.15.178:6811/4190845983","cluster_addr":"172.21.15.178:6813/4190845983","heartbeat_back_addr":"172.21.15.178:6817/4190845983","heartbeat_front_addr":"172.21.15.178:6815/4190845983","state":["exists","up"]},{"osd":4,"uuid":"47410335-85bd-4755-92ee-1203b0cef71f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6809","nonce":699064619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6811","nonce":699064619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6815","nonce":699064619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":699064619},{"type":"v1","addr":"172.21.15.186:6813","nonce":699064619}]},"public_addr":"172.21.15.186:6809/699064619","cluster_addr":"172.21.15.186:6811/699064619","heartbeat_back_addr":"172.21.15.186:6815/699064619","heartbeat_front_addr":"172.21.15.186:6813/699064619","state":["exists","up"]},{"osd":5,"uuid":"038ae36b-4552-4926-af63-dc0498229ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6809","nonce":3921910211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6811","nonce":3921910211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6815","nonce":3921910211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":3921910211},{"type":"v1","addr":"172.21.15.187:6813","nonce":3921910211}]},"public_addr":"172.21.15.187:6809/3921910211","cluster_addr":"172.21.15.187:6811/3921910211","heartbeat_back_addr":"172.21.15.187:6815/3921910211","heartbeat_front_addr":"172.21.15.187:6813/3921910211","state":["exists","up"]},{"osd":6,"uuid":"73f9efa5-f93a-48b9-83f7-af7901b11cc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6819","nonce":113445947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6821","nonce":113445947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6825","nonce":113445947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":113445947},{"type":"v1","addr":"172.21.15.178:6823","nonce":113445947}]},"public_addr":"172.21.15.178:6819/113445947","cluster_addr":"172.21.15.178:6821/113445947","heartbeat_back_addr":"172.21.15.178:6825/113445947","heartbeat_front_addr":"172.21.15.178:6823/113445947","state":["exists","up"]},{"osd":7,"uuid":"dc91a577-422e-4074-a0f8-35c3be1cac6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6816","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6817","nonce":4049803928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6819","nonce":4049803928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6823","nonce":4049803928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":4049803928},{"type":"v1","addr":"172.21.15.186:6821","nonce":4049803928}]},"public_addr":"172.21.15.186:6817/4049803928","cluster_addr":"172.21.15.186:6819/4049803928","heartbeat_back_addr":"172.21.15.186:6823/4049803928","heartbeat_front_addr":"172.21.15.186:6821/4049803928","state":["exists","up"]},{"osd":8,"uuid":"40ff8899-9c05-4331-8c78-e0d69b8571e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6817","nonce":1521942991}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6819","nonce":1521942991}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6823","nonce":1521942991}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":1521942991},{"type":"v1","addr":"172.21.15.187:6821","nonce":1521942991}]},"public_addr":"172.21.15.187:6817/1521942991","cluster_addr":"172.21.15.187:6819/1521942991","heartbeat_back_addr":"172.21.15.187:6823/1521942991","heartbeat_front_addr":"172.21.15.187:6821/1521942991","state":["exists","up"]},{"osd":9,"uuid":"22e44478-f2b6-4a84-a740-9bd31070b250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6827","nonce":2115000831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6829","nonce":2115000831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6833","nonce":2115000831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2115000831},{"type":"v1","addr":"172.21.15.178:6831","nonce":2115000831}]},"public_addr":"172.21.15.178:6827/2115000831","cluster_addr":"172.21.15.178:6829/2115000831","heartbeat_back_addr":"172.21.15.178:6833/2115000831","heartbeat_front_addr":"172.21.15.178:6831/2115000831","state":["exists","up"]},{"osd":10,"uuid":"2c36d6fd-c771-438e-916d-793a97beea8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6825","nonce":2093939166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6827","nonce":2093939166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6831","nonce":2093939166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":2093939166},{"type":"v1","addr":"172.21.15.186:6829","nonce":2093939166}]},"public_addr":"172.21.15.186:6825/2093939166","cluster_addr":"172.21.15.186:6827/2093939166","heartbeat_back_addr":"172.21.15.186:6831/2093939166","heartbeat_front_addr":"172.21.15.186:6829/2093939166","state":["exists","up"]},{"osd":11,"uuid":"dc27eb51-d50b-431d-9e60-f53a95da61af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6825","nonce":433464338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6827","nonce":433464338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6831","nonce":433464338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":433464338},{"type":"v1","addr":"172.21.15.187:6829","nonce":433464338}]},"public_addr":"172.21.15.187:6825/433464338","cluster_addr":"172.21.15.187:6827/433464338","heartbeat_back_addr":"172.21.15.187:6831/433464338","heartbeat_front_addr":"172.21.15.187:6829/433464338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:38.588594+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:26:59.931207+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:20.671647+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:27:46.417402+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:07.439852+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:28.869521+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:28:54.852549+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:16.593198+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:29:37.905075+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:03.802776+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:25.433098+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T14:30:47.277480+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":5,"to":6}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.178:6800/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2364174490":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/2438067538":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6801/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:6801/3157107959":"2023-10-17T14:25:06.501803+0000","172.21.15.178:0/3729267496":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3093526104":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/1552969994":"2023-10-17T14:25:06.501803+0000","172.21.15.178:6800/2960529415":"2023-10-17T14:24:39.404169+0000","172.21.15.178:0/3372273467":"2023-10-17T14:24:39.404169+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T14:45:45.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 msgr2=0x7fae740a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 0x7fae740a39e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fae7c000c00 tx=0x7fae7c005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 msgr2=0x7fae8c1a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fae8400ef30 tx=0x7fae8400c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 shutdown_connections 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fae740a1520 0x7fae740a39e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae8c102a40 0x7fae8c1a0320 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.663+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c100ea0 0x7fae8c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.664+0000 7fae919ff700 1 --2- 172.21.15.178:0/4063773363 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fae8c102080 0x7fae8c1992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.664+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 >> 172.21.15.178:0/4063773363 conn(0x7fae8c0fc420 msgr2=0x7fae8c0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:45.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.664+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 shutdown_connections 2023-10-16T14:45:45.669 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.664+0000 7fae919ff700 1 -- 172.21.15.178:0/4063773363 wait complete. 2023-10-16T14:45:45.687 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T14:45:45.888 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 -- 172.21.15.178:0/1231736413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 msgr2=0x7fb4a0109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/1231736413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a0109360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb498009f90 tx=0x7fb49800b330 comp rx=0 tx=0).stop 2023-10-16T14:45:45.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 -- 172.21.15.178:0/1231736413 shutdown_connections 2023-10-16T14:45:45.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/1231736413 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a0109360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/1231736413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a01012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.889 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/1231736413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 0x7fb4a0102500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 -- 172.21.15.178:0/1231736413 >> 172.21.15.178:0/1231736413 conn(0x7fb4a00fc420 msgr2=0x7fb4a00fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:45.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 -- 172.21.15.178:0/1231736413 shutdown_connections 2023-10-16T14:45:45.890 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.886+0000 7fb4a51ff700 1 -- 172.21.15.178:0/1231736413 wait complete. 2023-10-16T14:45:45.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 Processor -- start 2023-10-16T14:45:45.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 -- start start 2023-10-16T14:45:45.892 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 0x7fb4a01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a09e0 con 0x7fb4a0100ea0 2023-10-16T14:45:45.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a0b20 con 0x7fb4a0102a40 2023-10-16T14:45:45.893 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb4a51ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a01a0cb0 con 0x7fb4a0102080 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb49ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb49ed9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40900/0 (socket says 172.21.15.178:40900) 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb49ed9d700 1 -- 172.21.15.178:0/3587559522 learned_addr learned my addr 172.21.15.178:0/3587559522 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.887+0000 7fb49f59e700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb497fff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 0x7fb4a01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.894 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 -- 172.21.15.178:0/3587559522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 msgr2=0x7fb4a01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 0x7fb4a01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 -- 172.21.15.178:0/3587559522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 msgr2=0x7fb4a01a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:45.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a01a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:45.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 -- 172.21.15.178:0/3587559522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900095f0 con 0x7fb4a0100ea0 2023-10-16T14:45:45.895 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49ed9d700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb49000ebf0 tx=0x7fb49000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490009a10 con 0x7fb4a0100ea0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb49000cd40 con 0x7fb4a0100ea0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb498009c40 con 0x7fb4a0100ea0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490018820 con 0x7fb4a0100ea0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.888+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a01a1220 con 0x7fb4a0100ea0 2023-10-16T14:45:45.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.889+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a010c760 con 0x7fb4a0100ea0 2023-10-16T14:45:45.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.891+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fb490018980 con 0x7fb4a0100ea0 2023-10-16T14:45:45.897 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.893+0000 7fb49cd99700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 0x7fb48c0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:45.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.893+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb490014070 con 0x7fb4a0100ea0 2023-10-16T14:45:45.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.893+0000 7fb497fff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 0x7fb48c0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:45.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.894+0000 7fb497fff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 0x7fb48c0a3990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0100bc0 tx=0x7fb488009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:45.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:45.897+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fb490096a00 con 0x7fb4a0100ea0 2023-10-16T14:45:46.052 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:45 smithi178 ceph-mon[98495]: pgmap v700: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:45:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2135454286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.053 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:45 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4063773363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:45 smithi186 ceph-mon[134892]: pgmap v700: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:45:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2135454286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:45 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4063773363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.146 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.144+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb4a01a1410 con 0x7fb4a0100ea0 2023-10-16T14:45:46.149 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.147+0000 7fb49cd99700 1 -- 172.21.15.178:0/3587559522 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb49009a020 con 0x7fb4a0100ea0 2023-10-16T14:45:46.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 msgr2=0x7fb48c0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:46.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 0x7fb48c0a3990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0100bc0 tx=0x7fb488009260 comp rx=0 tx=0).stop 2023-10-16T14:45:46.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 msgr2=0x7fb4a0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:46.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb49000ebf0 tx=0x7fb49000ef00 comp rx=0 tx=0).stop 2023-10-16T14:45:46.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 shutdown_connections 2023-10-16T14:45:46.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fb48c0a14d0 0x7fb48c0a3990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fb4a0102a40 0x7fb4a01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4a0100ea0 0x7fb4a0198e60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 --2- 172.21.15.178:0/3587559522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4a0102080 0x7fb4a01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 >> 172.21.15.178:0/3587559522 conn(0x7fb4a00fc420 msgr2=0x7fb4a0107910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 shutdown_connections 2023-10-16T14:45:46.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.151+0000 7fb4a51ff700 1 -- 172.21.15.178:0/3587559522 wait complete. 2023-10-16T14:45:46.156 INFO:teuthology.orchestra.run.smithi178.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T14:45:46.174 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T14:45:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:45 smithi187 ceph-mon[135297]: pgmap v700: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2023-10-16T14:45:46.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2135454286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:45 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4063773363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-10-16T14:45:46.378 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 -- 172.21.15.178:0/2256653434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84101950 msgr2=0x7f1f84103d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:46.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 --2- 172.21.15.178:0/2256653434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84101950 0x7f1f84103d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009990 tx=0x7f1f74009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 -- 172.21.15.178:0/2256653434 shutdown_connections 2023-10-16T14:45:46.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 --2- 172.21.15.178:0/2256653434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f84109020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 --2- 172.21.15.178:0/2256653434 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84101950 0x7f1f84103d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.379 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 --2- 172.21.15.178:0/2256653434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84104280 0x7f1f84106670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 -- 172.21.15.178:0/2256653434 >> 172.21.15.178:0/2256653434 conn(0x7f1f840fb320 msgr2=0x7f1f840fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:46.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 -- 172.21.15.178:0/2256653434 shutdown_connections 2023-10-16T14:45:46.380 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.375+0000 7f1f889ff700 1 -- 172.21.15.178:0/2256653434 wait complete. 2023-10-16T14:45:46.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.376+0000 7f1f889ff700 1 Processor -- start 2023-10-16T14:45:46.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 -- start start 2023-10-16T14:45:46.382 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 0x7f1f84198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:46.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84104280 0x7f1f84199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:46.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:46.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841a0990 con 0x7f1f84106bb0 2023-10-16T14:45:46.383 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841a0ad0 con 0x7f1f84104280 2023-10-16T14:45:46.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f889ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841a0c60 con 0x7f1f84101950 2023-10-16T14:45:46.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f82d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:46.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f82d9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40906/0 (socket says 172.21.15.178:40906) 2023-10-16T14:45:46.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f82d9d700 1 -- 172.21.15.178:0/1077228565 learned_addr learned my addr 172.21.15.178:0/1077228565 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:46.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f8259c700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 0x7f1f84198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:46.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.377+0000 7f1f81d9b700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84104280 0x7f1f84199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:46.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 -- 172.21.15.178:0/1077228565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 msgr2=0x7f1f84198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:46.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 0x7f1f84198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 -- 172.21.15.178:0/1077228565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84104280 msgr2=0x7f1f84199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:46.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84104280 0x7f1f84199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:46.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 -- 172.21.15.178:0/1077228565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009520 con 0x7f1f84106bb0 2023-10-16T14:45:46.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f8259c700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 0x7f1f84198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T14:45:46.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f82d9d700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00ef30 tx=0x7f1f7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:46.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018070 con 0x7f1f84106bb0 2023-10-16T14:45:46.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740095f0 con 0x7f1f84106bb0 2023-10-16T14:45:46.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c00f040 con 0x7f1f84106bb0 2023-10-16T14:45:46.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00cd00 con 0x7f1f84106bb0 2023-10-16T14:45:46.387 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.378+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f841a11b0 con 0x7f1f84106bb0 2023-10-16T14:45:46.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.379+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8410c700 con 0x7f1f84106bb0 2023-10-16T14:45:46.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.381+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7f1f7c003a00 con 0x7f1f84106bb0 2023-10-16T14:45:46.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.382+0000 7f1f737fe700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 0x7f1f6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:46.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.383+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1f7c0ccd10 con 0x7f1f84106bb0 2023-10-16T14:45:46.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.383+0000 7f1f8259c700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 0x7f1f6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:46.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.383+0000 7f1f8259c700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 0x7f1f6c0a3940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b2e0 tx=0x7f1f74005fb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:46.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.386+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7f1f7c0d2020 con 0x7f1f84106bb0 2023-10-16T14:45:46.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:45:46.145+0000 7f6dfb3ea700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T14:45:46.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.577 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mds[170668]: -- 172.21.15.186:0/2972093944 <== mon.0 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600467651e0 con 0x56004673a800 2023-10-16T14:45:46.577 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: starting mds.h at 2023-10-16T14:45:46.577 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.577 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.578 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mds[171018]: -- 172.21.15.186:0/3983195902 <== mon.1 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560d4d75b1e0 con 0x560d4d730800 2023-10-16T14:45:46.578 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: starting mds.b at 2023-10-16T14:45:46.578 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.578 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.578 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mds[170326]: -- 172.21.15.186:0/1110354585 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559db96091e0 con 0x559db95de800 2023-10-16T14:45:46.579 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: starting mds.e at 2023-10-16T14:45:46.635 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:46.633+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1f841a13a0 con 0x7f1f84106bb0 2023-10-16T14:45:46.675 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.675 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.675 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mds[172901]: -- 172.21.15.187:0/3767117141 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4da4cf1e0 con 0x55f4da4a4800 2023-10-16T14:45:46.676 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: starting mds.f at 2023-10-16T14:45:46.676 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.676 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.676 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mds[173581]: -- 172.21.15.187:0/2344908740 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55de785271e0 con 0x55de784fd000 2023-10-16T14:45:46.677 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: starting mds.i at 2023-10-16T14:45:46.677 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: ignoring --setuser ceph since I am not root 2023-10-16T14:45:46.677 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: ignoring --setgroup ceph since I am not root 2023-10-16T14:45:46.677 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mds[173242]: -- 172.21.15.187:0/3724902247 <== mon.2 v2:172.21.15.186:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555738e911e0 con 0x555738e67000 2023-10-16T14:45:46.678 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: starting mds.c at 2023-10-16T14:45:47.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-16T14:45:47.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-16T14:45:47.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T14:45:47.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-16T14:45:47.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-16T14:45:47.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: osdmap e92: 12 total, 12 up, 12 in 2023-10-16T14:45:47.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-16T14:45:47.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:46 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:45:47.151 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.148+0000 7f1f737fe700 1 -- 172.21.15.178:0/1077228565 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1f7c095f30 con 0x7f1f84106bb0 2023-10-16T14:45:47.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.151+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 msgr2=0x7f1f6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:47.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.151+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 0x7f1f6c0a3940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b2e0 tx=0x7f1f74005fb0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 msgr2=0x7f1f841a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:47.154 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00ef30 tx=0x7f1f7c00c3a0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 shutdown_connections 2023-10-16T14:45:47.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7f1f6c0a1480 0x7f1f6c0a3940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f84106bb0 0x7f1f841a03c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f84101950 0x7f1f84198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.155 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 --2- 172.21.15.178:0/1077228565 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7f1f84104280 0x7f1f84199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 >> 172.21.15.178:0/1077228565 conn(0x7f1f840fb320 msgr2=0x7f1f841008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:47.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 shutdown_connections 2023-10-16T14:45:47.156 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.152+0000 7f1f889ff700 1 -- 172.21.15.178:0/1077228565 wait complete. 2023-10-16T14:45:47.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-16T14:45:47.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-16T14:45:47.174 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T14:45:47.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-16T14:45:47.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-16T14:45:47.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: osdmap e92: 12 total, 12 up, 12 in 2023-10-16T14:45:47.175 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-16T14:45:47.176 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:46 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:45:47.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-10-16T14:45:47.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-10-16T14:45:47.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-16T14:45:47.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: Health check cleared: MDS_DAMAGE (was: 1 MDSs report damaged metadata) 2023-10-16T14:45:47.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/3587559522' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-10-16T14:45:47.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: osdmap e92: 12 total, 12 up, 12 in 2023-10-16T14:45:47.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-10-16T14:45:47.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:46 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-10-16T14:45:47.179 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T14:45:47.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 -- 172.21.15.178:0/3845825229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102060 msgr2=0x7fd5041024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:47.384 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 --2- 172.21.15.178:0/3845825229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102060 0x7fd5041024e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009990 tx=0x7fd4f0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 -- 172.21.15.178:0/3845825229 shutdown_connections 2023-10-16T14:45:47.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 --2- 172.21.15.178:0/3845825229 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504102a20 0x7fd504109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 --2- 172.21.15.178:0/3845825229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504100e80 0x7fd5041012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.385 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 --2- 172.21.15.178:0/3845825229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102060 0x7fd5041024e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 -- 172.21.15.178:0/3845825229 >> 172.21.15.178:0/3845825229 conn(0x7fd5040fc400 msgr2=0x7fd5040fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:47.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.381+0000 7fd50abff700 1 -- 172.21.15.178:0/3845825229 shutdown_connections 2023-10-16T14:45:47.386 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.382+0000 7fd50abff700 1 -- 172.21.15.178:0/3845825229 wait complete. 2023-10-16T14:45:47.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.382+0000 7fd50abff700 1 Processor -- start 2023-10-16T14:45:47.388 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 -- start start 2023-10-16T14:45:47.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504100e80 0x7fd504198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:47.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504102060 0x7fd5041993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:47.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:47.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a09e0 con 0x7fd504102a20 2023-10-16T14:45:47.389 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a0b20 con 0x7fd504100e80 2023-10-16T14:45:47.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50abff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5041a0cb0 con 0x7fd504102060 2023-10-16T14:45:47.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50919c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:47.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50919c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40920/0 (socket says 172.21.15.178:40920) 2023-10-16T14:45:47.390 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50919c700 1 -- 172.21.15.178:0/4289633981 learned_addr learned my addr 172.21.15.178:0/4289633981 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:47.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd50899b700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504100e80 0x7fd504198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:47.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.383+0000 7fd503fff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504102060 0x7fd5041993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:47.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 -- 172.21.15.178:0/4289633981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504102060 msgr2=0x7fd5041993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:47.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504102060 0x7fd5041993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.391 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 -- 172.21.15.178:0/4289633981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504100e80 msgr2=0x7fd504198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:47.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504100e80 0x7fd504198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:47.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 -- 172.21.15.178:0/4289633981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00095f0 con 0x7fd504102a20 2023-10-16T14:45:47.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50919c700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc007ee0 tx=0x7fd4fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:47.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc00dd20 con 0x7fd504102a20 2023-10-16T14:45:47.392 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4fc00f040 con 0x7fd504102a20 2023-10-16T14:45:47.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5041a0f90 con 0x7fd504102a20 2023-10-16T14:45:47.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0048a0 con 0x7fd504102a20 2023-10-16T14:45:47.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.384+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5041a13c0 con 0x7fd504102a20 2023-10-16T14:45:47.393 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.385+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50410c740 con 0x7fd504102a20 2023-10-16T14:45:47.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.387+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fd4fc003a00 con 0x7fd504102a20 2023-10-16T14:45:47.394 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.388+0000 7fd501ffb700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:47.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.389+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd4fc0cca10 con 0x7fd504102a20 2023-10-16T14:45:47.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.389+0000 7fd50899b700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:47.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.389+0000 7fd50899b700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 0x7fd4f40a3940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd504101bd0 tx=0x7fd4f8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:47.395 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.392+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fd4fc04f100 con 0x7fd504102a20 2023-10-16T14:45:47.597 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:47.595+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd50410c8d0 con 0x7fd504102a20 2023-10-16T14:45:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: pgmap v702: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-10-16T14:45:48.076 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-16T14:45:48.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-16T14:45:48.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: Cluster is now healthy 2023-10-16T14:45:48.077 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:45:48.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6838/1013649794,v1:172.21.15.187:6839/1013649794] up:boot 2023-10-16T14:45:48.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6834/2424726433,v1:172.21.15.187:6836/2424726433] up:boot 2023-10-16T14:45:48.078 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.187:6835/2157090523,v1:172.21.15.187:6837/2157090523] up:boot 2023-10-16T14:45:48.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6838/893407906,v1:172.21.15.186:6839/893407906] up:boot 2023-10-16T14:45:48.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6834/2256238380,v1:172.21.15.186:6835/2256238380] up:boot 2023-10-16T14:45:48.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: mds.? [v2:172.21.15.186:6836/2325216218,v1:172.21.15.186:6837/2325216218] up:boot 2023-10-16T14:45:48.079 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: fsmap 12 up:standby 2023-10-16T14:45:48.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:45:48.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:45:48.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:45:48.080 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:45:48.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:45:48.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:45:48.081 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:47 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:48.153 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.151+0000 7fd501ffb700 1 -- 172.21.15.178:0/4289633981 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4fc099630 con 0x7fd504102a20 2023-10-16T14:45:48.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 msgr2=0x7fd4f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:48.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 0x7fd4f40a3940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd504101bd0 tx=0x7fd4f8009260 comp rx=0 tx=0).stop 2023-10-16T14:45:48.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 msgr2=0x7fd5041a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:48.157 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc007ee0 tx=0x7fd4fc00d3b0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 shutdown_connections 2023-10-16T14:45:48.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd504102a20 0x7fd5041a0410 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.158 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fd504100e80 0x7fd504198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 --2- 172.21.15.178:0/4289633981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd504102060 0x7fd5041993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 >> 172.21.15.178:0/4289633981 conn(0x7fd5040fc400 msgr2=0x7fd5040fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:48.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 shutdown_connections 2023-10-16T14:45:48.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.154+0000 7fd50abff700 1 -- 172.21.15.178:0/4289633981 wait complete. 2023-10-16T14:45:48.160 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_metadata' removed 2023-10-16T14:45:48.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: pgmap v702: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-10-16T14:45:48.175 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-16T14:45:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-16T14:45:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: Cluster is now healthy 2023-10-16T14:45:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:45:48.176 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6838/1013649794,v1:172.21.15.187:6839/1013649794] up:boot 2023-10-16T14:45:48.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6834/2424726433,v1:172.21.15.187:6836/2424726433] up:boot 2023-10-16T14:45:48.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.187:6835/2157090523,v1:172.21.15.187:6837/2157090523] up:boot 2023-10-16T14:45:48.177 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6838/893407906,v1:172.21.15.186:6839/893407906] up:boot 2023-10-16T14:45:48.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6834/2256238380,v1:172.21.15.186:6835/2256238380] up:boot 2023-10-16T14:45:48.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: mds.? [v2:172.21.15.186:6836/2325216218,v1:172.21.15.186:6837/2325216218] up:boot 2023-10-16T14:45:48.178 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: fsmap 12 up:standby 2023-10-16T14:45:48.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:45:48.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:45:48.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:45:48.179 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:45:48.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:45:48.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:45:48.180 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:47 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:48.181 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: pgmap v702: 129 pgs: 129 active+clean; 668 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 614 B/s wr, 6 op/s 2023-10-16T14:45:48.181 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-10-16T14:45:48.181 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-10-16T14:45:48.181 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: Cluster is now healthy 2023-10-16T14:45:48.181 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/1077228565' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-10-16T14:45:48.182 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6838/1013649794,v1:172.21.15.187:6839/1013649794] up:boot 2023-10-16T14:45:48.182 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6834/2424726433,v1:172.21.15.187:6836/2424726433] up:boot 2023-10-16T14:45:48.182 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.187:6835/2157090523,v1:172.21.15.187:6837/2157090523] up:boot 2023-10-16T14:45:48.183 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6838/893407906,v1:172.21.15.186:6839/893407906] up:boot 2023-10-16T14:45:48.183 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6834/2256238380,v1:172.21.15.186:6835/2256238380] up:boot 2023-10-16T14:45:48.184 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: mds.? [v2:172.21.15.186:6836/2325216218,v1:172.21.15.186:6837/2325216218] up:boot 2023-10-16T14:45:48.184 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: fsmap 12 up:standby 2023-10-16T14:45:48.184 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-10-16T14:45:48.185 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-10-16T14:45:48.185 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-10-16T14:45:48.185 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-10-16T14:45:48.185 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-10-16T14:45:48.185 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='mgr.14148 172.21.15.178:0/352796249' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-10-16T14:45:48.186 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:47 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:48.187 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T14:45:48.401 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.398+0000 7fecc5fff700 1 -- 172.21.15.178:0/3243476336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 msgr2=0x7fecc01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:48.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.398+0000 7fecc5fff700 1 --2- 172.21.15.178:0/3243476336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc01012a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009990 tx=0x7fecb0009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 -- 172.21.15.178:0/3243476336 shutdown_connections 2023-10-16T14:45:48.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 --2- 172.21.15.178:0/3243476336 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102a20 0x7fecc0109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.402 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 --2- 172.21.15.178:0/3243476336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc01012a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 --2- 172.21.15.178:0/3243476336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102060 0x7fecc01024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 -- 172.21.15.178:0/3243476336 >> 172.21.15.178:0/3243476336 conn(0x7fecc00fc400 msgr2=0x7fecc00fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:48.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 -- 172.21.15.178:0/3243476336 shutdown_connections 2023-10-16T14:45:48.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.399+0000 7fecc5fff700 1 -- 172.21.15.178:0/3243476336 wait complete. 2023-10-16T14:45:48.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 Processor -- start 2023-10-16T14:45:48.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 -- start start 2023-10-16T14:45:48.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:48.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 0x7fecc01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:48.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 0x7fecc01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:48.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01a09e0 con 0x7fecc0100e80 2023-10-16T14:45:48.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01a0b20 con 0x7fecc0102060 2023-10-16T14:45:48.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecc5fff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc01a0cb0 con 0x7fecc0102a20 2023-10-16T14:45:48.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecbf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:48.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecbf7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:40940/0 (socket says 172.21.15.178:40940) 2023-10-16T14:45:48.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.400+0000 7fecbf7fe700 1 -- 172.21.15.178:0/2753128025 learned_addr learned my addr 172.21.15.178:0/2753128025 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:48.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbeffd700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 0x7fecc01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:48.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbffff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 0x7fecc01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:48.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 -- 172.21.15.178:0/2753128025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 msgr2=0x7fecc01a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:48.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 0x7fecc01a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 -- 172.21.15.178:0/2753128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 msgr2=0x7fecc01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:48.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 0x7fecc01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 -- 172.21.15.178:0/2753128025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00095f0 con 0x7fecc0100e80 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbffff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 0x7fecc01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbeffd700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 0x7fecc01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbf7fe700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b680 tx=0x7fecb0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb001d070 con 0x7fecc0100e80 2023-10-16T14:45:48.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecb000fbb0 con 0x7fecc0100e80 2023-10-16T14:45:48.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc01a0f30 con 0x7fecc0100e80 2023-10-16T14:45:48.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0017650 con 0x7fecc0100e80 2023-10-16T14:45:48.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.401+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc01a12a0 con 0x7fecc0100e80 2023-10-16T14:45:48.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.402+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc010c7b0 con 0x7fecc0100e80 2023-10-16T14:45:48.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.404+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fecb000fd20 con 0x7fecc0100e80 2023-10-16T14:45:48.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.405+0000 7fecbcff9700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 0x7feca80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:48.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.406+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 8753+0+0 (secure 0 0 0) 0x7fecb00ce5a0 con 0x7fecc0100e80 2023-10-16T14:45:48.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.406+0000 7fecbeffd700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 0x7feca80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:48.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.406+0000 7fecbeffd700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 0x7feca80a3940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fecc0100ba0 tx=0x7fecb4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:48.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.409+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fecb0097a20 con 0x7fecc0100e80 2023-10-16T14:45:48.614 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:48.612+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fecc010c940 con 0x7fecc0100e80 2023-10-16T14:45:49.159 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.157+0000 7fecbcff9700 1 -- 172.21.15.178:0/2753128025 <== mon.0 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7fecb009b040 con 0x7fecc0100e80 2023-10-16T14:45:49.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.160+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 msgr2=0x7feca80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:49.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.160+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 0x7feca80a3940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fecc0100ba0 tx=0x7fecb4009260 comp rx=0 tx=0).stop 2023-10-16T14:45:49.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.160+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 msgr2=0x7fecc0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:49.163 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.160+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fecb000b680 tx=0x7fecb0005ef0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 shutdown_connections 2023-10-16T14:45:49.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7feca80a1480 0x7feca80a3940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.164 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fecc0102a20 0x7fecc01a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fecc0100e80 0x7fecc0198e60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 --2- 172.21.15.178:0/2753128025 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fecc0102060 0x7fecc01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 >> 172.21.15.178:0/2753128025 conn(0x7fecc00fc400 msgr2=0x7fecc00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:49.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 shutdown_connections 2023-10-16T14:45:49.165 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.161+0000 7fecc5fff700 1 -- 172.21.15.178:0/2753128025 wait complete. 2023-10-16T14:45:49.166 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_data' removed 2023-10-16T14:45:49.184 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-10-16T14:45:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:49 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:49 smithi187 ceph-mon[135297]: osdmap e93: 12 total, 12 up, 12 in 2023-10-16T14:45:49.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:49 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:49.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.448+0000 7fcf895ff700 1 -- 172.21.15.178:0/468823544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84102060 msgr2=0x7fcf841024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:49.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.448+0000 7fcf895ff700 1 --2- 172.21.15.178:0/468823544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84102060 0x7fcf841024e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009990 tx=0x7fcf74009ca0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.451 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.448+0000 7fcf895ff700 1 -- 172.21.15.178:0/468823544 shutdown_connections 2023-10-16T14:45:49.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.448+0000 7fcf895ff700 1 --2- 172.21.15.178:0/468823544 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 0x7fcf84109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.449+0000 7fcf895ff700 1 --2- 172.21.15.178:0/468823544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84100e80 0x7fcf841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.449+0000 7fcf895ff700 1 --2- 172.21.15.178:0/468823544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84102060 0x7fcf841024e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.449+0000 7fcf895ff700 1 -- 172.21.15.178:0/468823544 >> 172.21.15.178:0/468823544 conn(0x7fcf840fc400 msgr2=0x7fcf840fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:49.452 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.449+0000 7fcf895ff700 1 -- 172.21.15.178:0/468823544 shutdown_connections 2023-10-16T14:45:49.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.449+0000 7fcf895ff700 1 -- 172.21.15.178:0/468823544 wait complete. 2023-10-16T14:45:49.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.450+0000 7fcf895ff700 1 Processor -- start 2023-10-16T14:45:49.453 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.451+0000 7fcf895ff700 1 -- start start 2023-10-16T14:45:49.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.451+0000 7fcf895ff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:49.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf895ff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84102060 0x7fcf8419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:49.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf895ff700 1 --2- >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 0x7fcf841950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:49.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf895ff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841ad660 con 0x7fcf84102060 2023-10-16T14:45:49.455 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf895ff700 1 -- --> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841ad7a0 con 0x7fcf84102a20 2023-10-16T14:45:49.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf895ff700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841ad930 con 0x7fcf84100e80 2023-10-16T14:45:49.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:49.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.178:60518/0 (socket says 172.21.15.178:60518) 2023-10-16T14:45:49.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 -- 172.21.15.178:0/2660111035 learned_addr learned my addr 172.21.15.178:0/2660111035 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-10-16T14:45:49.456 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf837fe700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 0x7fcf841950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:49.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 -- 172.21.15.178:0/2660111035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 msgr2=0x7fcf841950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:49.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 0x7fcf841950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 -- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84102060 msgr2=0x7fcf8419a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:49.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.452+0000 7fcf82ffd700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84102060 0x7fcf8419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:49.457 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf82ffd700 1 -- 172.21.15.178:0/2660111035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740095f0 con 0x7fcf84100e80 2023-10-16T14:45:49.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf82ffd700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00b9f0 tx=0x7fcf7c00bd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:49.458 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00c950 con 0x7fcf84100e80 2023-10-16T14:45:49.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf841adc10 con 0x7fcf84100e80 2023-10-16T14:45:49.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf7c014440 con 0x7fcf84100e80 2023-10-16T14:45:49.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c0125a0 con 0x7fcf84100e80 2023-10-16T14:45:49.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.453+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf841ae170 con 0x7fcf84100e80 2023-10-16T14:45:49.459 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.455+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf84104ab0 con 0x7fcf84100e80 2023-10-16T14:45:49.463 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.461+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 18) v1 ==== 136004+0+0 (secure 0 0 0) 0x7fcf7c0148b0 con 0x7fcf84100e80 2023-10-16T14:45:49.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.463+0000 7fcf6bfff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 0x7fcf6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T14:45:49.466 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.464+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8252+0+0 (secure 0 0 0) 0x7fcf7c006e40 con 0x7fcf84100e80 2023-10-16T14:45:49.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.464+0000 7fcf827fc700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 0x7fcf6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T14:45:49.467 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.464+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180459 (secure 0 0 0) 0x7fcf7c0cd250 con 0x7fcf84100e80 2023-10-16T14:45:49.470 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.468+0000 7fcf827fc700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 0x7fcf6c0a3940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b680 tx=0x7fcf74005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T14:45:49.552 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:49 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:49 smithi178 ceph-mon[98495]: osdmap e93: 12 total, 12 up, 12 in 2023-10-16T14:45:49.553 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:49 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:49.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:49 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/4289633981' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:49.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:49 smithi186 ceph-mon[134892]: osdmap e93: 12 total, 12 up, 12 in 2023-10-16T14:45:49.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:49 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:49.681 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:49.679+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcf8410c7e0 con 0x7fcf84100e80 2023-10-16T14:45:50.167 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.164+0000 7fcf6bfff700 1 -- 172.21.15.178:0/2660111035 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v95) v1 ==== 178+0+0 (secure 0 0 0) 0x7fcf7c096580 con 0x7fcf84100e80 2023-10-16T14:45:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 msgr2=0x7fcf6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 0x7fcf6c0a3940 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400b680 tx=0x7fcf74005e80 comp rx=0 tx=0).stop 2023-10-16T14:45:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 msgr2=0x7fcf84194ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T14:45:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00b9f0 tx=0x7fcf7c00bd00 comp rx=0 tx=0).stop 2023-10-16T14:45:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 shutdown_connections 2023-10-16T14:45:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:6800/706630784,v1:172.21.15.178:6801/706630784] conn(0x7fcf6c0a1480 0x7fcf6c0a3940 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:50.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x7fcf84102a20 0x7fcf841950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:50.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf84100e80 0x7fcf84194ac0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:50.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 --2- 172.21.15.178:0/2660111035 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf84102060 0x7fcf8419a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T14:45:50.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 >> 172.21.15.178:0/2660111035 conn(0x7fcf840fc400 msgr2=0x7fcf84107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T14:45:50.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 shutdown_connections 2023-10-16T14:45:50.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-10-16T14:45:50.168+0000 7fcf895ff700 1 -- 172.21.15.178:0/2660111035 wait complete. 2023-10-16T14:45:50.173 INFO:teuthology.orchestra.run.smithi178.stderr:pool 'cephfs_data_ec' removed 2023-10-16T14:45:50.197 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:45:50.268 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:45:50.312 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:45:50.343 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-10-16T14:45:50.343 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-16T14:45:50.387 DEBUG:teuthology.orchestra.run.smithi186:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-16T14:45:50.405 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-10-16T14:45:50.422 INFO:tasks.cephadm:Stopping all daemons... 2023-10-16T14:45:50.423 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-10-16T14:45:50.424 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a 2023-10-16T14:45:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:50 smithi187 ceph-mon[135297]: pgmap v704: 97 pgs: 97 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:45:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:50 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:50 smithi187 ceph-mon[135297]: osdmap e94: 12 total, 12 up, 12 in 2023-10-16T14:45:50.425 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:50 smithi187 ceph-mon[135297]: from='client.? 172.21.15.178:0/2660111035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.426 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:50 smithi187 ceph-mon[135297]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.492 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 ceph-mon[98495]: pgmap v704: 97 pgs: 97 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:45:50.492 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:50.492 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 ceph-mon[98495]: osdmap e94: 12 total, 12 up, 12 in 2023-10-16T14:45:50.493 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 ceph-mon[98495]: from='client.? 172.21.15.178:0/2660111035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.493 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 ceph-mon[98495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:50 smithi186 ceph-mon[134892]: pgmap v704: 97 pgs: 97 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2023-10-16T14:45:50.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:50 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2753128025' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-10-16T14:45:50.576 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:50 smithi186 ceph-mon[134892]: osdmap e94: 12 total, 12 up, 12 in 2023-10-16T14:45:50.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:50 smithi186 ceph-mon[134892]: from='client.? 172.21.15.178:0/2660111035' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.577 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:50 smithi186 ceph-mon[134892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-10-16T14:45:50.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:50 smithi178 systemd[1]: Stopping Ceph mon.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:51.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:45:51.051+0000 7f6e00bf5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:45:51.332 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a[98471]: 2023-10-16T14:45:51.051+0000 7f6e00bf5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-10-16T14:45:51.802 INFO:journalctl@ceph.mon.a.smithi178.stdout:Oct 16 14:45:51 smithi178 bash[178326]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-a 2023-10-16T14:45:51.918 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.a.service' 2023-10-16T14:45:52.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:52.031 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-10-16T14:45:52.032 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-10-16T14:45:52.032 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.b 2023-10-16T14:45:52.326 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 systemd[1]: Stopping Ceph mon.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:52.705 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-b[134868]: 2023-10-16T14:45:52.420+0000 7f7f2f2c4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:45:52.706 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-b[134868]: 2023-10-16T14:45:52.420+0000 7f7f2f2c4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-10-16T14:45:52.955 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 bash[175941]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-b 2023-10-16T14:45:52.956 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.b.service: Succeeded. 2023-10-16T14:45:52.956 INFO:journalctl@ceph.mon.b.smithi186.stdout:Oct 16 14:45:52 smithi186 systemd[1]: Stopped Ceph mon.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:45:52.958 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.b.service' 2023-10-16T14:45:53.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:53.015 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-10-16T14:45:53.015 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-10-16T14:45:53.015 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.c 2023-10-16T14:45:53.400 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:53 smithi187 systemd[1]: Stopping Ceph mon.c for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:53.400 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-c[135271]: 2023-10-16T14:45:53.395+0000 7f4c34abc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:45:53.674 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-c[135271]: 2023-10-16T14:45:53.395+0000 7f4c34abc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-10-16T14:45:54.053 INFO:journalctl@ceph.mon.c.smithi187.stdout:Oct 16 14:45:53 smithi187 bash[179039]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mon-c 2023-10-16T14:45:54.060 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mon.c.service' 2023-10-16T14:45:54.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:54.103 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-10-16T14:45:54.104 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-10-16T14:45:54.104 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x 2023-10-16T14:45:54.552 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:45:54 smithi178 systemd[1]: Stopping Ceph mgr.x for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:54.899 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:45:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:45:54.622+0000 7f068009c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:45:54.900 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:45:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x[98745]: 2023-10-16T14:45:54.622+0000 7f068009c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-10-16T14:45:55.302 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Oct 16 14:45:54 smithi178 bash[178476]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-x 2023-10-16T14:45:55.380 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.x.service' 2023-10-16T14:45:55.467 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:55.467 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-10-16T14:45:55.467 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-10-16T14:45:55.468 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.y 2023-10-16T14:45:55.826 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:45:55 smithi186 systemd[1]: Stopping Ceph mgr.y for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:56.357 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:45:56 smithi186 bash[176090]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-y 2023-10-16T14:45:56.358 INFO:journalctl@ceph.mgr.y.smithi186.stdout:Oct 16 14:45:56 smithi186 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-10-16T14:45:56.361 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.y.service' 2023-10-16T14:45:56.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:56.402 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-10-16T14:45:56.402 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-10-16T14:45:56.403 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.z 2023-10-16T14:45:56.828 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:45:56 smithi187 systemd[1]: Stopping Ceph mgr.z for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:57.111 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:45:57 smithi187 bash[179199]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mgr-z 2023-10-16T14:45:57.290 INFO:journalctl@ceph.mgr.z.smithi187.stdout:Oct 16 14:45:57 smithi187 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-10-16T14:45:57.293 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mgr.z.service' 2023-10-16T14:45:57.330 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:45:57.331 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-10-16T14:45:57.331 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-10-16T14:45:57.332 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.0 2023-10-16T14:45:57.802 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:45:57 smithi178 systemd[1]: Stopping Ceph osd.0 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:45:58.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:45:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:45:57.846+0000 7fd7e53e3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:45:58.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:45:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:45:57.846+0000 7fd7e53e3700 -1 osd.0 95 *** Got signal Terminated *** 2023-10-16T14:45:58.303 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:45:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0[117314]: 2023-10-16T14:45:57.846+0000 7fd7e53e3700 -1 osd.0 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:03.802 INFO:journalctl@ceph.osd.0.smithi178.stdout:Oct 16 14:46:03 smithi178 bash[178623]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-0 2023-10-16T14:46:05.455 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.0.service' 2023-10-16T14:46:05.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:05.716 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-10-16T14:46:05.717 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-10-16T14:46:05.717 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.1 2023-10-16T14:46:06.076 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:46:05 smithi186 systemd[1]: Stopping Ceph osd.1 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:06.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:46:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:46:06.077+0000 7f98f16dd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:06.576 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:46:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:46:06.077+0000 7f98f16dd700 -1 osd.1 95 *** Got signal Terminated *** 2023-10-16T14:46:06.577 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:46:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1[139934]: 2023-10-16T14:46:06.077+0000 7f98f16dd700 -1 osd.1 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:12.076 INFO:journalctl@ceph.osd.1.smithi186.stdout:Oct 16 14:46:11 smithi186 bash[176237]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-1 2023-10-16T14:46:13.447 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.1.service' 2023-10-16T14:46:13.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:13.489 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-10-16T14:46:13.489 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-10-16T14:46:13.489 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.2 2023-10-16T14:46:13.855 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:46:13 smithi187 systemd[1]: Stopping Ceph osd.2 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:14.175 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:46:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:46:13.850+0000 7f9a1bd0f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:14.175 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:46:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:46:13.850+0000 7f9a1bd0f700 -1 osd.2 95 *** Got signal Terminated *** 2023-10-16T14:46:14.175 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:46:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2[141101]: 2023-10-16T14:46:13.850+0000 7f9a1bd0f700 -1 osd.2 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:19.924 INFO:journalctl@ceph.osd.2.smithi187.stdout:Oct 16 14:46:19 smithi187 bash[179354]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-2 2023-10-16T14:46:21.038 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.2.service' 2023-10-16T14:46:21.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:21.078 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-10-16T14:46:21.079 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-10-16T14:46:21.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.3 2023-10-16T14:46:21.552 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:21 smithi178 systemd[1]: Stopping Ceph osd.3 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:22.052 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:21.591+0000 7f27ad120700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:22.053 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:21.591+0000 7f27ad120700 -1 osd.3 95 *** Got signal Terminated *** 2023-10-16T14:46:22.053 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:21 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:21.591+0000 7f27ad120700 -1 osd.3 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:24.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:24.372+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:24.802 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:24 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:24.524+0000 7f27a6f11700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:02.509251+0000 front 2023-10-16T14:46:02.509282+0000 (oldest deadline 2023-10-16T14:46:23.609466+0000) 2023-10-16T14:46:25.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:25 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:25.193+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:25.672 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:25.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:25.802 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:25 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:25.546+0000 7f27a6f11700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:02.509251+0000 front 2023-10-16T14:46:02.509282+0000 (oldest deadline 2023-10-16T14:46:23.609466+0000) 2023-10-16T14:46:25.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:25.668+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:26.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:26 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:26.193+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:26.648 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:26.348+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:26.802 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:26 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3[125863]: 2023-10-16T14:46:26.529+0000 7f27a6f11700 -1 osd.3 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:02.509251+0000 front 2023-10-16T14:46:02.509282+0000 (oldest deadline 2023-10-16T14:46:23.609466+0000) 2023-10-16T14:46:26.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:26.646+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:27.552 INFO:journalctl@ceph.osd.3.smithi178.stdout:Oct 16 14:46:27 smithi178 bash[178896]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-3 2023-10-16T14:46:27.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:27 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:27.235+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:27.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:27 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:27.236+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:27.660 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:27.398+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:27.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:27.658+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:28.472 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:28 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:28.186+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:28.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:28 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:28.246+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:28.650 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:28.375+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:28.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:28.648+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:29.373 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.3.service' 2023-10-16T14:46:29.460 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:29 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:29.188+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:29.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:29.463 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-10-16T14:46:29.464 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-10-16T14:46:29.464 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.4 2023-10-16T14:46:29.495 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:29 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:29.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:29.653 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:29 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:29.343+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:29.819 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:46:29 smithi186 systemd[1]: Stopping Ceph osd.4 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:29.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:29 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:29.650+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:30.075 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:46:29 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:46:29.815+0000 7feb7f89d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:30.076 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:46:29 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:46:29.815+0000 7feb7f89d700 -1 osd.4 95 *** Got signal Terminated *** 2023-10-16T14:46:30.076 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:46:29 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4[147857]: 2023-10-16T14:46:29.815+0000 7feb7f89d700 -1 osd.4 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:30.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:30 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:30.209+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:30.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:30 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:30.245+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:30.634 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:30 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:30.379+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:30.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:30 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:30.632+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:31.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:31.194+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:31.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:31 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:31.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:31.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:31 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:31.366+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:32.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:31 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:31.678+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:32.465 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:32.167+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:32.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:32 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:32.207+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:32.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:32 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:32.367+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:32.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:32 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:32.367+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:32.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:32 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:32.462+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:33.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:32 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:32.604+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:33.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:32 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:32.728+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:33.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:32 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:32.927+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:33.480 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:33 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:33.180+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:33.480 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:33 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:33.180+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:33.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:33 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:33.211+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:33.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:33 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:33.376+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:33.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:33 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:33.376+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:33.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:33 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:33.477+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:34.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:33 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:33.620+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:34.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:33 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:33.957+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:34.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:33 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:33.774+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:34.552 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:34.453+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:34.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:34.208+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:34.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:34.208+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:34.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:34 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:34.173+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:34.728 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:34 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:34.422+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:34.728 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:34 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:34.422+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:35.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:34 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:34.597+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:35.174 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:34 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:34.919+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:35.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:34 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:34.725+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:35.488 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:35.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:35.488 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:35.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:35.561 INFO:journalctl@ceph.osd.4.smithi186.stdout:Oct 16 14:46:35 smithi186 bash[176520]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-4 2023-10-16T14:46:35.562 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:35 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:35.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:35.562 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:35 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:35.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:35.717 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:35 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:35.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:35.718 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:35 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:35.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:35.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:35 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:35.485+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:35.825 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:35 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:35.566+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:36.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:35 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:35.904+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:36.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:35 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:35.715+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:36.480 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:36 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:36.168+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:36.480 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:36 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:36.168+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:36.528 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:36.215+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:36.528 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:36.215+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:36.756 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:36 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:36.425+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:36.798 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:36 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:36.425+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:36.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:36.525+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:36.822 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:36 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:36.565+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:36.829 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.4.service' 2023-10-16T14:46:36.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:36.916 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-10-16T14:46:36.916 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-10-16T14:46:36.916 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.5 2023-10-16T14:46:37.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:36 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:36.752+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:37.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:36 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:36.864+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:37.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:36 smithi187 systemd[1]: Stopping Ceph osd.5 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:37.512 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:37.185+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:37.513 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:37.185+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:37.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:37 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:37.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:37.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:37 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:37.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:37.674 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:37.272+0000 7f83be07a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:37.675 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:37.272+0000 7f83be07a700 -1 osd.5 95 *** Got signal Terminated *** 2023-10-16T14:46:37.675 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:37.272+0000 7f83be07a700 -1 osd.5 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:37.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:37.430+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:37.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:37.430+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:37.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:37 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:37.509+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:38.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:37 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:37.613+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:38.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:37.714+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:38.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:37 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:37.896+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:38.526 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:38 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:38.202+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:38.527 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:38 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:38.202+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:38.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:38 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:38.175+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:38.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:38 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:38.175+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:38.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:38 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:38.408+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:38.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:38 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:38.408+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:38.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:38 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:38.523+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:39.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:38 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:38.643+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:39.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:38 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:38.763+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:39.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:38 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:38.863+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:39.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:39.159+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:39.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:39.159+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:39.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:39 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:39.200+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:39.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:39 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:39.200+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:39.720 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:39 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:39.454+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:39.720 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:39 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:39.454+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:40.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:39 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:39.559+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:40.075 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:39 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:39.686+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:40.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:39 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:39.881+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:40.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:39 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:39.718+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:40.540 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:40 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:40.111+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:40.541 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:40 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:40.111+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:40.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:40 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:40.151+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:40.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:40 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:40.151+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:40.763 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:40.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:40.763 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:40.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:40.802 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:40 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:40.537+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:41.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:40 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:40.697+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:41.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:40.883+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:41.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:40.883+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:15.545617+0000 front 2023-10-16T14:46:15.545649+0000 (oldest deadline 2023-10-16T14:46:40.845824+0000) 2023-10-16T14:46:41.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:40.761+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:41.176 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:40 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:40.761+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:41.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:41 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:41.115+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:41.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:41 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:41.115+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:41.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:41 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:41.176+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:41.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:41 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:41.176+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:41.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:41.389+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:41.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:41.389+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:42.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:41 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:41.573+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:42.075 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:41 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:41.732+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:42.174 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:41.855+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:10.944479+0000 front 2023-10-16T14:46:10.944580+0000 (oldest deadline 2023-10-16T14:46:32.044509+0000) 2023-10-16T14:46:42.175 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5[149635]: 2023-10-16T14:46:41.855+0000 7f83b6e69700 -1 osd.5 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:15.545617+0000 front 2023-10-16T14:46:15.545649+0000 (oldest deadline 2023-10-16T14:46:40.845824+0000) 2023-10-16T14:46:42.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:41.714+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:42.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:41 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:41.714+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:42.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:42.125+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:42.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:42.125+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:42.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:42 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:42.218+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:42.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:42 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:42.218+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:42.766 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:42 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:42.429+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:42.766 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:42 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:42.429+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:43.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:42.608+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:43.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:42.608+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:43.057 INFO:journalctl@ceph.osd.5.smithi187.stdout:Oct 16 14:46:42 smithi187 bash[179623]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-5 2023-10-16T14:46:43.057 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:42 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:42.763+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:43.057 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:42 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:42.763+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:43.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:42 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:42.746+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:43.077 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:42 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:42.746+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:43.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:43 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:43.105+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:43.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:43 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:43.105+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:43.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:43 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:43.105+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:43.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:43 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:43.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:43.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:43 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:43.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:43.703 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:43.436+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:43.703 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:43.436+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:43.703 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:43.436+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:44.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:43 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:43.610+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:44.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:43 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:43.611+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:44.074 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:43.761+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:44.074 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:43 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:43.761+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:44.075 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:43 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:43.776+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:44.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:43 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:43.776+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:44.447 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.5.service' 2023-10-16T14:46:44.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:44.487 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-10-16T14:46:44.488 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-10-16T14:46:44.488 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.6 2023-10-16T14:46:44.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:44.139+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:44.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:44.139+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:44.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:44.139+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:44.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:44 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:44.294+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:44.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:44 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:44.294+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:44.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:44.387+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:44.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:44.387+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:44.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:44.387+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:45.007 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:44 smithi178 systemd[1]: Stopping Ceph osd.6 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:45.007 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:44.650+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:45.008 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:44 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:44.650+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:45.008 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:45.003+0000 7f6d16429700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:45.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:44 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:44.778+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:45.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:44 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:44.778+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:45.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:44.721+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:45.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:44 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:44.721+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:45.302 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:45.003+0000 7f6d16429700 -1 osd.6 95 *** Got signal Terminated *** 2023-10-16T14:46:45.303 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:45.003+0000 7f6d16429700 -1 osd.6 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:45.303 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:45.188+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:45.303 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:45.188+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:45.303 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:45.188+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:45.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:45.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:45.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:45.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:45.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:45.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:45.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:45 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:45.408+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:45.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:45 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:45.408+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:45.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:45 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:45.408+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:46.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:45.669+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:46.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:45 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:45.669+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:46.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:45.792+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:46.076 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:45.792+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:46.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:45 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:45.679+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:46.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:45 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:45.679+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:46.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:46.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:46.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:46.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:46.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:46.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:46.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:46.250+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:46.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:46.250+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:46.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:46.250+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:46.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:46.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:46.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:46.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:46.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:46.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:47.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:46.676+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:47.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:46 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:46.676+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:47.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:46.680+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:47.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:46 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:46.680+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:47.208 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:46.835+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:47.208 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:46 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:46.835+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:47.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:47.219+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:47.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:47.219+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:47.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:47.219+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:47.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:47 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:47.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:47.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:47 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:47.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:47.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:47 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:47.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:47.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:47.405+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:47.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:47.405+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:47.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:47.405+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:48.052 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:47.707+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:48.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:47 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:47.707+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:48.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:47.714+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:48.176 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:47 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:47.714+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:48.235 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:47 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:47.839+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:48.236 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:47 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:47.839+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:48.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:48.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:48.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:48.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:48.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:48.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:48.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:48.221+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:48.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:48.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:48.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:48.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:48.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:48.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:48.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:48.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:48.712 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:48 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:48.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:48.713 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:48 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:48.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:48.713 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:48 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:48.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:49.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:48.694+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:49.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:48.694+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:49.054 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:48 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:48.694+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.342995+0000 front 2023-10-16T14:46:22.342926+0000 (oldest deadline 2023-10-16T14:46:48.143947+0000) 2023-10-16T14:46:49.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:48 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:48.709+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:49.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:48 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:48.709+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:49.218 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:48.878+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:49.218 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:48 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:48.878+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:49.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:49.266+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:49.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:49.266+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:49.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:49.266+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:49.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:49.266+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:49.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:49.214+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:49.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:49.214+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:49.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:49.214+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:49.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:49.214+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:49.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:49 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:49.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:49.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:49 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:49.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:49.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:49 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:49.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:50.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:49.652+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:05.439611+0000 front 2023-10-16T14:46:05.439693+0000 (oldest deadline 2023-10-16T14:46:31.840443+0000) 2023-10-16T14:46:50.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:49.652+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.642225+0000 front 2023-10-16T14:46:17.642108+0000 (oldest deadline 2023-10-16T14:46:42.342315+0000) 2023-10-16T14:46:50.053 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:49 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6[133898]: 2023-10-16T14:46:49.652+0000 7f6d1221e700 -1 osd.6 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.342995+0000 front 2023-10-16T14:46:22.342926+0000 (oldest deadline 2023-10-16T14:46:48.143947+0000) 2023-10-16T14:46:50.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:49 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:49.753+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:50.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:49 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:49.753+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:50.247 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:49.923+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:50.247 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:49 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:49.923+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:50.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:50 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:50.262+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:50.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:50 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:50.262+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:50.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:50 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:50.262+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:50.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:50 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:50.262+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:50.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:50.244+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:50.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:50.244+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:50.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:50.244+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:50.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:50.244+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:50.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:50 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:50.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:50.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:50 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:50.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:50.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:50 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:50.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:51.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:50 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:50.704+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:51.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:50 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:50.704+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:51.221 INFO:journalctl@ceph.osd.6.smithi178.stdout:Oct 16 14:46:50 smithi178 bash[179180]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-6 2023-10-16T14:46:51.230 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:50.941+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:51.230 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:50.941+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:51.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:51.217+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:51.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:51.217+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:51.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:51.217+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:51.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:51 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:51.217+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:51.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:51.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:51.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:51.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:51.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:51.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:51.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:51.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:51.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:51 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:51.378+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:51.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:51 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:51.378+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:51.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:51 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:51.378+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:52.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:51 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:51.719+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:52.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:51 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:51.719+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:52.184 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:51.930+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:52.185 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:51.930+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:52.185 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:51 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:51.930+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:52.511 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:52 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:52.206+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:52.511 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:52 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:52.206+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:52.512 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:52 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:52.206+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:52.512 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:52 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:52.206+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:52.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:52.181+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:52.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:52.181+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:52.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:52.181+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:52.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:52.181+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:52.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:52.334+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:52.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:52.334+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:52.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:52.334+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:52.857 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.6.service' 2023-10-16T14:46:52.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:46:52.947 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-10-16T14:46:52.947 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-10-16T14:46:52.947 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.7 2023-10-16T14:46:53.174 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:52.686+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:53.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:52.686+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:53.175 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:52 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:52.686+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:53.314 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:53.134+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:53.314 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:53.134+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:53.315 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:53.134+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:53.315 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:53.134+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:53.315 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:52.892+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:53.315 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:52.892+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:53.315 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:52 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:52.892+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:53.316 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:52 smithi186 systemd[1]: Stopping Ceph osd.7 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:46:53.316 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.311+0000 7f05d6138700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:46:53.316 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.311+0000 7f05d6138700 -1 osd.7 95 *** Got signal Terminated *** 2023-10-16T14:46:53.316 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.311+0000 7f05d6138700 -1 osd.7 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:46:53.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:53 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:53.209+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:53.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:53 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:53.209+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:53.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:53 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:53.209+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:53.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:53 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:53.209+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:53.647 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:53.364+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:53.647 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:53.364+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:53.648 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:53.364+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:53.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:53.643+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:53.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:53.643+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:53.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:53 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:53.644+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:54.146 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.865+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:54.147 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.865+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:54.147 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:53 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:53.865+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:54.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:54.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:54.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:54.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:54.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:54.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:54.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:54 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:54.232+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:54.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:54.143+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:54.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:54.143+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:54.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:54.143+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:54.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:54.143+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:54.675 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:54.654+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:54.675 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:54.654+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:54.675 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:54.654+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:54.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:54.409+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:54.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:54.409+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:54.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:54 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:54.409+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:55.124 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:54.845+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:55.125 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:54.845+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:55.125 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:54 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:54.845+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:55.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:55.279+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:55.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:55.279+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:55.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:55.279+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:55.553 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:55 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:55.279+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:55.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:55.121+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:55.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:55.121+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:55.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:55.121+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:55.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:55.121+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:55.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:55.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:55.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:55.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:55.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:55.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:55.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:55.673+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:55.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:55.673+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:55.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:55.673+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:55.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:55 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:55.673+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:46:56.132 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:55.857+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:56.133 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:55.857+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:56.133 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:55.857+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:56.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:56.129+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:56.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:56.129+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:56.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:56.129+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:56.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:56.129+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:56.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:56 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:56.324+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:56.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:56 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:56.324+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:56.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:56 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:56.324+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:56.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:56 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:56.324+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:56.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:56.674+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:56.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:56.674+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:56.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:56.674+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:56.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:56.674+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:46:56.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:56.440+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:56.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:56.440+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:56.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:56.440+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:56.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:56 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:56.440+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:46:57.181 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:56.846+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:57.182 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:56.846+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:57.182 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:56 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:56.846+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:57.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:57.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:57.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:57.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:57.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:57.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:57.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:57.178+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:57.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:57.321+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:57.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:57.321+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:57.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:57.321+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:57.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:57 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:57.321+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:57.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:57.697+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:57.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:57.697+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:57.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:57.697+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:57.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:57.697+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:46:57.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:57.480+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:57.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:57.480+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:57.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:57.480+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:57.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:57 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:57.480+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:46:58.230 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:57.870+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:07.204122+0000 front 2023-10-16T14:46:07.204153+0000 (oldest deadline 2023-10-16T14:46:32.504285+0000) 2023-10-16T14:46:58.231 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:57.870+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.205583+0000 front 2023-10-16T14:46:17.205562+0000 (oldest deadline 2023-10-16T14:46:41.905767+0000) 2023-10-16T14:46:58.231 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:57.870+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.006916+0000 front 2023-10-16T14:46:26.006877+0000 (oldest deadline 2023-10-16T14:46:51.906886+0000) 2023-10-16T14:46:58.231 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:57 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7[155789]: 2023-10-16T14:46:57.870+0000 7f05d072a700 -1 osd.7 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:33.607844+0000 front 2023-10-16T14:46:33.607878+0000 (oldest deadline 2023-10-16T14:46:57.108086+0000) 2023-10-16T14:46:58.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:58.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:58.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:58.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:58.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:58.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:58.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:58.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:58.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:58 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:58.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:46:58.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:58.332+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:58.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:58.332+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:58.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:58.332+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:58.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:58 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:58.332+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:58.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:58.652+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:58.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:58.652+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:58.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:58.652+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:58.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:58.652+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:46:58.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:58.472+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:58.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:58.472+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:58.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:58.472+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:58.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:58 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:58.472+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:46:59.187 INFO:journalctl@ceph.osd.7.smithi186.stdout:Oct 16 14:46:59 smithi186 bash[176794]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-7 2023-10-16T14:46:59.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:59.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:46:59.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:59.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:46:59.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:59.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:46:59.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:59.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:46:59.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:46:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:46:59.227+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:46:59.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:59.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:46:59.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:59.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:46:59.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:59.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:46:59.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:59.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:46:59.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:46:59 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:46:59.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:46:59.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:59.627+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:46:59.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:59.627+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:46:59.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:59.627+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:46:59.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:46:59.627+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:46:59.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:59.423+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:46:59.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:59.423+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:46:59.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:59.423+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:46:59.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:46:59 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:46:59.423+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:00.440 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:00.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:00.441 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:00.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:00.441 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:00.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:00.441 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:00.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:00.441 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:00 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:00.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:00.447 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.7.service' 2023-10-16T14:47:00.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:00.488 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-10-16T14:47:00.488 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-10-16T14:47:00.488 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.8 2023-10-16T14:47:00.649 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 systemd[1]: Stopping Ceph osd.8 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:00.649 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:00.380+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:00.650 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:00.380+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:00.650 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:00.380+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:00.650 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:00.380+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:00.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:00.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:00.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:00.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:00.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:00.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:00.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:00.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:00.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:00 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:00.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:00.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.645+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:00.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.645+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:00.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.645+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:00.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.645+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:00.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.850+0000 7f61c65e5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:00.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.850+0000 7f61c65e5700 -1 osd.8 95 *** Got signal Terminated *** 2023-10-16T14:47:00.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:00 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:00.850+0000 7f61c65e5700 -1 osd.8 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:47:01.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:01 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:01.269+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:01.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:01 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:01.269+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:01.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:01 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:01.269+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:01.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:01 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:01.269+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:01.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:01 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:01.269+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:01.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:01.396+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:01.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:01.396+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:01.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:01.396+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:01.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:01.396+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:01.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:01.636+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:01.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:01.636+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:01.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:01.636+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:01.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:01 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:01.636+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:01.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:01.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:01.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:01.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:01.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:01.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:01.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:01.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:01.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:01 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:01.346+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:02.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:02 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:02.292+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:02.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:02 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:02.292+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:02.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:02 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:02.292+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:02.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:02 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:02.292+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:02.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:02 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:02.292+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:02.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:02.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:02.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:02.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:02.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:02.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:02.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:02.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:02.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:02.628+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:02.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:02.628+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:02.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:02.628+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:02.676 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:02.628+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:02.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:02.392+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:02.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:02.392+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:02.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:02.392+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:02.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:02.392+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:02.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:02 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:02.392+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:03.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:03 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:03.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:03.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:03 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:03.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:03.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:03 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:03.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:03.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:03 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:03.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:03.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:03 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:03.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:03.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:03.440+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:03.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:03.440+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:03.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:03.440+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:03.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:03.440+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:03.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:03 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:03.440+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:03.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:03.427+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:03.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:03.427+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:03.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:03.427+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:03.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:03.427+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:03.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:03.599+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:03.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:03.599+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:03.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:03.599+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:03.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:03.599+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:03.927 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:03 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:03.599+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.396177+0000 front 2023-10-16T14:46:41.396202+0000 (oldest deadline 2023-10-16T14:47:03.096294+0000) 2023-10-16T14:47:04.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:04 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:04.255+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:04.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:04 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:04.255+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:04.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:04 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:04.255+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:04.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:04 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:04.255+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:04.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:04 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:04.255+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:04.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:04.398+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:04.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:04.398+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:04.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:04.398+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:04.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:04.398+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:04.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:04 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:04.398+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:04.924 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:04.589+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:04.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:04.589+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:04.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:04.589+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:04.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:04.589+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:04.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:04.589+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.396177+0000 front 2023-10-16T14:46:41.396202+0000 (oldest deadline 2023-10-16T14:47:03.096294+0000) 2023-10-16T14:47:04.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:04.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:04.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:04.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:04.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:04.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:04.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:04.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:04.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:04 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:04.459+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:05.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:05.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:05.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:05.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:05.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:05.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:05 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:05.247+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:05.805 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:05.388+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:05.805 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:05.388+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:05.806 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:05.388+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:05.806 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:05.388+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:05.806 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:05 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:05.388+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:05.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:05.555+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.187787+0000 front 2023-10-16T14:46:00.187823+0000 (oldest deadline 2023-10-16T14:46:25.487817+0000) 2023-10-16T14:47:05.925 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:05.555+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.392971+0000 front 2023-10-16T14:46:17.392999+0000 (oldest deadline 2023-10-16T14:46:40.293230+0000) 2023-10-16T14:47:05.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:05.555+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:26.194268+0000 front 2023-10-16T14:46:26.194221+0000 (oldest deadline 2023-10-16T14:46:52.094327+0000) 2023-10-16T14:47:05.926 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:05.555+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.094995+0000 front 2023-10-16T14:46:32.094844+0000 (oldest deadline 2023-10-16T14:46:55.594965+0000) 2023-10-16T14:47:05.927 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8[157557]: 2023-10-16T14:47:05.555+0000 7f61c0bd7700 -1 osd.8 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.396177+0000 front 2023-10-16T14:46:41.396202+0000 (oldest deadline 2023-10-16T14:47:03.096294+0000) 2023-10-16T14:47:05.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:05.442+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:05.927 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:05.442+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:05.928 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:05.442+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:05.928 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:05.442+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:05.929 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:05 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:05.442+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:06.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:06.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:06.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:06.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:06.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:06.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:06 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:06.262+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:06.675 INFO:journalctl@ceph.osd.8.smithi187.stdout:Oct 16 14:47:06 smithi187 bash[179902]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-8 2023-10-16T14:47:06.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:06.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:06.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:06.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:06.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:06.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:06.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:06.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:06.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:06 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:06.397+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:06.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:06.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:06.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:06.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:06.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:06.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:06 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:06.354+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:07.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:07.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:07.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:07.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:07.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:07.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:07 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:07.270+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:07.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:07.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:07.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:07.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:07.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:07.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:07.393+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:07.855 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:07.432+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:07.856 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:07.432+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:07.856 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:07.432+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:07.856 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:07.432+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:07.856 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:07.432+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:07.862 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.8.service' 2023-10-16T14:47:07.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:07.903 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-10-16T14:47:07.903 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-10-16T14:47:07.903 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.9 2023-10-16T14:47:08.302 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:07 smithi178 systemd[1]: Stopping Ceph osd.9 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:08.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:08.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:08.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:08.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:08.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:08.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:08 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:08.289+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:08.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:08 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:08.383+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:08.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:08 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:08.383+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:08.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:08 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:08.383+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:08.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:08 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:08.383+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:08.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:08 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:08.383+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:08.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:08.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:08.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:08.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:08.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:08.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.406+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:08.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.425+0000 7f1dcf994700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:08.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.425+0000 7f1dcf994700 -1 osd.9 95 *** Got signal Terminated *** 2023-10-16T14:47:08.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:08 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:08.425+0000 7f1dcf994700 -1 osd.9 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:47:09.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:09.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:09.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:09.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:09.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:09.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:09 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:09.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:09.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:09.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:09.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:09.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:09.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:09.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:09 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:09.439+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:09.924 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:09 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:09.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:09.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:09 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:09.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:09.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:09 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:09.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:09.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:09 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:09.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:09.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:09 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:09.433+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:10.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:10.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:10.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:10.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:10.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:10.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:10 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:10.260+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:10.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:10.410+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:10.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:10.410+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:10.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:10.410+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:10.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:10.410+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:10.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:10 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:10.410+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:10.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:10.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:10.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:10.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:10.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:10.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:10 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:10.433+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:11.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:11.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:11.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:11.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:11.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:11.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:11 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:11.242+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:11.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:11.399+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:11.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:11.399+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:11.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:11.399+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:11.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:11.399+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:11.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:11.399+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:11.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:11.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:11.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:11.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:11.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:11.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:11 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:11.410+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:12.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:12.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:12.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:12.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:12.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:12.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:12 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:12.199+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:12.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:12.377+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:12.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:12.377+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:12.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:12.377+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:12.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:12.377+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:12.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:12 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:12.377+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:12.802 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:01.238429+0000 front 2023-10-16T14:46:01.238378+0000 (oldest deadline 2023-10-16T14:46:26.538730+0000) 2023-10-16T14:47:12.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:06.539484+0000 front 2023-10-16T14:46:06.539624+0000 (oldest deadline 2023-10-16T14:46:32.439649+0000) 2023-10-16T14:47:12.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.943032+0000 front 2023-10-16T14:46:16.943200+0000 (oldest deadline 2023-10-16T14:46:42.243212+0000) 2023-10-16T14:47:12.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:22.243658+0000 front 2023-10-16T14:46:22.243684+0000 (oldest deadline 2023-10-16T14:46:48.143800+0000) 2023-10-16T14:47:12.803 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:34.045213+0000 front 2023-10-16T14:46:34.045355+0000 (oldest deadline 2023-10-16T14:46:59.246290+0000) 2023-10-16T14:47:12.804 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:12 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9[141767]: 2023-10-16T14:47:12.429+0000 7f1dc9f86700 -1 osd.9 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:41.547535+0000 front 2023-10-16T14:46:41.547608+0000 (oldest deadline 2023-10-16T14:47:05.647366+0000) 2023-10-16T14:47:13.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:13.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:13.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:13.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:13.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:13.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:13.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:13 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:13.204+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:13.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:13.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:13.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:13.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:13.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:13.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:13 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:13.424+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:14.552 INFO:journalctl@ceph.osd.9.smithi178.stdout:Oct 16 14:47:14 smithi178 bash[179454]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-9 2023-10-16T14:47:14.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:14.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:14.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:14.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:14.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:14.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:14.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:14 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:14.156+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:14.924 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:14.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:14.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:14.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:14.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:14.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:14 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:14.426+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:15.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:15.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:15.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:15.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:15.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:15.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:15.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:15 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:15.192+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:15.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.436+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:15.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.436+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:15.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.437+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:15.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.437+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:15.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.437+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:15.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:15.437+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:16.193 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.9.service' 2023-10-16T14:47:16.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:16.281 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-10-16T14:47:16.282 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-10-16T14:47:16.282 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.10 2023-10-16T14:47:16.575 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:16.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:16.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:16.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:16.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:16.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:16.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.224+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:16.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 systemd[1]: Stopping Ceph osd.10 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:16.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:16.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:16.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:16.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:16.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:16.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:16 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:16.395+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:17.076 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.633+0000 7fcb3c5cc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:17.076 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.633+0000 7fcb3c5cc700 -1 osd.10 95 *** Got signal Terminated *** 2023-10-16T14:47:17.076 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:16 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:16.633+0000 7fcb3c5cc700 -1 osd.10 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:47:17.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:17.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:17.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:17.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:17.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:17.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:17.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:17 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:17.272+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:17.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:17.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:17.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:17.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:17.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:17.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:17 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:17.371+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:18.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:18.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:18.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:18.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:18.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:18.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:18.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:18 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:18.296+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:18.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:18.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:18.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:18.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:18.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:18.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:18 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:18.386+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:19.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:19.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:19.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:19.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:19.578 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:19.578 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:19.578 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:19 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:19.265+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:19.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:19.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:19.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:19.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:19.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:19.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:19 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:19.416+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:20.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:20.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:20.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:20.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:20.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:20.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:20.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:20 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:20.232+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:20.924 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:20.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:20.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:20.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:20.925 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:20.926 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:20 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:20.447+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:21.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:46:00.758846+0000 front 2023-10-16T14:46:00.758815+0000 (oldest deadline 2023-10-16T14:46:24.258764+0000) 2023-10-16T14:47:21.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.559773+0000 front 2023-10-16T14:46:09.559839+0000 (oldest deadline 2023-10-16T14:46:34.859953+0000) 2023-10-16T14:47:21.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:16.561334+0000 front 2023-10-16T14:46:16.561211+0000 (oldest deadline 2023-10-16T14:46:44.762015+0000) 2023-10-16T14:47:21.576 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6814 osd.3 since back 2023-10-16T14:46:24.762579+0000 front 2023-10-16T14:46:24.762660+0000 (oldest deadline 2023-10-16T14:46:47.662589+0000) 2023-10-16T14:47:21.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:32.665070+0000 front 2023-10-16T14:46:32.665100+0000 (oldest deadline 2023-10-16T14:46:57.965226+0000) 2023-10-16T14:47:21.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:40.866399+0000 front 2023-10-16T14:46:40.866305+0000 (oldest deadline 2023-10-16T14:47:04.366172+0000) 2023-10-16T14:47:21.577 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:21 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10[163889]: 2023-10-16T14:47:21.206+0000 7fcb37bc0700 -1 osd.10 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:48.967677+0000 front 2023-10-16T14:46:48.967771+0000 (oldest deadline 2023-10-16T14:47:13.067608+0000) 2023-10-16T14:47:21.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:21.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:21.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:21.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:21.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:21.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:21.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:21 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:21.404+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:22.421 INFO:journalctl@ceph.osd.10.smithi186.stdout:Oct 16 14:47:22 smithi186 bash[177054]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-10 2023-10-16T14:47:22.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:22.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:22.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:22.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:22.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:22.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:22.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:22 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:22.359+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:23.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:23.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:23.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:23.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:23.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:23.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:23.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:23.373+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:23.788 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.10.service' 2023-10-16T14:47:23.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:23.829 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-10-16T14:47:23.829 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-10-16T14:47:23.829 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.11 2023-10-16T14:47:24.179 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:23 smithi187 systemd[1]: Stopping Ceph osd.11 for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:24.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.204+0000 7f2f3707b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:24.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.204+0000 7f2f3707b700 -1 osd.11 95 *** Got signal Terminated *** 2023-10-16T14:47:24.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.204+0000 7f2f3707b700 -1 osd.11 95 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-10-16T14:47:24.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:24.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:24.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:24.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:24.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:24.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:24.677 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:24 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:24.350+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:25.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:25.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:25.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:25.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:25.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:25.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:25.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:25 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:25.342+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:26.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:26.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:26.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:26.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:26.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:26.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:26.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:26 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:26.316+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:27.674 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:27.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:27.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:27.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:27.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:27.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:27.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:27.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:27 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:27.338+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2023-10-16T14:47:00.684882+0000 front 2023-10-16T14:47:00.684898+0000 (oldest deadline 2023-10-16T14:47:26.584918+0000) 2023-10-16T14:47:28.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6806 osd.0 since back 2023-10-16T14:45:57.675558+0000 front 2023-10-16T14:45:57.675682+0000 (oldest deadline 2023-10-16T14:46:24.076220+0000) 2023-10-16T14:47:28.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6804 osd.1 since back 2023-10-16T14:46:09.977474+0000 front 2023-10-16T14:46:09.977432+0000 (oldest deadline 2023-10-16T14:46:31.677360+0000) 2023-10-16T14:47:28.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6804 osd.2 since back 2023-10-16T14:46:17.578295+0000 front 2023-10-16T14:46:17.578350+0000 (oldest deadline 2023-10-16T14:46:42.878466+0000) 2023-10-16T14:47:28.675 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6812 osd.4 since back 2023-10-16T14:46:29.780500+0000 front 2023-10-16T14:46:29.780474+0000 (oldest deadline 2023-10-16T14:46:55.581213+0000) 2023-10-16T14:47:28.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6812 osd.5 since back 2023-10-16T14:46:39.082023+0000 front 2023-10-16T14:46:39.082045+0000 (oldest deadline 2023-10-16T14:47:04.282555+0000) 2023-10-16T14:47:28.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.178:6822 osd.6 since back 2023-10-16T14:46:47.183824+0000 front 2023-10-16T14:46:47.183746+0000 (oldest deadline 2023-10-16T14:47:12.483405+0000) 2023-10-16T14:47:28.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.186:6820 osd.7 since back 2023-10-16T14:46:54.784273+0000 front 2023-10-16T14:46:54.784295+0000 (oldest deadline 2023-10-16T14:47:20.684365+0000) 2023-10-16T14:47:28.676 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:28 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11[166362]: 2023-10-16T14:47:28.329+0000 7f2f31e6e700 -1 osd.11 95 heartbeat_check: no reply from 172.21.15.187:6820 osd.8 since back 2023-10-16T14:47:00.684882+0000 front 2023-10-16T14:47:00.684898+0000 (oldest deadline 2023-10-16T14:47:26.584918+0000) 2023-10-16T14:47:29.877 INFO:journalctl@ceph.osd.11.smithi187.stdout:Oct 16 14:47:29 smithi187 bash[180170]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-osd-11 2023-10-16T14:47:31.243 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@osd.11.service' 2023-10-16T14:47:31.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:31.286 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-10-16T14:47:31.287 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-10-16T14:47:31.287 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.a 2023-10-16T14:47:31.802 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:47:31 smithi178 systemd[1]: Stopping Ceph mds.a for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:32.302 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:47:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-a[149595]: 2023-10-16T14:47:31.803+0000 7fb59c4da700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:32.303 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:47:31 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-a[149595]: 2023-10-16T14:47:31.803+0000 7fb59c4da700 -1 mds.a *** got signal Terminated *** 2023-10-16T14:47:33.159 INFO:journalctl@ceph.mds.a.smithi178.stdout:Oct 16 14:47:33 smithi178 bash[179725]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-a 2023-10-16T14:47:33.456 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.a.service' 2023-10-16T14:47:33.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:33.543 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-10-16T14:47:33.544 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-10-16T14:47:33.544 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.d 2023-10-16T14:47:33.802 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:47:33 smithi178 systemd[1]: Stopping Ceph mds.d for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:34.555 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:47:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-d[148589]: 2023-10-16T14:47:34.053+0000 7f43b5455700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:34.555 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:47:34 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-d[148589]: 2023-10-16T14:47:34.053+0000 7f43b5455700 -1 mds.d *** got signal Terminated *** 2023-10-16T14:47:36.052 INFO:journalctl@ceph.mds.d.smithi178.stdout:Oct 16 14:47:35 smithi178 bash[179865]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-d 2023-10-16T14:47:36.275 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.d.service' 2023-10-16T14:47:36.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:36.358 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-10-16T14:47:36.359 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-10-16T14:47:36.359 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.g 2023-10-16T14:47:36.552 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:47:36 smithi178 systemd[1]: Stopping Ceph mds.g for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:37.303 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:47:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-g[149204]: 2023-10-16T14:47:36.868+0000 7f9cdd815700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:37.303 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:47:36 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-g[149204]: 2023-10-16T14:47:36.868+0000 7f9cdd815700 -1 mds.g *** got signal Terminated *** 2023-10-16T14:47:41.303 INFO:journalctl@ceph.mds.g.smithi178.stdout:Oct 16 14:47:41 smithi178 bash[180012]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-g 2023-10-16T14:47:41.550 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.g.service' 2023-10-16T14:47:41.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:41.637 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-10-16T14:47:41.638 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-10-16T14:47:41.638 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.j 2023-10-16T14:47:41.802 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:41 smithi178 systemd[1]: Stopping Ceph mds.j for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:42.552 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-j[148198]: 2023-10-16T14:47:42.168+0000 7f3e9e6a8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:42.553 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:42 smithi178 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-j[148198]: 2023-10-16T14:47:42.168+0000 7f3e9e6a8700 -1 mds.j *** got signal Terminated *** 2023-10-16T14:47:44.906 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:44 smithi178 bash[180165]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-j 2023-10-16T14:47:45.118 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.j.service' 2023-10-16T14:47:45.179 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:45 smithi178 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.j.service: Succeeded. 2023-10-16T14:47:45.179 INFO:journalctl@ceph.mds.j.smithi178.stdout:Oct 16 14:47:45 smithi178 systemd[1]: Stopped Ceph mds.j for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:47:45.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:45.201 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-10-16T14:47:45.202 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-10-16T14:47:45.202 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.b 2023-10-16T14:47:45.557 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:47:45 smithi186 systemd[1]: Stopping Ceph mds.b for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:45.826 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:47:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: 2023-10-16T14:47:45.553+0000 7f2bc24cf700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:45.826 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:47:45 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b[170994]: 2023-10-16T14:47:45.553+0000 7f2bc24cf700 -1 mds.b *** got signal Terminated *** 2023-10-16T14:47:50.009 INFO:journalctl@ceph.mds.b.smithi186.stdout:Oct 16 14:47:49 smithi186 bash[177329]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-b 2023-10-16T14:47:50.014 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.b.service' 2023-10-16T14:47:50.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:50.050 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-10-16T14:47:50.050 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-10-16T14:47:50.051 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.e 2023-10-16T14:47:50.325 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:47:50 smithi186 systemd[1]: Stopping Ceph mds.e for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:50.826 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:47:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: 2023-10-16T14:47:50.419+0000 7f4c2b34c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:50.826 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:47:50 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e[170302]: 2023-10-16T14:47:50.419+0000 7f4c2b34c700 -1 mds.e *** got signal Terminated *** 2023-10-16T14:47:54.575 INFO:journalctl@ceph.mds.e.smithi186.stdout:Oct 16 14:47:54 smithi186 bash[177467]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-e 2023-10-16T14:47:54.786 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.e.service' 2023-10-16T14:47:54.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:54.821 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-10-16T14:47:54.821 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-10-16T14:47:54.821 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.h 2023-10-16T14:47:55.076 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:47:54 smithi186 systemd[1]: Stopping Ceph mds.h for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:55.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:47:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: 2023-10-16T14:47:55.202+0000 7f8f69d34700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:55.576 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:47:55 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h[170643]: 2023-10-16T14:47:55.202+0000 7f8f69d34700 -1 mds.h *** got signal Terminated *** 2023-10-16T14:47:58.567 INFO:journalctl@ceph.mds.h.smithi186.stdout:Oct 16 14:47:58 smithi186 bash[177607]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-h 2023-10-16T14:47:58.751 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.h.service' 2023-10-16T14:47:58.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:47:58.785 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-10-16T14:47:58.785 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-10-16T14:47:58.785 DEBUG:teuthology.orchestra.run.smithi186:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.k 2023-10-16T14:47:59.076 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:47:58 smithi186 systemd[1]: Stopping Ceph mds.k for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:47:59.578 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:47:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-k[169961]: 2023-10-16T14:47:59.152+0000 7f31e461b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:47:59.578 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:47:59 smithi186 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-k[169961]: 2023-10-16T14:47:59.152+0000 7f31e461b700 -1 mds.k *** got signal Terminated *** 2023-10-16T14:48:02.076 INFO:journalctl@ceph.mds.k.smithi186.stdout:Oct 16 14:48:01 smithi186 bash[177743]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-k 2023-10-16T14:48:02.132 DEBUG:teuthology.orchestra.run.smithi186:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.k.service' 2023-10-16T14:48:02.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:48:02.167 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-10-16T14:48:02.168 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-10-16T14:48:02.168 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.c 2023-10-16T14:48:02.525 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:48:02 smithi187 systemd[1]: Stopping Ceph mds.c for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:48:02.924 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:48:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: 2023-10-16T14:48:02.521+0000 7f563999f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:48:02.925 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:48:02 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c[173218]: 2023-10-16T14:48:02.521+0000 7f563999f700 -1 mds.c *** got signal Terminated *** 2023-10-16T14:48:06.674 INFO:journalctl@ceph.mds.c.smithi187.stdout:Oct 16 14:48:06 smithi187 bash[180441]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-c 2023-10-16T14:48:06.721 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.c.service' 2023-10-16T14:48:06.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:48:06.759 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-10-16T14:48:06.759 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-10-16T14:48:06.759 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.f 2023-10-16T14:48:07.149 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:06 smithi187 systemd[1]: Stopping Ceph mds.f for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:48:07.424 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: 2023-10-16T14:48:07.146+0000 7f2c3d163700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:48:07.425 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:07 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f[172877]: 2023-10-16T14:48:07.146+0000 7f2c3d163700 -1 mds.f *** got signal Terminated *** 2023-10-16T14:48:10.789 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.f.service' 2023-10-16T14:48:10.813 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:10 smithi187 bash[180591]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-f 2023-10-16T14:48:10.813 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:10 smithi187 systemd[1]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.f.service: Succeeded. 2023-10-16T14:48:10.814 INFO:journalctl@ceph.mds.f.smithi187.stdout:Oct 16 14:48:10 smithi187 systemd[1]: Stopped Ceph mds.f for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7. 2023-10-16T14:48:10.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:48:10.824 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-10-16T14:48:10.825 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-10-16T14:48:10.825 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.i 2023-10-16T14:48:11.174 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:48:10 smithi187 systemd[1]: Stopping Ceph mds.i for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:48:11.674 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:48:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:48:11.219+0000 7fc8c14bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:48:11.675 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:48:11 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i[173557]: 2023-10-16T14:48:11.219+0000 7fc8c14bb700 -1 mds.i *** got signal Terminated *** 2023-10-16T14:48:14.841 INFO:journalctl@ceph.mds.i.smithi187.stdout:Oct 16 14:48:14 smithi187 bash[180732]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-i 2023-10-16T14:48:14.846 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.i.service' 2023-10-16T14:48:14.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:48:14.879 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-10-16T14:48:14.880 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-10-16T14:48:14.880 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.l 2023-10-16T14:48:15.174 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:48:14 smithi187 systemd[1]: Stopping Ceph mds.l for 8c31bde4-6c2f-11ee-8db6-212e2dc638e7... 2023-10-16T14:48:15.675 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:48:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-l[172535]: 2023-10-16T14:48:15.253+0000 7f4410171700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-10-16T14:48:15.675 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:48:15 smithi187 ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-l[172535]: 2023-10-16T14:48:15.253+0000 7f4410171700 -1 mds.l *** got signal Terminated *** 2023-10-16T14:48:19.599 INFO:journalctl@ceph.mds.l.smithi187.stdout:Oct 16 14:48:19 smithi187 bash[180880]: ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7-mds-l 2023-10-16T14:48:19.743 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c31bde4-6c2f-11ee-8db6-212e2dc638e7@mds.l.service' 2023-10-16T14:48:19.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-10-16T14:48:19.779 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-10-16T14:48:19.780 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-16T14:48:20.176 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:48:23.391 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-16T14:48:23.701 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:48:25.332 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force --keep-logs 2023-10-16T14:48:25.640 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:48:27.222 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:48:27.300 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:48:27.336 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-10-16T14:48:27.366 INFO:tasks.cephadm:Archiving crash dumps... 2023-10-16T14:48:27.368 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi178/crash 2023-10-16T14:48:27.369 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash -- . 2023-10-16T14:48:27.437 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-16T14:48:27.437 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2023-10-16T14:48:27.441 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi186/crash 2023-10-16T14:48:27.442 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash -- . 2023-10-16T14:48:27.494 INFO:teuthology.orchestra.run.smithi186.stderr:tar: /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-16T14:48:27.495 INFO:teuthology.orchestra.run.smithi186.stderr:tar: Error is not recoverable: exiting now 2023-10-16T14:48:27.496 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi187/crash 2023-10-16T14:48:27.497 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash -- . 2023-10-16T14:48:27.548 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/crash: Cannot open: No such file or directory 2023-10-16T14:48:27.548 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2023-10-16T14:48:27.549 INFO:tasks.cephadm:Checking cluster log for badness... 2023-10-16T14:48:27.550 DEBUG:teuthology.orchestra.run.smithi178:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-10-16T14:48:27.627 INFO:teuthology.orchestra.run.smithi178.stderr:grep: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph.log: No such file or directory 2023-10-16T14:48:27.632 INFO:tasks.cephadm:Compressing logs... 2023-10-16T14:48:27.632 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T14:48:27.637 DEBUG:teuthology.orchestra.run.smithi186:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T14:48:27.639 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T14:48:27.673 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-10-16T14:48:27.673 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-16T14:48:27.674 INFO:teuthology.orchestra.run.smithi186.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-16T14:48:27.674 INFO:teuthology.orchestra.run.smithi186.stderr:: No such file or directory 2023-10-16T14:48:27.681 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.5.log 2023-10-16T14:48:27.682 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.2.log 2023-10-16T14:48:27.682 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.11.log 2023-10-16T14:48:27.682 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.5.log: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.z.log 2023-10-16T14:48:27.682 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.8.log 2023-10-16T14:48:27.685 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-16T14:48:27.685 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.f.log 2023-10-16T14:48:27.686 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.c.log 2023-10-16T14:48:27.686 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.i.log 2023-10-16T14:48:27.686 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.c.log 2023-10-16T14:48:27.686 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-16T14:48:27.686 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.l.log 2023-10-16T14:48:27.688 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.10.log 2023-10-16T14:48:27.688 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.h.log 2023-10-16T14:48:27.688 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.e.log 2023-10-16T14:48:27.688 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.4.log 2023-10-16T14:48:27.688 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.1.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.k.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.7.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.b.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.b.log 2023-10-16T14:48:27.689 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.7.log: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.y.log 2023-10-16T14:48:27.690 INFO:teuthology.orchestra.run.smithi186.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.b.log: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.k.log.gz 2023-10-16T14:48:27.690 INFO:teuthology.orchestra.run.smithi186.stderr: 92.8% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.y.log.gz 2023-10-16T14:48:27.691 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.l.log: 92.9% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.z.log.gz 2023-10-16T14:48:27.691 INFO:teuthology.orchestra.run.smithi187.stderr: 94.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.l.log.gz 2023-10-16T14:48:27.699 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-10-16T14:48:27.700 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2023-10-16T14:48:27.700 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.0.log 2023-10-16T14:48:27.701 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.g.log 2023-10-16T14:48:27.702 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.a.log 2023-10-16T14:48:27.703 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.6.log 2023-10-16T14:48:27.705 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.3.log 2023-10-16T14:48:27.707 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.d.log 2023-10-16T14:48:27.707 INFO:teuthology.orchestra.run.smithi178.stderr: 94.5% 93.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.g.log.gz 2023-10-16T14:48:27.707 INFO:teuthology.orchestra.run.smithi178.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-16T14:48:27.708 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.3.log: 94.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.a.log.gz 2023-10-16T14:48:27.708 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.x.log 2023-10-16T14:48:27.709 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.9.log 2023-10-16T14:48:27.711 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log 2023-10-16T14:48:27.715 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.9.log: /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.d.log.gz 2023-10-16T14:48:27.715 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.j.log 2023-10-16T14:48:27.718 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.a.log 2023-10-16T14:48:27.720 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-10-16T14:48:27.723 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.log: 91.0% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-16T14:48:27.725 INFO:teuthology.orchestra.run.smithi178.stderr: 94.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.j.log.gz 2023-10-16T14:48:27.726 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-10-16T14:48:27.759 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-10-16T14:48:27.986 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mgr.x.log.gz 2023-10-16T14:48:28.584 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.c.log.gz 2023-10-16T14:48:28.616 INFO:teuthology.orchestra.run.smithi186.stderr: 91.0% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-16T14:48:28.701 INFO:teuthology.orchestra.run.smithi178.stderr: 90.4% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.a.log.gz 2023-10-16T14:48:28.821 INFO:teuthology.orchestra.run.smithi186.stderr: 91.9% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mon.b.log.gz 2023-10-16T14:48:28.983 INFO:teuthology.orchestra.run.smithi187.stderr: 91.0% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-volume.log.gz 2023-10-16T14:48:43.169 INFO:teuthology.orchestra.run.smithi187.stderr: 95.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.f.log.gz 2023-10-16T14:48:56.272 INFO:teuthology.orchestra.run.smithi187.stderr: 91.6% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.8.log.gz 2023-10-16T14:48:56.645 INFO:teuthology.orchestra.run.smithi187.stderr: 95.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.c.log.gz 2023-10-16T14:48:57.396 INFO:teuthology.orchestra.run.smithi178.stderr: 91.1% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.0.log.gz 2023-10-16T14:49:01.111 INFO:teuthology.orchestra.run.smithi186.stderr: 95.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.e.log.gz 2023-10-16T14:49:01.249 INFO:teuthology.orchestra.run.smithi178.stderr: 91.2% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.3.log.gz 2023-10-16T14:49:02.119 INFO:teuthology.orchestra.run.smithi178.stderr: 91.2% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.6.log.gz 2023-10-16T14:49:03.622 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.9.log.gz 2023-10-16T14:49:03.627 INFO:teuthology.orchestra.run.smithi178.stderr: 2023-10-16T14:49:03.627 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m35.953s 2023-10-16T14:49:03.627 INFO:teuthology.orchestra.run.smithi178.stderr:user 2m5.463s 2023-10-16T14:49:03.627 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m9.448s 2023-10-16T14:49:12.151 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.2.log.gz 2023-10-16T14:49:12.235 INFO:teuthology.orchestra.run.smithi187.stderr: 91.4% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.11.log.gz 2023-10-16T14:49:23.230 INFO:teuthology.orchestra.run.smithi187.stderr: 91.3% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.5.log.gz 2023-10-16T14:49:23.756 INFO:teuthology.orchestra.run.smithi186.stderr: 91.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.10.log.gz 2023-10-16T14:49:24.727 INFO:teuthology.orchestra.run.smithi186.stderr: 91.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.1.log.gz 2023-10-16T14:49:37.627 INFO:teuthology.orchestra.run.smithi186.stderr: 91.4% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.7.log.gz 2023-10-16T14:50:02.065 INFO:teuthology.orchestra.run.smithi186.stderr: 91.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-osd.4.log.gz 2023-10-16T14:50:27.300 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.i.log.gz 2023-10-16T14:50:27.302 INFO:teuthology.orchestra.run.smithi187.stderr: 2023-10-16T14:50:27.302 INFO:teuthology.orchestra.run.smithi187.stderr:real 1m59.646s 2023-10-16T14:50:27.302 INFO:teuthology.orchestra.run.smithi187.stderr:user 4m19.665s 2023-10-16T14:50:27.302 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m9.004s 2023-10-16T14:51:33.970 INFO:teuthology.orchestra.run.smithi186.stderr: 93.5% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.h.log.gz 2023-10-16T14:52:56.264 INFO:teuthology.orchestra.run.smithi186.stderr: 93.1% -- replaced with /var/log/ceph/8c31bde4-6c2f-11ee-8db6-212e2dc638e7/ceph-mds.b.log.gz 2023-10-16T14:52:56.274 INFO:teuthology.orchestra.run.smithi186.stderr: 2023-10-16T14:52:56.274 INFO:teuthology.orchestra.run.smithi186.stderr:real 4m28.618s 2023-10-16T14:52:56.274 INFO:teuthology.orchestra.run.smithi186.stderr:user 6m32.325s 2023-10-16T14:52:56.275 INFO:teuthology.orchestra.run.smithi186.stderr:sys 0m15.252s 2023-10-16T14:52:56.275 INFO:tasks.cephadm:Archiving logs... 2023-10-16T14:52:56.275 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi178/log 2023-10-16T14:52:56.277 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T14:53:17.616 DEBUG:teuthology.misc:Transferring archived files from smithi186:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi186/log 2023-10-16T14:53:17.618 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T14:54:19.059 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi187/log 2023-10-16T14:54:19.060 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T14:54:56.938 INFO:tasks.cephadm:Removing cluster... 2023-10-16T14:54:56.939 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force 2023-10-16T14:54:57.335 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:54:58.577 DEBUG:teuthology.orchestra.run.smithi186:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force 2023-10-16T14:54:59.223 INFO:teuthology.orchestra.run.smithi186.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:55:00.127 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 --force 2023-10-16T14:55:00.718 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 8c31bde4-6c2f-11ee-8db6-212e2dc638e7 2023-10-16T14:55:01.509 INFO:tasks.cephadm:Removing cephadm ... 2023-10-16T14:55:01.509 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-16T14:55:01.558 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-16T14:55:01.593 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/cephadm 2023-10-16T14:55:01.621 INFO:tasks.cephadm:Teardown complete 2023-10-16T14:55:01.622 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T14:55:01.634 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T14:55:01.635 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T14:55:01.640 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T14:55:01.642 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T14:55:01.796 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T14:55:01.807 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T14:55:01.807 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:55:01.812 DEBUG:teuthology.orchestra.run.smithi186:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:55:01.814 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T14:55:01.830 INFO:teuthology.orchestra.run.smithi186.stderr:bash: ntpq: command not found 2023-10-16T14:55:01.831 INFO:teuthology.orchestra.run.smithi187.stderr:bash: ntpq: command not found 2023-10-16T14:55:01.850 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-10-16T14:55:01.853 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:55:01.853 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2023-10-16T14:55:01.854 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 +455us[ +454us] +/- 87ms 2023-10-16T14:55:01.854 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 +747us[ +747us] +/- 60ms 2023-10-16T14:55:01.854 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 -1444us[-1444us] +/- 82ms 2023-10-16T14:55:01.854 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:55:01.857 INFO:teuthology.orchestra.run.smithi186.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:55:01.857 INFO:teuthology.orchestra.run.smithi186.stdout:=============================================================================== 2023-10-16T14:55:01.857 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 19 +466us[ +467us] +/- 87ms 2023-10-16T14:55:01.857 INFO:teuthology.orchestra.run.smithi186.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 +734us[ +734us] +/- 60ms 2023-10-16T14:55:01.857 INFO:teuthology.orchestra.run.smithi186.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -1458us[-1457us] +/- 82ms 2023-10-16T14:55:01.858 INFO:teuthology.orchestra.run.smithi186.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:55:01.859 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-16T14:55:01.859 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-10-16T14:55:01.860 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 +498us[ +498us] +/- 87ms 2023-10-16T14:55:01.860 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 22 +775us[ +814us] +/- 60ms 2023-10-16T14:55:01.860 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 -1411us[-1411us] +/- 82ms 2023-10-16T14:55:01.860 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-10-16T14:55:01.860 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T14:55:01.871 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T14:55:01.871 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T14:55:01.881 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T14:55:02.072 DEBUG:teuthology.orchestra.run.smithi186:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T14:55:02.252 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-16T14:55:02.356 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:55:02.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:02.434 DEBUG:teuthology.orchestra.run.smithi186:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:55:02.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:02.470 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-16T14:55:02.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:02.502 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T14:55:02.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T14:55:02.523 INFO:teuthology.task.internal:Duration was 2825.367710 seconds 2023-10-16T14:55:02.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T14:55:02.539 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T14:55:02.539 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T14:55:02.544 DEBUG:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T14:55:02.547 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T14:55:02.621 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:55:02.655 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:55:02.656 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-16T14:55:02.941 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T14:55:02.942 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2023-10-16T14:55:02.942 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T14:55:03.002 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi186.front.sepia.ceph.com 2023-10-16T14:55:03.003 DEBUG:teuthology.orchestra.run.smithi186:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T14:55:03.044 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2023-10-16T14:55:03.045 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T14:55:03.071 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T14:55:03.072 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T14:55:03.077 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T14:55:03.087 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T14:55:03.172 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T14:55:03.173 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T14:55:03.178 DEBUG:teuthology.orchestra.run.smithi186:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T14:55:03.180 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T14:55:03.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T14:55:03.357 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T14:55:03.357 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T14:55:03.426 DEBUG:teuthology.orchestra.run.smithi186:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T14:55:03.495 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T14:55:03.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T14:55:03.535 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.539 DEBUG:teuthology.orchestra.run.smithi186:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.542 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.573 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = core 2023-10-16T14:55:03.602 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2023-10-16T14:55:03.608 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2023-10-16T14:55:03.645 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:03.688 DEBUG:teuthology.orchestra.run.smithi186:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:03.704 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T14:55:03.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T14:55:03.720 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T14:55:03.731 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T14:55:03.732 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi178 2023-10-16T14:55:03.733 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T14:55:03.816 DEBUG:teuthology.misc:Transferring archived files from smithi186:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi186 2023-10-16T14:55:03.817 DEBUG:teuthology.orchestra.run.smithi186:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T14:55:03.856 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046/remote/smithi187 2023-10-16T14:55:03.857 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T14:55:03.898 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T14:55:03.898 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T14:55:03.903 DEBUG:teuthology.orchestra.run.smithi186:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T14:55:03.906 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T14:55:03.947 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T14:55:03.961 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T14:55:03.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T14:55:03.973 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T14:55:03.973 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T14:55:03.988 DEBUG:teuthology.orchestra.run.smithi186:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T14:55:03.990 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T14:55:04.004 INFO:teuthology.orchestra.run.smithi186.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 14:55 /home/ubuntu/cephtest 2023-10-16T14:55:04.006 INFO:teuthology.orchestra.run.smithi187.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 14:55 /home/ubuntu/cephtest 2023-10-16T14:55:04.026 INFO:teuthology.orchestra.run.smithi178.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 14:55 /home/ubuntu/cephtest 2023-10-16T14:55:04.030 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T14:55:04.043 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T14:55:04.136 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T14:55:04.163 DEBUG:teuthology.nuke:shortname: smithi178 2023-10-16T14:55:04.163 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T14:55:04.189 DEBUG:teuthology.nuke:shortname: smithi186 2023-10-16T14:55:04.190 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T14:55:04.195 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.685532', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:55:04.215 DEBUG:teuthology.nuke:shortname: smithi187 2023-10-16T14:55:04.216 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T14:55:04.221 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi186.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.687067', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:55:04.249 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-10-16_12:17:50-fs-rishabh-2023oct13-testing-default-smithi/7429046', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 13:58:46.688296', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-10-16T14:55:04.257 INFO:teuthology.orchestra.console.smithi178:Power off 2023-10-16T14:55:04.258 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T14:55:04.296 INFO:teuthology.orchestra.console.smithi186:Power off 2023-10-16T14:55:04.297 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T14:55:04.309 DEBUG:teuthology.orchestra.console.smithi178:power off output: Chassis Power Control: Down/Off 2023-10-16T14:55:04.310 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:04.320 DEBUG:teuthology.orchestra.console.smithi186:power off output: Chassis Power Control: Down/Off 2023-10-16T14:55:04.320 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:04.332 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-10-16T14:55:04.347 INFO:teuthology.orchestra.console.smithi187:Power off 2023-10-16T14:55:04.347 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T14:55:04.356 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-16T14:55:04.369 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2023-10-16T14:55:04.370 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:04.393 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-10-16T14:55:08.333 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:08.357 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:08.394 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:08.445 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is on 2023-10-16T14:55:08.468 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-16T14:55:08.505 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-10-16T14:55:12.448 DEBUG:teuthology.orchestra.console.smithi178:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:12.469 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:12.506 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:12.558 DEBUG:teuthology.orchestra.console.smithi178:check power output: Chassis Power is off 2023-10-16T14:55:12.580 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is on 2023-10-16T14:55:12.616 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2023-10-16T14:55:12.660 INFO:teuthology.orchestra.console.smithi178:Power off completed 2023-10-16T14:55:16.582 DEBUG:teuthology.orchestra.console.smithi186:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:16.618 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T14:55:16.693 DEBUG:teuthology.orchestra.console.smithi186:check power output: Chassis Power is off 2023-10-16T14:55:16.728 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2023-10-16T14:55:16.795 INFO:teuthology.orchestra.console.smithi186:Power off completed 2023-10-16T14:55:16.830 INFO:teuthology.orchestra.console.smithi187:Power off completed 2023-10-16T14:55:16.935 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 47672119149b103ec7cec2cc65c063ccbb001ac7 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/suites/blogbench}} duration: 2825.367710351944 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-10-16T14:55:16.936 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T14:55:17.008 INFO:teuthology.run:FAIL